The project development is suspended, parsers will fail to launch on current data
Hic et nunc is a new eco-friendly NFT marketplace, built on top of Tezos blockchain.
It is especially popular in generative graphics and data viz community, so I've decided to share data and all scripts that I've made for https://hashquine.github.io/hicetnunc rating.
It is published under CC BY license, so that it is even possible to sell NFTs that use that data (or modified scripts) as long as there is the following phrase somewhere in the token description: based on @hashquine dataset
.
Since hicetnunc servers are already under an extreme load due to quick growth, I've reorganized code, so that all data is taken from Tezos blockchain and IPFS without any calls to the hicetnunc.xyz website or API.
- Blockchain transactions by TzStats API (better-call.dev was not used in order not to interfere with hicetnunc backend).
- IPFS by cloudflare-ipfs.com and ipfs.io depending on mime type (same sources as in hicetnunc frontend).
- Wallet address owner metadata (name, Twitter etc.) from api.tzkt.io (same source as in hicetnunc frontend).
- Money data: list of all purchases, prices and commissions.
- All NFTs raw files, their previews and thumbnails, although 3d files and interactive SVG/HTML files are not yet processed properly.
- Authors metadata verified via tzkt.io like Twitter account address.
- Token transfers: list of changes of tokens owners including burns and direct transfers.
- All metadata available for tokens.
- Swaps and mints.
Data not available:
- Everything connected with hDAO tokens and hDAO feed. Although all related transactions are already being collected, they are not analysed yet.
- Twitter statistics like the number of followers.
- Direct money transfers between users, when NFT tokens are not transferred in the same transaction.
The goal was to simplify data analysis and visualization with a wide range of existing tools, so there are lots of redundant fields, which contain precalculated aggregations and different representations of the same data.
All files have two equivalent versions: JSON and CSV.
- JSON files are dictionary of dictionaries with rows of CSV files are indexed by the
*_id
field. - CSV files have commas as delimiters.
- Fields values are ether numbers or strings, empty values represented by
-1
or""
. - All identifiers are strings.
Any field, which references some event in the blockchain (for example, mint time) have 4 representations:
mint_iso_date
— string with UTC date and time:"2021-03-01T15:00:00Z"
,mint_stamp
— integer Unix timestamp in seconds:1614610800
,mint_hash
— string with transaction hash, where event occurred:"oom5Ju6X9nYpBCi..."
,mint_row_id
— integer with global unique operation id (internal to TzStats) with that event:42181049
Any field, which references a set of values (like the set of prices of sold works), have following aggregations:
sold_count
— values count,sold_nonzero_count
— number of positive values,sold_zero_count
— number of zeros,sold_price_min
— minimum value (excl. zeros),sold_price_max
— maximum value,sold_price_sum
— sum of values,sold_price_avg
— average value (sum divided by count excl. zeros).
tokens.json and tokens.csv — of all NFTs tokens
There is a confusing fact, that in hicetnunc each NFT can have multiple identical instances, which are fungible. In this document term "token" refers to the set of all that instances.
There are following invariants:
mint_count = author_owns_count + available_count + available_zero_count + other_own_count + burn_count author_sent_count <= other_own_count + available_count + available_zero_count
field | type | example | description |
---|---|---|---|
token_id |
string |
"152" |
Token identifier: numeric strings with OBJKT ids from hicetnunc.xyz/objkt url. |
issuer |
string |
"tz1UBZUk..." |
Address of the person, who intiated mint transaction and receives royalties. It is also referred to as token author in this dataset. |
mint_count |
integer |
100 |
How many token instances were minted. This number does not change: it is impossible to mint additional token instances in hicetnunc for now. |
mint_iso_date mint_stamp mint_hash mint_row_id |
string integer string integer |
"2021-03-01T15:00:03Z" 1614610803 "oom5Ju6X9nYpBCi..." 42181049 |
Mint event. |
artifact_ipfs |
string |
"Qma11k..." |
IPFS address of NFT content. |
artifact_mime |
string |
"image/gif" |
Mime type of artifact_ipfs . |
artifact_file_size |
integer |
2043418 |
File size of artifact_ipfs in bytes. |
artifact_preview_width |
integer |
1024 |
Preview image width of artifact_ipfs in pixels. Not more, than 1024. |
artifact_preview_height |
integer |
781 |
Preview image height of artifact_ipfs in pixels. Not more, than 1024. |
info_title |
string |
"Dali tower" |
Title of the NFT specified by creator. |
info_description |
string |
"generated by ..." |
Description of the NFT specified by creator. |
info_tags |
string |
"tag1 tag2" |
List of the NFT tags specified by creator delimited with space character. |
ban_status |
string |
"banned" |
If empty - token is ok, banned - token is banned, author_banned - token author is banned. Ban status is taken from hicetnunc repository |
Statistics fields | |||
author_sold_count author_sold_nonzero_count author_sold_zero_count author_sold_prices_min author_sold_prices_max author_sold_prices_sum author_sold_prices_avg |
integer integer float float float float |
100 10 1.25 2.5 33.7 1.325 |
Prices of sold works (in swaps) by author in XTZ. Note, that theoretically single NFT instance can be sold by author multiple times in case the buyer returns it back. |
secondary_sold_count secondary_sold_nonzero_count secondary_sold_zero_count secondary_sold_prices_min secondary_sold_prices_max secondary_sold_prices_sum secondary_sold_prices_avg |
integer integer float float float float |
100 10 1.25 2.5 33.7 1.325 |
Prices of sold works (in swaps) by other users in XTZ. |
available_count available_nonzero_count available_zero_count available_prices_min available_prices_max available_prices_sum available_prices_avg |
integer integer float float float float |
100 10 1.25 2.5 33.7 1.325 |
Prices of available works (in swaps) both from author and on secondary market in XTZ. |
burn_count |
integer |
15 |
How many token instances were burned by creator or other users. |
author_owns_count |
integer |
5 |
How many token instances are owned by author (not including available_count ). |
other_own_count |
integer |
95 |
How many token instances are owned by other users (not by burn, swap or author). |
author_sent_count |
integer |
5 |
How much instances were directly transferred by author to users without official swaps. |
Auxiliary fields | |||
info_ipfs |
string |
"Qma11k..." |
IPFS identifier of the NFT JSON metadata |
display_uri_ipfs |
string |
"Qma11k..." |
Token preview image identifier on IPFS or empty string. Note, that it is rare and only used for specific file types like HTML |
royalties |
float |
10.0 |
"Royalties" parameter passed during token mint in percent. |
info_creator |
string |
"tz1UBZUk..." |
Address of the token creator as specified in token meta JSON file. In 99.99% the same as issuer field, but sometimes empty probably due to some bug. |
mint_tokens_receiver |
string |
"tz1UBZUk..." |
Address of the user, who received tokens after mint transaction. In 99% the same as issuer field. |
mint_ah_row_id |
integer |
None |
Row id of mint operation by "Art house" contract (not by NFT contract). |
addrs.json and addrs.csv — of all hicetnunc users
All users, who ever created or owned NFT token.
field | type | example | description |
---|---|---|---|
address |
string |
"tz1UBZUk..." |
Address of the user in Tezos blockchain. Can be contract address as well. |
first_action_iso_date first_action_stamp first_action_hash first_action_row_id |
string integer string integer |
"2021-03-01T15:00:03Z" 1614610803 "oom5Ju6X9nYpBCi..." 42181049 |
When first transaction (related to hicetnunc) with this address occurred. |
tzkt_info_name |
string |
"dacosta works" |
User alias field according to tzkt.io metadata. |
tzkt_info_twitter |
string |
"dacosta_works" |
User Twitter account according to tzkt.io metadata. |
tzkt_info_email |
string |
"sartist@gmail.com" |
User e-mail according to tzkt.io metadata. |
tzkt_info_instagram |
string |
"dacosta_works" |
User Instagram account according to tzkt.io metadata. |
tzkt_info_site |
string |
"https://jsh.sh" |
User website according to tzkt.io metadata. |
tzkt_info_description |
string |
"NFT artist ..." |
User description according to tzkt.io metadata. |
tzkt_info_logo |
string |
"huson.png" |
User logo according to tzkt.io metadata. Add prefix services.tzkt.io/v1/avatars2/ to get the full url |
tzkt_info_github |
string |
"josim" |
User GitHub account according to tzkt.io metadata. |
tzkt_info_telegram |
string |
"thebaskia" |
User Telegram account according to tzkt.io metadata. |
tzkt_info_facebook |
string |
"barbeaucodeart" |
User Facebook account according to tzkt.io metadata. |
tzkt_info_reddit |
string |
"user/mathmakesart" |
User Reddit account according to tzkt.io metadata. |
ban_status |
string |
"banned" |
If empty - author is ok, banned - user is banned, some_tokens_banned - user is not banned, but it minted banned tokens. Ban status is taken from hicetnunc repository |
Statistics fields | |||
mint_count |
integer |
125 |
Number of minted works. |
bought_count bought_nonzero_count bought_zero_count bought_prices_min bought_prices_max bought_prices_sum bought_prices_avg |
integer integer float float float float |
100 10 1.25 2.5 33.7 1.325 |
Prices of bought works (in swaps) by this address in XTZ. |
author_sold_count author_sold_nonzero_count author_sold_zero_count author_sold_prices_min author_sold_prices_max author_sold_prices_sum author_sold_prices_avg |
integer integer float float float float |
100 10 1.25 2.5 33.7 1.325 |
Prices of sold works (in swaps) by author in XTZ. |
secondary_sold_count secondary_sold_nonzero_count secondary_sold_zero_count secondary_sold_prices_min secondary_sold_prices_max secondary_sold_prices_sum secondary_sold_prices_avg |
integer integer float float float float |
100 10 1.25 2.5 33.7 1.325 |
Prices of sold works on secondary market (i.e. swaps not by author) in XTZ. |
available_count available_nonzero_count available_zero_count available_prices_min available_prices_max available_prices_sum available_prices_avg |
integer integer float float float float |
100 10 1.25 2.5 33.7 1.325 |
Prices of available works (in swaps) both by author and other users in XTZ. |
in_op_count |
integer |
25 |
Number of incoming transactions |
out_op_count |
integer |
33 |
Number of outcoming transactions |
money_received |
float |
2.78999 |
Total volume of XTZ received by address in transactions related to hicetnunc |
money_sent |
float |
10 |
Total volume of XTZ sent by address in transactions related to hicetnunc |
sells.json and sells.csv — of all purchases via "official" hicetnunc swaps
There is the following invariant:
price * count = total_royalties + total_comission + total_seller_income
field | type | example | description |
---|---|---|---|
tr_iso_date tr_stamp tr_hash tr_row_id |
string integer string integer |
"2021-03-01T15:00:03Z" 1614610803 "oom5Ju6X9nYpBCi..." 42181049 |
When transaction occurred. |
token_id |
string |
"205" |
Numeric string with token identifier. |
count |
integer |
10 |
Number of token instances transferred. |
seller |
string |
"tz1UBZUk..." |
Seller address i.e. user who created the swap |
buyer |
string |
"tz1MwvWa..." |
Buyer address |
price |
float |
2.5 |
Price per item in XTZ (how much buyer payed per item) |
swap_id |
string |
"503" |
Numeric string with swap identifier. |
by_author |
boolean |
1 0 |
If 1 - swap was created by author, if 0 - by other user. |
author |
string |
"tz1UBZUk..." |
Address of token issuer, which receives royalties. |
Statistics fields | |||
total_royalties |
float |
12.75 |
Author income from royalties (already multiplied by count ) for secondary sells and 0 otherwise |
total_comission |
float |
1.3 |
Comission per item in XTZ. Usually 2.5% of price. |
total_seller_income |
float |
1.3 |
Seller income (already multiplied by count ). |
transfers.json and transfers.csv — all token transfers
field | type | example | description |
---|---|---|---|
tr_iso_date tr_stamp tr_hash tr_row_id |
string integer string integer |
"2021-03-01T15:00:03Z" 1614610803 "oom5Ju6X9nYpBCi..." 42181049 |
When transaction occurred. |
category |
string |
mint->author author->swap swap->author swap->user user->user author->burn user->burn user->ext ... |
String, which characterizes transfer in following terms: author , swap , user , burn , ext . This division is a convention used in this repository only. |
token_id |
string |
"207" |
Numeric string with token identifier. |
price |
float |
1.5 |
Price per item in XTZ or 0 for direct transfers. For external swaps price is guessed heuristically as half of sum of abolute values of transaction money transers (since each money transaction is counted twice both as incoming and outcoming). |
count |
integer |
5 |
Number of token instances transferred. |
swap_id |
string |
"523" |
Numeric string with swap identifier or empty string if it is a direct transfer. |
sender |
string |
"tz1UBZUk..." |
Sender address or empty string in case of mint |
receiver |
string |
"tz1MwvWa..." |
Receiver address |
method |
string |
"mint_OBJKT" |
Called method of hicetnunc or external contract, otherwise empty string |
swaps.json and swaps.csv — all "official" hicetnunc swaps ever created
field | type | example | description |
---|---|---|---|
swap_id |
string |
"503" |
Numeric string with swap identifier. |
token_id |
string |
"207" |
Numeric string with token identifier. |
price |
float |
1.5 |
Price per item in XTZ. |
total_count |
integer |
100 |
Total number of instances in swap. |
created_iso_date created_stamp created_hash created_row_id |
string integer string integer |
"2021-03-01T15:00:03Z" 1614610803 "oom5Ju6X9nYpBCi..." 42181049 |
When swap was created. |
closed_iso_date closed_stamp closed_hash closed_row_id |
string integer string integer |
"2021-03-01T15:00:03Z" 1614610803 "oom5Ju6X9nYpBCi..." 42181049 |
When swap was closed (cancelled). If closed, swap cannot be reopened again. |
by_author |
boolean |
1 0 |
If 1 - swap was created by author, if 0 - by other user. |
created_by |
string |
"tz1UBZUk..." |
Address of the user, which created swap. |
Statistics fields | |||
sold_count |
integer |
99 |
Number of sold (including with zero price) instances. |
available_count |
integer |
1 |
Number of available instances if the swap is not closed, 0 otherwise. |
returned_count |
integer |
0 |
Number of unsold instances returned to the swap creator if the swap was closed, 0 otherwise. |
sold_price_sum |
float |
148.5 |
Total price of sold instances in XTZ. |
There are 3 Tezos addresses, which are common to most of hicetnunc transactions:
- "NFT" contract: KT1RJ6PbjHpwc3M5rw5s2Nbmefwbuwbdxton.
- This is the registry of all owners of all NFT tokens.
- This is the typical way how most of NFTs works on Ethereum or on Tezos.
- Click on "Bigmap #511" tab in Tezos explorer to see registry of token owners.
- This contract is the single "source of truth" about current owners of all NFT tokens issued by hicetnunc. If there is no information about token owner in the registry, than that person doesn't own any tokens.
- This contract also the registry of tokens infos (metadata).
- Token info is a small JSON structure stored on on IPFS.
- Here is an example of such structure.
- It contains link to IPFS with NFT binary contents (some image, for example).
- It also contains title, description, creator and tags.
- It does not contain price or related information.
- Only link to IPFS is stored on the blockchain.
- Note, that, however, there is no way to alter token metadata after minting.
- Click on "Bigmap #514" tab in Tezos explorer to see mapping from tokens to IPFS urls.
- Token info is a small JSON structure stored on on IPFS.
- Every token owner can call "transfer" method of the contract to send tokens to other address.
- This contract can't do any money related operations. Money logic should be implemented in other contracts, which call "NFT" contract as a part of transaction operation.
- There is also a "mint" method in this contract, but it can only be called by the "Art house" contract.
- This is the registry of all owners of all NFT tokens.
- "Art house" contract: KT1Hkg5qeNhfwpKW4fXvq7HGZB9z2EnmCCA9.
- This contract implements money related operations on hicetnunc.
- It's main structure is a swap. It is some amount of tokens, which are available of sale for specific price.
- Click on "Bigmap #523" tab in Tezos explorer to see all current swaps.
- Note, that there may be other contracts implementing swap mechanism. These contracts may decide not to pay comission or royalties.
- Objects can be minted only with this contract by calling "mint_OBJKT" method.
- This contract keeps track of royalties and assigns tokens ids.
- Comission wallet: tz1UBZUkXpKGhYsP5KtzDNqLLchwF4uHrGjw.
- 2.5% of every purchase via "Art house" contract swaps is sent to this wallet.
There are several other contracts related to curation and hDAO mechanisms, which are independent from the contracts mentioned above.
Actually, hicetnunc was not created just as another NFT marketplace, it has much broader mission as hDAO (hicetnunc DAO). You can get the idea of the creators vision on hicetnunc blog. As a result, only a small subset of contract's logic is actually used during hicetnunc website operation.
-
Official swap mechanism by hicetnunc "Art house" contract.
- Any token bought on hicetnunc website is a part of some swap.
- Swap is just some amount of tokens, which are offered for sale by specific price.
- When swap is created, the seller sends all offered tokens to the "Art house" smart contract.
- Then anybody can send the required amount of money to the "collect" method of contract and get tokens in return.
- 2.5% of comission is transferred to hicetnunc comission wallet.
- 10% of royalties (this parameter is configurable in general) is transferred to token author.
- rest of money is sent to swap creator.
- proportional amount of hDAO tokens are also sent to buyer, seller, token author and comission wallet.
- Seller can cancel swap any time and get unsold tokens back.
- Swaps can be created by any token owner any number of times.
- In this dataset official swaps are treated as
author->swap
,other->swap
,swap->author
,swap->other
transfers.
-
External swap mechanisms.
- Since "transfer" method of "NFT" contract can be called by any token owner directly, it is possible to make custom smart contracts, which implement any desired logic.
- These custom contracts are not required not pay comission or royalties to hicetnunc.
- In general, swap contracts can be used to exchange any types entities.
- Example: https://quipuswap.com/swap
- In this dataset external swaps are treated as
other->other
transfers. The related price is guessed heuristically (as half of money transferred in all operations) and may not be always correct.
In contrast to NFT definition, each NFT artwork in hicetnunc can have multiple copies, which are fungible. The NFT contract only tracks the amount of copies owned by each address. This means, that there is no way (even in theory) to track history of single copy like it can be done on OpenSea, for example.
It is possible, however, to track history of token groups to some extent. Here is a list of possible owner types in this dataset:
author
— the person, who created the tokens during mint.user
— any other hicetnunc user.ext
— any external contract (external swap mechanism, for example).burn
— reserved address for burning tokens.swap
— when tokens are offered on sale in official swaps.
List of possible transitions:
mint->author
,mint->user
First, every token should be minted.- For each token type there may exist only single mint operation. It is impossible to mint additional tokens later.
- The only way to mint a token is to call "mint_OBJKT" method in "Art house" contract.
- Here is a typical mint transaction
- Internally it calls "mint" method in "NFT" contract.
- In dataset the sender is empty for mint operations.
- As result of mint operation, all tokens are transferred to some address. In 99% of cases this is the transaction sender, but sometimes it is different.
- Royalties are always sent to the mint transaction sender.
author->swap
,user->swap
Any token owner can create official swap.- Hicetnunc swap is created by calling "swap" method in "Art house" contract.
- Here is a typical swap creation transaction by author.
- Internally tokens are transferred to the "Art house" address.
swap->author
,swap->other
There are two situations, when tokens may be transferred from a swap.- Purchase
- When token is purchased on hicetnunc website, it is transferred to the buyer. This is the main operation on hicetnunc.
- Buyer should call "collect" method of "Art house" contract and send required amount of money with it.
- Here is an example of "collect" transaction.
- First 3 internal operations send money to token creator (royalties), hicetnunc wallet (comission) and to the seller (which is the same as token creator in some cases) in that order.
- Fourth operation creates hDAO tokens and sends them to the buyer, seller and hicetnunc wallet. These tokens have special meaning and are not tracked in this dataset.
- Last internal operation does the actual token transfer.
- Note, that case of zero price is handled differently.
- Here is an example of purchase with zero price.
- Swap cancel
- When swap creator decides to cancel swap, all remaining tokens are transferred back to him.
- Purchase
author->user
,user->user
Any token owner can transfer tokens directly to other users for free by calling "transfer" method of "NFT" contract.- Here is an example of direct transfer transaction from author to other user.
author->ext
,user->ext
,ext->user
,ext->author
— external swaps- Here is an example of external swap.
author->burn
,other->burn
Any token owner can transfer tokens to burn addresstz1burnburnburnburnburnburnburjAYjjX
.- Here is and example of burn transfer from author.
- Tokens can never be transferred from burn address since it is impossible to retrieve its private key (similar to how it is impossible to reverse hash containing all zeros).
mint_sender
The address of the sender of the "mint" transaction.- This is the person, who receives royalties in hicetnunc.
- In this dataset token author this is equivalent to token author.
issuer
The address of the receiver of tokens after the mint transaction.- It is also the first parameter of the "mint" call in "Art house" contract.
- Here is an example of mint, where transaction sender and token issuer are different.
info_creator
Field "creators" in JSON in token metadata.- Here is an example of mint, where metadata creator field is different from transaction sender and issuer.
- As of 4th of April, it always has single entry.
- Sometimes it is empty.
- Here is an example of mint with empty metadata creator field.
- Note, that corresponding token page has a bug, that it shows token owner controls on token page.
- Any user can send any NFT tokens to "NFT" or "Art house contract"
- Technically, it has the same effect as sending this tokens to burn address, since contracts were not programmed to send their own NFTs (except from swap mechanism) under any circumstances.
- Comission wallet sometimes mint NFTs and buys them from other users.
- Since it is not a contract and manipulated by a real person (hicetnunc creator).
- https://tzstats.com/opYoTN4LUvNq7F5oiq7a6frW1Q5UubQuXSrDmrqrJGJzHnKaJQQ/47335724
- https://tzstats.com/oo7Sr2daVXamKTDtajRRvJGmTmYAantjJeLoPkTD6hraVHXQR1B/47335862
Note, that the code is still experimental and may require substantial changes to make it run. But you can read introduction in HACKING.md