The Approach
Last updated
Last updated
Phase 1: Data Preparation and Model Construction
During the initial phase of our NFT price evaluation tool development, which extended until May, TeaDAO focused on data preparation and model construction. In this stage, we aimed to gather comprehensive data on NFTs, including their traits and transaction volumes across various marketplaces.
Data Preparation:
To ensure the availability of diverse and reliable data, TeaDAO implemented a web crawling process to extract NFT information from multiple sources, such as on-chain data and popular marketplaces like: Foundation, LarvaLabs Contract, LooksRare, NFTX, OpenSea, Rarible, and SuperRare. This extensive data collection allowed us to compile a rich dataset encompassing a wide range of NFTs from different collections.
NFT's Traits: One crucial aspect of data preparation involved analyzing NFT traits to identify their impact on the asset's price. To accomplish this, we utilized data from OpenSea and visually mapped NFT prices corresponding to specific traits. By observing the relationships between trait values and prices, we were able to identify and exclude traits that exhibited minimal influence on NFT valuation. For instance, in the case of the NFT collection "Otherdeeds," we discovered that traits like "Environment" and "Eastern" had limited significance in determining the NFT's price.
Transaction Volumes: To supplement the data from OpenSea, which primarily provided transaction histories for NFTs, we incorporated Dune to access transaction volumes from a diverse range of marketplaces. By combining data from various sources, including Foundation, LarvaLabs Contract, LooksRare, NFTX, OpenSea, Rarible, and SuperRare, we obtained a comprehensive understanding of NFT price trends and transaction dynamics.
Model Construction:
With a well-prepared dataset in hand, TeaDAO proceeded to construct the NFT price prediction models. We adopted a standard approach, dividing the dataset into two parts: 80% for training and 20% for testing. To explore different modeling techniques, we employed two regression models: linear regression and regression based on k-nearest neighbors.
Test and Evaluate the Results:
Following model construction, we subjected our AI models to rigorous testing and evaluation. The performance of the models was assessed using the Mean Absolute Percentage Error (MAPE) metric, which measures the accuracy of predictions. Unfortunately, the results did not meet our expectations, with the overall accuracy hovering around 65%.
Lessons Learned:
The first phase of our NFT price evaluation tool development provided valuable insights into the challenges and complexities of predicting NFT prices with AI models. We identified several key lessons that will inform our approach in the subsequent phases:
Model Complexity: The presence of projects with a multitude of traits rendered the AI model cumbersome and difficult to handle. Addressing this complexity will be crucial in enhancing the model's efficiency and performance.
Heterogeneity of NFT Traits: The wide variation in traits across different projects presented a significant obstacle to scalability. The need to account for diverse traits in the model requires further attention and optimization.
Model Diversity: Our preliminary efforts with two regression models highlighted the importance of expanding our model repertoire. Integrating a diverse set of AI models and techniques may yield more accurate predictions across a broader spectrum of NFT projects.
Phase 2: Compact Parameters and Enhanced Model Tuning
In Phase 1, we encountered challenges when attempting to input every aspect of an NFT into the pricing model, leading us to explore a new and improved methodology. During Phase 2, we successfully addressed these issues by compacting NFT information into four main parameters: rarityScore, similarityScore, reputationScore, and utilityScore. As a result, our AI-driven model exhibited better tuning, leading to significantly improved results with an accuracy of approximately 83%.
RarityScore and UtilityScore:
Recognizing the complexity of handling diverse traits across different NFT projects, we focused on two key parameters to better reflect the impact of traits and determine NFT prices accurately:
RarityScore: To evaluate the scarcity and uniqueness of each NFT, we calculated rarity scores for the Otherdeed NFT collection. An example of rarity scores for 100,000 NFT items in the collection can be accessed at the following link:
UtilityScore: The utilityScore for GameFi projects, such as Axie Infinity, required a project-specific approach. For instance, in Axie Infinity, the utilityScore is derived from variables like "health," "defense," "attack," and "cards." Considering these unique aspects enabled a more accurate assessment of the practical usefulness and functionality of NFTs.
Enriching Data with SimilarityScore:
The challenge of low transaction volume for certain NFTs, such as the Otherdeeds collection, was addressed by employing similarityScore. To enrich the dataset, we tracked and analyzed prices of comparable NFTs that had been sold. The Jaccard Similarity function was employed to measure the degree of similarity between NFTs, allowing us to determine appropriate prices for NFTs with limited transaction history.
Impact of SimilarityScore:
Applying similarityScore with a threshold of 0.9 for the Otherdeeds collection resulted in a substantial increase in the number of NFT transactions. The transaction volume rose from approximately 30,000 to around 45,000, providing a more comprehensive and informative dataset for evaluation.
Expanded Model Testing:
In addition to the models used in Phase 1, we introduced several new models to enhance the predictive accuracy of our NFT price evaluation tool. The models we tested included:
DecisionTreeRegressor: Scikit-learn DecisionTreeRegressor Documentation
RandomForestRegressor: Scikit-learn RandomForestRegressor Documentation
BaggingRegressor: Scikit-learn BaggingRegressor Documentation
Among these models, RandomForestRegressor demonstrated the best results, achieving a remarkable Mean Absolute Percentage Error (MAPE) of 0.042034438274596166.
Continued Testing and Evaluation:
Throughout Phase 2, we maintained the same rigorous testing and evaluation approach from Phase 1. The continuous improvement and fine-tuning of our AI-driven models led to the attainment of significantly improved results. Phase 2 represented a pivotal milestone in our research, yielding the best predictive accuracy achieved thus far.
