Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
AstraAI is a collaborative data marketplace and the infrastructure provider for data ecosystems. It brings together the power of community-driven data analysis with the datasets of some of the most successful modern businesses. The marketplace allows both grassroots data enthusiasts, SMEs and institutional partners to access new data sources as well as ways to put your own untapped data to use. This allows them to gain new insights, find new revenue streams within the boundaries of regulations as well as stay within the framework of their own data governance.
Our vision is to fuel the new data economy. We provide an easy-to-use platform for sharing, requesting and understanding data. This uncovers key insights, improves processes, and opens up new business opportunities while supporting next-generation Large Language Models and AI. All supported by a fair system that rewards those who publish or contribute data while charging those who consume data.
Bringing together the power of artificial intelligence and blockchain, AstraAI will accelerate the tokenization of real-world assets (RWA). It will facilitate new avenues for the vast amounts of data these assets will rely on for their valuation and provide the infrastructure for this data to be utilized, monetized, and enhanced by other players.
By developing the AstraAI data ecosystem on its own public blockchain, along with the use of subnet technology for creating private data consortiums AstraAI is able to provide the benefits of trustless access and sharing of data to the fragmented data landscape. At the same time, by empowering Decentralized Physical Infrastructure Network (DePIN) with standardized data sources AstraAI will enable companies to tap into the data ecosystem produced by the emerging Economy of Things (EoT) industry.
AstraAI, as its name suggests, is the nucleus, the core of the new data economy. It will unify the scattered segments of the data landscape, further enhancing and advancing the capabilities of next-generation large language models (LLMs) and artificial intelligence (AI).
If you want to learn more about our network and ecosystem, read our AstraAI network econonmics.
Contributing data on AstraAI’s public data marketplace is an important part of AstraAI's data ecosystem, opening your profile up for new business collaborations and revenue opportunities as other AstraAI members subscribe to your datasets.
In this user guide, we will show you how to publish your first dataset.
When you arrive on the AstraAI Marketplace, you will see a button, Upload Dataset. When you click this button, you will need to choose between two options:
Dataset: Choose this option if you want to upload a dataset with you as the sole contributor.
Community Dataset: Choose this option if you want to upload a dataset where contributions are open to any user.
Once you have chosen your desired dataset, you will be asked to fill in a title for the dataset. The title can still be changed at a later time. You will be asked to provide a file up to 2GB in size. If you have larger data files that you would like to publish.
CSV
JSON
XML
Parquet
When you click the Upload button, the file will be processed. This can take several minutes. You can view your dataset on the My Datasets page. When the status updates from “processing” to “draft”, your dataset has been processed successfully. We now recommend you query your dataset via the user interface by clicking on the Query button (rocket icon) to check if all data has been correctly processed in the way you intended.
Once your dataset is in draft, you have 14 days to finalize the publication of the dataset. In this phase, you are still able to delete the dataset. After publishing, you will not be able to delete the dataset anymore. If you do wish to remove the dataset from the AstraAI data marketplace.
When you click on the Edit button (pencil icon), you will have several options:
Edit Dataset Information: Here you can add a description of your dataset, update the title, and provide additional information about your dataset.
Edit Metadata: Here you can enrich the metadata of your dataset by adding tags and descriptions to the metadata.
To continue publishing your dataset, you will need to add some information to your dataset. You do this by clicking on the Edit button underneath the dataset information card.
Dataset Name (mandatory): Title of the dataset that will be displayed on the AstraAI data marketplace.
Dataset Sample (optional): Choose if you want the user to see a sample of your dataset. A sample consists of 5 records within your dataset.
The last step is to choose a license under which you want to publish your dataset. If you have a commercial license for your dataset, you can choose to create a Custom License. You have to create this license only once; it will be accessible for you for future usage.
Abbreviation: Create a simple two to four letter abbreviation of your license.
Full License Name: Write out the full name of your license.
To continue, click the Save button. Congratulations, you are now one step closer to publishing your dataset.
Before publishing, we recommend you enrich the metadata of your dataset to provide more context about the data within your dataset. Please follow the following user guide to enrich your metadata. In this guide, you will also learn how to delete certain fields from your dataset before publishing. You may want to delete some fields if these fields contain sensitive information.
To finalize the publishing of your dataset, click on the Publish button on the “Pricing & Publishing” card in order to finalize and publish the dataset to the AstraAI data marketplace.
You will now have to choose the base price of your dataset. The base price is the amount of $USDC.e you would ask for subscribing for one day. If a user subscribes for 30 days, the price of the subscription will be 30 x the base price.
In the overview on the right, you will be able to see an example calculation, exactly showcasing how much you earn from a subscription.
Since AstraAI is a collaborative marketplace, you can decide exactly how the revenue is distributed amongst contributors to your dataset.
Platform Fee: The fee AstraAI charges for the usage of the platform. If you upload the data to the AstraAI data environment, you’ll be charged a 35% management fee. If you host the data yourself (coming soon), AstraAI will charge you a 15% management fee.
Management Fee (Community Datasets only): The fee you will receive for the management of the community dataset, such as updating the information of the dataset and promoting the dataset. The maximum fee that can be set is 50%.
Data Revenue Share: This fee is split amongst the providers of the data (in several parts in the case of a community dataset).
When you fill in all the fields, you can click the Save button. MetaMask will now pop up and ask you to sign two transactions. Please do not refresh your page.
When you sign the transactions, a dataset NFT is minted. You will need some AVAX in your wallet to pay for the gas fees for minting the dataset NFT. When you want to transfer the ownership of the dataset, you will need to send your dataset NFT to another wallet. The ownership of the dataset will be automatically updated on the platform to the new wallet.
Great news! Anyone can start using the AstraAI marketplace! 🌟
However, to make the most of the AstraAI platform, certain skills will be invaluable:
Understand the Basics of Data: Gain a fundamental understanding of data and metadata enrichment.
Data Analysis Skills: Have basic data analysis skills to interpret and work with data effectively.
Collaborative Approach: Engage in community-driven data enrichment processes.
Blockchain Fundamentals: Acquire a basic understanding of blockchain technology and its role in decentralized ecosystems.
Digital Wallets: Learn about digital wallets, including their setup, use, and security measures in a blockchain context.
Creating a digital wallet is a crucial step for participating within AstraAI marketplace. To do this, you can select a reputable wallet service like MetaMask.
The process involves downloading the wallet application, setting it up with secure login credentials, and safely storing your private keys. For a comprehensive tutorial on setting up and using MetaMask, you can visit the official MetaMask documentation . This guide will provide step-by-step instructions to ensure a smooth and secure wallet setup.
Once you have your wallet up and running, you will need a couple of things:
Switch your network to the Avalanch network.
Have some AVAX in your wallet to pay for gas fees on the Avalanche network.
Have $USDC in your wallet on the Avalanche C-chain network.
And, you’re good to go!
Enriching the metadata of a dataset is a central part of the AstraAI data marketplace. By deeply enriching the metadata of a dataset you provide context about the data to users and artificial intelligence, and makes use of this data.
Simply put, this ensures that data becomes more useful for other users, which increases the chances of more people subscribing to the dataset. By allowing multiple contributors to enrich the metadata of community datasets, the quality of the data improves for all of its contributors and increases the number of their rewards.
This user guide will guide you through the steps for metadata enrichment to community datasets. If you wish to contribute data to community datasets, follow the guide on data contributions.
When you visit the AstraAI data marketplace and browse datasets, you will encounter a section within the details of the dataset that allows you to either contribute metadata or contribute data.
Before you decide if it's worth contributing metadata to the dataset, you need to know how much revenue the community dataset manager (initial publisher) charges for the management of the community dataset. If you would like to contribute metadata you should look to the Metadata shares percentage. This is the percentage of revenue that will be distributed between all metadata contributors.
If, for example, a subscription of $1000 is sold and metadata contributors get 10% of all revenue, then $100 is distributed among all metadata contributors. Let's say that Metadata Contributor 1, contributed metadata and Metadata Contributor 2 contributed metadata as well. Then Contributor 1 will earn 50% of $100 and Contributor 2 will earn 50% of $100.
When you can’t find the data you are looking for in AstraAI's data marketplace, you can ask for this specific data by making a request on the bulletin board.
To make a request click the Create Request button. A new screen will appear where you will be asked to fill in more information about your data request.
Title: the title of your request
Category: what data category does the request belong to
Once you want to start the process of contributing metadata enrichments, you click on the Contribute Metadata button.
You will see two tabs:
Schema, where you will be able to view the schema, the existing metadata and a way to edit the metadata of the dataset.
View Sample: a few sample records so you can get a grasp of the data within the dataset in order to help you improve the metadata.
In order to add a tag to the metadata, click the three dots [...] next to tags. You will now be asked to add tags to the column within the dataset. Tags are meant to provide an alternative definition of the column you are enriching.
For example, the column "City" can be enriched with tags such as: Town, Capital City, Principal City, Metropolis. These are all alternative definitions for the column "City". You add a tag by typing and then hitting Enter on your keyboard to confirm. To save the tags, click on the blue tick to save or on the red cross to cancel. You will now be able to view the tags that you have added to the column.
In order to add a description to the column, you click the three dots [...] next to the description. You are now able to provide a deeper context about the column.
For example, the column "City" only has capital cities and refers to the capital cities where holiday accommodations are based. So we can add this as a description to our metadata. When you want to save the description, click on the blue tick to save or the red cross to cancel. You are now able to view the description that you have added to the column.
Repeat this process for as many columns as you wish within the dataset. After you have completed your contribution, you should write a message to the publisher of the dataset. It is recommended that you clearly explain the changes you have applied to the metadata, as this increases the chance of your contribution being accepted.
Click the Send request button and sign the transaction in your wallet to finalize the request.
Congratulations, a request to enrich the metadata has now been sent to the dataset publisher. You can now view your pending request in the Contribution Request page.
The dataset publisher will now receive a notification to review your request for contribution to the metadata of the dataset.
They are able to see:
Schema: the current schema of the dataset, including the already existing tags and descriptions.
Updated schema: the changes you have made to the metadata and if the request is accepted they will be the new contents of the dataset.
Publishers can either choose to accept or reject your request. If they decide to reject the request it is mandatory for them to write a message and give a reason why the request was rejected. If they accept the request a message is optional. The request is finalized when the publisher signs a transaction in your wallet.This is also the moment when the metadata of the dataset is updated.
You will receive a notification if the publisher accepted or rejected your request. In the Contribution Request page you will now see that the status of your request is either "approved" or "rejected".
If your request was approved you are now earning revenue with your contributed metadata. In the My Datasets page you will be able to claim any revenue that is generated.
Congratulations, by following this guide you have contributed a metadata enrichment to a dataset and are now eligible to earn rewards on it.
Price Range: what would be the minimum and the maximum price in $USDC you would be willing to pay on a monthly basis to get access to the data you are requesting.
When you have provided this information you are now able to publish your request. If you still want to edit your request afterwards, you can click the Edit request button (pencil icon) underneath your request. If you want to delete your request, click the Delete request button (bin).
Other users are now able to reply to your request for data, if the suggested data is to your satisfaction, you can archive your request by clicking the Solve request button (pin icon). The status of your request will now change to ‘Solved’. You can always reopen the request again by clicking the Reopen request button (pin icon).
Provide link to dataset (optional): this is a link of a dataset that is published on our data marketplace.
Message: write a message as a reply to the request, you can use this field to write a comment or ask follow up questions on the data request.
By clicking the comment button, you will reply to the request. You can edit your comment by clicking the Edit comment button (pencil icon), you are also able to remove your comment by clicking the Delete comment button (trash icon).
Congratulations you have now successfully published your first data request and also replied to a data request of another user.
Getting access to data on AstraAI's public data marketplace is a simple and straightforward process. You'll be able choose exactly how long you wish to subscribe to the data and once you've subscribed you can query the data via our user interface or public API.
In this user guide, we will show you how to subscribe to your first dataset.
When you arrive on the AstraAImarketplace, you will see several datasets listed. When you click on one of these datasets, you will get more information about that dataset.
Before you decide to make a purchase, you can view some essential information about the dataset:
The first thing you might want to do is to download a sample of the dataset by clicking the Download sample button. Dataset publishers have the option to make a sample available of their dataset, this sample can be downloaded for further analysis and is made available for you in JSON format. You can open this sample for further inspection with your preferred code editor.
Other information you might want to review before making a purchase:
Dataset Rating: how other users rate this dataset, subscriptions have rated the dataset.
Subscriptions: the amount of subscriptions that have been sold for this dataset
Published: the date that this dataset has become public.
Update Frequency: the commitment of the data publisher means how often they will update the dataset with new data.
Total Updates: how many updates have been made to the data of the dataset
Last Update: the date of the latest update to the data of the dataset
License: the license regarding the usage of this data is important to view. Some data publishers might want you to use the data for certain goals.
Once you are ready to make a purchase, you can click the Subscribe button. You can either type the amount of days you would like to subscribe to this dataset, or you can use the slider to adjust the amount of days. The price updates automatically so you can see how much this subscription will cost you.
You may see a button Approve USDC.e. This means you first need to approve the USDC.e you are willing to spend on the subscription before making the purchase. You approve the USDC.e by signing the transaction in your wallet.
Now the button Subscribe should be visible, when you click this button you will be asked to sign another transaction in your wallet. A confirmation message appears once you have subscribed successfully.
You will now be able to see your subscription in your subscriptions panel. If you click the renew button, you can either renew an expired subscription or extend an existing subscription.
Once you subscribed to the dataset, you are now able to query the dataset. You have two options to query the dataset, either through the user interface or through our public API. In this userguide we will discuss how to query the data through the user interface.
To query the dataset through the user interface, click the query button (rocket icon), you will now see 3 tabs:
Query: you will now see an example SQL query prefilled. This query is designed to query the first 5 records out of the dataset.
Tree view: here the results of the query are displayed once the query is executed.
Metadata view: View the metadata of the dataset, the metadata can provide you in depth contextual information about the contents of the dataset.
To execute the SQL query click on the query button (rocket icon). Once the query is executed you will be able to view the result of your query in the "Tree view". A maximum of 1000 records are displayed, if you wish to query more than 1000 records, you will be required to use our public API.
In order to reset your SQL query to the default query click the reset button (trash icon).
Congratulations, you've successfully subscribed to your first dataset and have also executed a first query!
Community datasets are designed in such a way that other users can contribute to these datasets. This creates a dataset that is rich with data from multiple contributors, who are all fairly rewarded for their efforts.
This user guide will guide you through the steps for contributing data to community datasets. If you wish to contribute metadata to some datasets and community datasets, follow the guide on metadata contributions.
When you visit the AstraAI data marketplace and browse datasets, you will encounter a section within the details of the dataset that allows you to either contribute metadata or contribute data.
Before you decide if it's worth contributing to the dataset, you need to know how much revenue the community dataset manager (initial publisher) charges for the management of the community dataset. If you would like to contribute data you should look to the Data shares percentage. This is the percentage of revenue that will be distributed between all data contributors.
If for example a subscription of $1000 is sold and data contributors get 25% of all revenue, then $250 is distributed among all data contributors. Let's say that data contributor 1, contributed data and data contributor 2 contributed data as well. Then Contributor 1 will earn 50% of $250 and Contributor 2 will earn 50% of $250.
Once you want to start the process of contributing data to the dataset, you click on the Contribute Data button.
You will see two tabs:
Schema: where you will be able to view the schema of the dataset, including the datatype of each column.
View Sample: a few sample records so you can get a grasp of the data within the dataset.
In the right corner, you'll see a section where you can upload a file, the data you want to contribute. Be aware that datasets might have format restrictions and will only accept a certain format such as CSV.
While your data is being uploaded and processed you can take your time to write a message to the community dataset manager. We recommend you to take the time to write a clear message to the community dataset manager, so they get a better understanding of the data you would like to contribute. This increases the chance of your data being accepted and added to the dataset.
You can click the Send a request button to send your request. You will now be asked to sign a transaction in your wallet.
Congratulations, a request to add your data has now been sent to the community dataset manager. You can now view your pending request in the Contribution Request page.
The community dataset manager will now receive a notification to review your request. They are able to see a sample of the data you submitted and will also be able to read the message you sent with the request.
They can either choose to accept or reject your request. If they decide to reject the request it is mandatory for them to write a message and give a reason why the request was rejected. If they accept the request a message is optional. The request is finalized when the community dataset manager signs a transaction in Metamask. This is also the moment when the data is added to the dataset.
You will receive a notification if the community dataset accepted or rejected your request. In the Contribution Request page you will now see that the status of your request is either "approved" or "rejected".
If your request was approved you are now earning revenue with your contributed data. In the My Datasets page you will be able to claim any revenue that is generated.
Congratulations, by following this guide you have contributed data to a community dataset and are now eligible to earn revenue on it.
Enriching the metadata of a dataset is a central part of the AstraAI data marketplace. By deeply enriching the metadata of your dataset, you provide context about the data to users and artificial intelligence, and makes use of this data. Simply put, this ensures your data becomes more useful for other users, which increases the chances even more people will subscribe to your datasets.
This user guide will guide you through the steps of enriching the metadata of your dataset.
To start enriching the metadata of your dataset, you will need to have a datasets in the status "draft" or "published" on the AstraAI data marketplace. Follow this guide on publishing data if you haven't published a dataset yet.
When you want to start enriching the metadata of your dataset, go to the my datasets page and click the edit dataset button (pencil icon).
Click the edit button on the "Metadata" card, you will now see two tabs:
Tree view: here you can start editing the metadata of your dataset, by adding tags and descriptions to each column of your dataset. You will also be able to delete certain fields from your dataset if you did not publish the dataset yet.
View Sample: view a sample of the data from your dataset to understand the context of the data in detail.
When your dataset is still in the status "draft" you will be able to still delete columns from your dataset before publishing. The delete button (trash bin icon) will be red when you are able to delete the column and gray when you won't be able to delete the column anymore.
Click the delete button (trash bin icon) and click confirm to delete the column from your dataset. The update is now being processed and can take a while. This action is irreversible, if you deleted a field by mistake we recommend you to delete the complete dataset and start the upload process again.
To add a tag to the metadata, click the three dots [...] next to tags. You will now be asked to add tags to the column within the dataset. Tags are meant to provide an alternative definition of the column you are enriching.
For example, the column "City" can be enriched with tags such as: Town, Capital City, Principal City, Metropolis. These are all alternative definitions for the column "City". You add a tag by typing and then hitting Enter on your keyboard to confirm. To save the tags, click on the blue tick to save or on the red cross to cancel. You will now be able to view the tags that you have added to the column.
To also add a description to the column, you click the three dots [...] next to description. You are now able to provide a deeper context about the column.
For example, the column "City" only has capital cities and refers to the capital cities where holiday accommodations are based. So we can add this as a description to our metadata. When you want to save the description, click on the blue tick to save or the red cross to cancel. You are now able to view the description that you have added to the column.
Repeat this process for each column in your dataset.
Congratulations! You have now successfully enriched the metadata of your dataset.
This section will guide you through the installation and initial configuration of astraivm, providing a step-by-step approach to setting up the environment and configuring the necessary parameters to get the Virtual Machine running on your local system or in a production setting.
Installation
Before you begin, ensure that your system meets the following prerequisites:
Operating System:
Linux (Ubuntu 20.04 recommended)
MacOS
Windows Subsystem for Linux (WSL2)
Memory: At least 8 GB of RAM
Storage: At least 10 GB of free disk space
Step 1: Clone the Repository
Start by cloning the astraivm repository from GitHub to your local machine. Open a terminal window and run the following command:
Copy
Checkoutv0.1.0 git tag to use the stable version
Checkout main branch to use the latest version
Step 2: Build the Project
After cloning the repository, build the astraivm project to compile the source code into executable binaries:
Copy
This script will handle the compilation of all components required for astraivm to run.
Running astraivm
To start astraivm, use the following command:
Copy
You should see output indicating that the VM is initializing, two subnets running the astraivm , and 5 validators running on each subnet. Once it starts, astraivm will begin processing transactions and blocks according to the rules defined in our run.sh file.
git clone https://github.com/Astrai/astraivm.git
cd astraivm./scripts/build.sh./scripts/run.shHow to Contribute
Contributing to astraivm is an opportunity to enhance the functionality of the system, fix bugs, improve documentation, and participate in the broader development community. Whether you're fixing a bug or adding a new feature, here's how you can contribute to making astraivm better.
Step 1: Find an Issue
Choose an issue that interests you and matches your skills. If you have a new idea or a bug you've discovered, please create a new issue, ensuring it has not already been reported or addressed.
Step 2: Fork and Clone the Repository
Fork the astraivm repository to your GitHub account and clone it locally to make your changes.
Copy
Step 3: Set Up Your Development Environment
Ensure your development environment is set up according to the setup instructions in the repository's README. This may involve installing certain software dependencies, setting up virtual environments, or ensuring compatibility with your local development tools.
Step 4: Create a Feature Branch
Always work on a new branch for your changes:
Copy
Step 5: Make Your Changes
Make the changes in the codebase. Write clean, readable code and adhere to existing coding conventions.
Step 6: Test Your Changes
Run the test suite to make sure your changes do not break existing functionality. Add new tests if you are adding new features or fixing bugs.
Copy
Step 7: Commit Your Changes
Write clear, concise commit messages that describe your changes. Here’s a good example:
Copy
Step 8: Push to Your Fork and Submit a Pull Request
Push your branch to your fork on GitHub:
Copy
Then, go to the astraivm repository on GitHub and submit a pull request. Describe your changes and link the pull request to any relevant issues.
Step 9: Code Review
Maintainers will review your pull request and may suggest changes. Engage in a conversation with maintainers and make necessary revisions to your pull requests.
Step 10: Merge
Once your pull request is approved, a maintainer will merge it into the main branch. Congratulations, and thank you for contributing!
Contributing to Documentation
If writing code isn't your thing, you can contribute by improving the project's documentation. Good documentation helps users and developers understand and use astraivm more effectively. Update documentation to reflect new features, enhance readability, or correct errors.
Reporting Bugs
If you find a bug and don’t have the time or expertise to fix it, you can contribute by reporting it in the GitHub issues. Be sure to provide a detailed report with reproducible steps, expected outcomes, and actual outcomes.
Participating in Code Reviews
Even if you're not contributing code, you can participate in code reviews. Offering feedback on pull requests is a valuable contribution and helps maintain the quality of the project.
Conclusion
Contributing to astraivm not only helps improve the project but also enriches your understanding of blockchain technologies and networking protocols. Whether you contribute code, participate in discussions, or improve documentation, your involvement is warmly welcomed and highly appreciated.
Imagine a data economy where data is not locked idle in silos, but instead is actively in motion between its participants for various use cases: every byte can be put to good use and every data point carries the potential to bring new innovations. AstraAI's mission is to establish a unified data landscape that is inclusive and interoperable.
AstraAI introduces a novel modular decentralized network that powers a public data sharing ecosystem and allows the deployment of custom private or semi-private data-sharing networks (data consortiums)
Many industries have faced disruption of traditional business models in recent years, and in the coming years many more industries will face disruptions too while startups innovate more rapidly than ever. After OpenAI took the world by storm with ChatGPT, artificial intelligence will play its own disruptive role in virtually every industry. Traditional businesses are forced to protect existing business models, and explore new ones in order to stay ahead of the competition. However, there's something that these traditional enterprises all have in common: they have gathered very large amounts of data.
All companies and individuals generate data, day in and day out. We generate data by using our phones, taking public transport, driving our cars or while shopping for groceries. This generated data usually serves a clear purpose, like targeted advertising, optimization of availability of buses and trains, reporting of congestions or to build a purchasing strategy.
Beyond its original scope, this data is largely ignored and remains locked away on private servers. Currently, fragmentation of the data landscape leads to a high barrier to monetize data: you'll need to use different tools to access data from different sources, build custom connectors or find the right ingestion tool to combine the data from these different sources. Moreover, you'll face expensive business intelligence platforms that have more features than your organization will ever need in order to leverage the insights that come from this data. Even just building out a pilot project to test the waters can become a lengthy and costly process quickly, defeating its purpose entirely. While data should be an organization's asset, it's becoming a liability.
Enterprises that look for new business models in order to stay ahead, need to leverage their vast amounts of data, but are facing these persistent challenges to be able to quickly experiment and adapt.
LLMs are trained on massive amounts of unstructured data. This generally means they are a pleasant conversational partner that can assist you in many useful ways, they can even code for you! The downside of current LLMs is that when it comes to fact-based information, details are largely hallucinated.
Incorporating structured data into LLM training can significantly enhance their capabilities in fact-based conversations and reasoning. This adaptation could lead to the emergence of new use cases for LLMs, such as more sophisticated analytical tasks and specialized professional consultations, catering to industries like legal, healthcare, and finance, where accuracy and up-to-date information are crucial.
The challenge lies in the fragmented nature of data landscapes. Accessing structured data feeds is difficult due to their varied formats and the necessity for numerous custom connectors. This fragmentation hampers the integration process and consequently complicates the task of rewarding data feed owners fairly and transparently.
Creating data-sharing ecosystems that integrate with LLMs can address these issues and unlock great potential for niche use cases. Such ecosystems would allow for efficient and equitable data exchange, leveraging an LLMs' ability to provide accurate, context-aware insights. For instance, in healthcare, real-time patient data can enable LLMs to offer better diagnostic support, while in finance, up-to-the-minute market data can lead to more accurate financial forecasting.
To fully exploit these integrations, third-party developers would require easy-to-use APIs. These APIs would enable seamless integration of AI capabilities into existing systems, allowing businesses to leverage advanced LLM functionalities without the need for extensive technical expertise.
Lastly, the cost of training custom models or running inference on large datasets can be prohibitive, particularly for small and medium-sized businesses (SMBs). This necessitates a shift towards distributed computing power, which would democratize access to advanced AI capabilities, allowing SMBs to compete on a level playing field with larger corporations.
AstraAI revolutionizes data management and utilization in a way that seamlessly blends with the needs of modern businesses. One of the platform’s core strengths is its ability to effortlessly upload and store datasets of different formats, automatically structuring them into an efficient, generalized format. This uniformity ensures that when users access multiple datasets, they encounter a consistent interface, significantly simplifying data manipulation and analysis.
The platform's capability to combine multiple datasets opens possibilities for generating new insights. Such combinations allow for the exploration of connections and trends that were previously undiscoverable due to the isolation of these datasets. This feature is particularly revolutionary, as it enables the synthesis of knowledge from diverse domains in a single query, unlocking entirely new possibilities and insights.
AstraAI also offers opportunities for external contributors to monetize their skills. These contributors can enhance the platform by enhancing the metadata of the dataset. This richly described metadata is then more effectively utilized, for example in LLM integrations or in AI-driven analyses to draw connections between seemingly unrelated datasets.
The platform further simplifies the data analysis process with its visual data pipeline editor. This tool allows users to create data pipelines for deriving insights from combined datasets without the need for expertise in SQL, Python, or similar languages, making advanced data analysis attainable for a broader range of users.
AstraAI is built on a foundation of fairness and inclusivity. Contributors of data and metadata are rewarded appropriately and transparently, ensuring a sustainable and thriving community-driven collaborative ecosystem. Additionally, the platform's LLM integrations, which can be trained or run inference using distributed computing power, enable users to interact with their own or others’ data, bringing a more intuitive and human dimension to data analysis.
AstraAI addresses the contemporary challenges of fragmentation of the data landscape and accessibility by offering a collaborative, community-driven, unified, efficient, and user-friendly platform that will power the next generation of LLMs. It empowers businesses of all sizes to leverage the full potential of their data, enabling them to innovate and compete more effectively.
If you want to get a better understanding of the AstraAI Network design. Please navigate to the Next Page! For alternative options, check the side menu.
Overview
Welcome to the astraivm documentation! astraivm is a blockchain Virtual Machine (VM) built using the hypersdk framework from Ava Labs. It is designed to run as one of the subnets in the Avalanche network, leveraging the unique features of Avalanche to enhance the capabilities of blockchain applications.
astraivm combines the functionalities of previously developed VMs, such as morpheusvm and tokenvm, and introduces new features like native token staking and an emission balancer. This makes it a versatile and powerful tool for developers looking to deploy scalable and efficient blockchain solutions.
This section will provide you with a high-level understanding of astraivm
astraivmPurpose of astraivm
astraivm aims to provide a robust and flexible environment for deploying and managing blockchain applications. Its integration into the Avalanche network allows it to benefit from the network's high throughput, low latency, and secure consensus mechanism. Here are a few key purposes of astraivm:
Tokenization: Facilitate the creation, minting, and management of digital assets and tokens.
Interoperability: Enable seamless asset transfers and communication across different subnets within the Avalanche ecosystem through Avalanche Warp Messaging (AWM).
Decentralized Finance (DeFi): Support a wide range of DeFi applications by providing tools for asset trading, staking, and governance.
Scalability: Handle large volumes of transactions efficiently, thanks to the underlying hypersdk framework, designed for high-performance blockchain solutions.
Customization: Allow developers to customize and extend the VM's capabilities to suit specific application needs.
astraivm and Avalanche Network
astraivm operates as a subnet within the Avalanche network, which is a highly scalable, interoperable, and secure platform for decentralized applications. By running as a subnet, astraivm benefits from:
Security: Leverage the security model of Avalanche, which ensures the integrity and security of transactions and operations within astraivm.
Custom Consensus: Implement custom consensus protocols tailored to specific use cases, optimizing performance and responsiveness.
Network Effects: Benefit from the broader Avalanche ecosystem, including integration with other subnets and access to a wide user base and application marketplace.
Astraivm features
Actions
☑ Transfer both the native asset ATAI and any other token created by users within the same subnet
☑ Transfer both the native asset ATAI and any other token created by users to another subnet using Avalanche Warp Messaging(AWM)
☑ Create a token
☑ Mint a token
☑ Burn a token
☑ Export both the native asset ATAI and any other user tokens to another subnet that is also a astraivm
☑ Import both the native asset ATAI and any other user tokens from another subnet that is also a astraivm
☑ Register validator for staking
☑ Unregister validator from staking
☑ Delegate ATAI to any currently staked validator
☑ Undelegate ATAI from a staked validator
☑ Claim Validator staking rewards
☑ Claim User delegation rewards
Emission Balancer
☑ Tracks total supply of ATAI, max supply of ATAI, staking rewards per block and the emission address to direct 50% of all fees to
☑ Register validator for staking
☑ Unregister validator from staking
☑ Delegate ATAI to a validator
☑ Undelegate ATAI from a validator
☑ Claim the staking/delegation rewards
☑ Track the staking information for each users and validators
☑ Distribute 50% fees to emission balancer address and 50% to all the staked validators per block
☑ Distribute ATAI as staking rewards to the validators that have a minimum stake of at least 100 ATAI per block
Getting Started
The next sections of this documentation will guide you through setting up astraivm, developing applications, and utilizing its full range of features. Whether you are setting up a local test environment, integrating astraivm into your development workflow, or deploying production-ready applications, this guide will provide the necessary steps and resources.
Stay tuned as we dive deeper into the functionalities and development practices with astraivm in the upcoming sections.
git clone https://github.com/your-username/astraivm.git
cd astraivmgit checkout -b your-branch-name./scripts/tests.integration.shgit commit -am "Add a new staking feature to improve transaction handling"git push origin your-branch-nameAstraAI's architecture is designed in such a way that data sharing, requesting, and querying can all take place on-chain and in a decentralized way. This enables builders to create trustless data-focused technologies and platforms on top of the AstraAI network. The network supports next-generation Large Language Models (LLMs) and artificial intelligence (AI) by offering a decentralized computational network for training and using AI-driven solutions.
The use of a public network is complemented by private data-sharing subnets, which enable secure and trustless data sharing via the establishment of data consortiums. This is particularly beneficial for builders focusing on specific industries or communities that exchange sensitive or private data.
The Emission Balancer is a critical component of the staking and rewards distribution mechanism within the Astrai network. It is designed to manage and balance the emissions of ATAI tokens, ensuring fair and proportional distribution based on stake contributions and validator participation. This documentation provides an in-depth look at its functionalities, the underlying mechanisms, staking rewards calculations, and other relevant features.
Existing data-sharing networks and intelligence platforms heavily rely on centralized services. When businesses share data, especially with potential competitors, they expose themselves to risk through their valuable assets. Data-sharing consortiums need a high level of trust among all network participants, which is challenging to establish and maintain. Who will own and maintain the infrastructure needed? Who is appointed to control accounting, and how will you detect fraudulent activity? Decentralization offers a solution by removing the need for trust. In a decentralized network, participants can engage confidently, knowing they remain in control over their data and their interests are protected.
AstraAI is an ecosystem that, unlike traditional platforms that rely on central entities for operation and maintenance, is designed to be self-sustaining, ensuring its longevity and resilience. In a decentralized ecosystem like AstraAI, the absence of reliance on central parties for backend services is a significant advantage. This structure guarantees that the ecosystem remains operational and efficient, even if individual companies within it face challenges or cease operations.
Contributors outside of the network contribute to datasets and enhance metadata. It will be too much hassle to set up an agreement between each contributor and data provider to ensure each contributor is fairly rewarded for their contributions. Decentralizing these agreements through code and tracking the contributions on-chain ensures that rewards are transparently distributed as agreed without having to trust an intermediary.
The network of nodes and subnets serves as a way to secure the network and as a way of distributing compute power to network participants that need to run big data pipelines or train custom large language models using the data in AstraAI's ecosystem.
Compute Nodes serve as the distributed computational power of the AstraAI network. They provide CPU and GPU power that can perform a variety of complex tasks such as training custom AI (like LLMs). Those that run compute nodes get compensated in NAI for sharing their idle resources.
Validator Nodes function as the auditors, looking for errors or attempts to compromise the network with false information. They ensure the integrity of computations and transactions that take place within the network. Validator nodes also ensure that the network's specific tasks are executed and they are responsible for the emission balancer.
The nodes within AstraAI's ecosystem provide distributed computing power, capable of managing extensive data pipelines or training custom large language models. For instance, in the field of meteorology, this computational capacity is instrumental in enhancing weather forecasting and climate modeling. Such advancements are vital for sectors like agriculture and disaster management. Similarly, in healthcare, the distributed network supports federated learning, facilitating the development of medical AI models while prioritizing patient privacy.
For small and medium-sized businesses, which typically lack the resources to operate and maintain advanced technological infrastructures, this distributed compute network opens the door to a wide array of applications that were previously inaccessible.
Compensation for compute node operators is issued in NAI tokens. Upon the initiation of a compute power request, the requisite amount of NAI is determined and reserved until the task is completed and verified. A portion of each transaction, specifically 25%, is allocated to the emission balancer protocol.
Participation as a compute node requires operators to stake NAI tokens (500,000 NAI). This stake is at risk of being reduced, or 'slashed', should the node exhibit unreliable outputs or suffer from recurring or significant downtime.
The validators of the network ensure the decentralization and thus the security of the network. Their primary role is to validate the transactions and that each action on the network adheres to the network's protocol and rules. Additionally, they are also tasked with confirming the compute resources that have been spent by the compute nodes, run the emission balancer and execute the network-specific tasks, like:
Distribute dataset revenue to all stakeholders, ensuring that contributors are fairly rewarded.
Record actions, like querying datasets and data pipeline executions, to ensure that results can be traced back to their origins.
Enforce access control.
The minimum stake for node operators is set at 1.5 million NAI, which must be maintained for a minimum duration of six months to avoid slashing penalties. For the initial 100 nodes, the annual percentage rate (APR) on their stake is set at 25%. Beyond this threshold, the APR is proportionately distributed among all nodes. Consequently, with 200 validator nodes, the APR would be 12.5%. In the network's initial phase, validator nodes require authorization before joining the network.
With each transaction on the network that mutates the state of the blockchain, a transaction fee needs to be paid to reward the nodes for validating and executing these transactions. These fees are paid in NAI. Transaction fees bear a minimum, determined by the units of work that will take place but can be increased to convince a node to pick up the transaction before others when a quick time of execution is important. Of each transaction fee paid, 50% is automatically submitted to the emission balancer. The remainder is used to reward the nodes for validating the network.
The AstraAI network requires active participation by its curators. These individuals or companies play a significant role in adding value to the network by introducing new data, curating existing data, transactions that take place, contributions that have been made, and by ensuring that ethical standards are upheld. Newly sourced data acts as a diverse source of information for data analysis and applications. Since these individuals or companies spent significant resources introducing unique, relevant, and often hard-to-source datasets they bring tremendous value to the network. This data can then be used within advanced analytics, machine learning, or the training of other artificial intelligence models such as large language models.
By introducing new unique data, curators help maintain the network's value. In addition to those that contribute data, curators in the AstraAI network as 'contributors' increase the utility of existing datasets through deeply enriching the metadata of the datasets. This process contextualizes the data by annotating it and tagging it in various ways. This increases the accessibility, interpretability, and applicability of the data. By doing this work, curators increase the value of the raw data and transform it into a more valuable asset.
Curators also combine different datasets with each other, thereby unveiling new insights and correlations. The synthesis of these diverse datasets results in new datasets of their own. Such datasets may reveal patterns and trends that were hidden before, opening the door for potential innovation and problem-solving. Curators are key players that are driving the network's value.
As a decentralized network that powers the new data economy, the AstraAI network requires a network token to function. This network token has several utilities within the ecosystem:
Means of access and toll: Each transaction that is done on the network is registered and validated. Participants of the network pay a fee for the usage of the network. This fee is paid with the NAI token.
Means of reward: Contributors to the network are incentivized through NAI tokens in order to bootstrap the network and increase the level of decentralization of the network.
Means of data-control: When data is shared within (semi) private consortium networks, new AstraAI subnets are deployed and are required to be connected and secured by the main AstraAI network. In order to secure and validate these subnets, NAI is required as well.
Means of compute power: Computational power that is added to the network and is used by participants to execute large data pipelines, train their AI, or custom large language models on, are rewarded in NAI for providing the computational power to the network. Depending on how long and how complex these computations are, the rewards in NAI will vary.
Means of governance: Participants in the network determine the future of the network and the token serves as a democratic and decentralized way of decision making that are in the best interest of the participants of the network.
The AstraAI network follows a system in which inflation is reduced over time by means of reducing the block rewards gradually. Each month the inflation rate gradually drops according to the activity on the network, until the maximum supply of 10 billion NAI tokens is reached.
On top of that, the AstraAI network incentivizes sufficient decentralization by rewarding nodes that secure the network to provide the decentralized security that is needed. The token distribution of the network is optimized for both node decentralization and the formation of the DAO treasury.
Token distribution: NAI will be launched with an initial supply at genesis of 853 million NAI. An Emission Balancer is implemented as a mechanism to avoid unlimited growth, ensuring stabilization of the maximum supply of 10 billion NAI, assuming sufficient utilization of the main network and the computational nodes.
The DAO will be installed to give each stakeholder in the ecosystem a voice and a vote in decision-making for the future of the AstraAI network. Anyone that is a holder of NAI is able to make a proposal. Where validators provide technical decentralization, the DAO makes sure that the governance of the network is decentralized. It is foreseen that there are a few important topics that the DAO can decide upon:
Spending & Budget limits of the DAO can be increased or decreased depending on the DAO budget.
The maximum APY validator nodes receive at a given time.
DAO token allocations to community incentives, marketing and business development, and further technical development of the ecosystem.
The DAO will receive its first allocation of NAI at the inception of the mainnet. It will continuously receive NAI until a total allocation of 1.3B is emitted and can only be utilized by the DAO itself.
In order to balance the token emissions of the network and achieve a theoretical maximum supply, mechanisms to reduce the emission of tokens will be put in place:
When computational nodes are used within the network, 25% of their income will be used within the emission balancer.
50% of all transaction fees will be subject to the emission balancer at all times.
50% of all transaction fees and 25% of the computational income will be accrued in the Emission Balancer, totaling 3M tokens over the coming month.
2M tokens are minted over the coming month to reward the validator nodes.
Instead of minting 2M new tokens, the Emission Balancer's treasury is used to reward the validator nodes.
1M tokens are left in the Emission Balancer, which will reset to 0 in the next Emission Balancer epoch (approximately one month).
Note, that even though in the example a period of a month is used, new token emissions and emission balancer transactions take place per block.
The following scenarios illustrate how the AstraAI network will create a new data economy and will showcase the utility of the network token.
Data Consortiums are established by multiple organizations that benefit from sharing data with other consortium members. Within these consortiums, there is a need to control the data that is shared and who has access to which datasets, which can be limited to a set of approved partners. Some of the industries where data consortiums are likely to be formed are within the web3 ecosystem, the agricultural technology sector, the automotive industry, and many others.
An example of a data consortium would be a car manufacturer ecosystem, where various companies in the supply and value chain work together with each other. The car manufacturer can share data (either for free or paid) with dealerships and vice versa. Other data might be shared with start-ups that are working to improve some technological innovation they are working on or to train an AI. Data usage can be tracked and micro-payments can take place where relevant.
Companies and individuals alike can contribute to datasets, as it's tracked who provided what data or metadata. Data collaborators can work together to enhance the metadata of datasets, even optimized for different specific purposes (human-readability, AI integration, etc.). The incremental benefits this brings, is that smaller companies or even groups of individuals could pool together their data and monetize it. This creates a leveled playing field against larger competitors within the data space.
Capgemini research showed that the majority of global enterprises have large amounts of datasets that remain underutilized. At the same time, companies are looking outside of their organization for valuable data sources for various reasons. Such as improving their machine learning or artificial intelligence models or providing their large language models with structured data. All of this to get better business insights and do better forecasting. With the introduction of AstraAI, we intend to break down the barriers that currently exist when companies want to monetize their underutilized data or purchase data from third parties.
Compute power is increasingly becoming a critical resource for businesses and researchers, especially in the fields that require intensive computational tasks such as federated learning, machine learning, and the training of artificial intelligence models. A robust and scalable computational resource is required to process large amounts of data, redefining algorithms, and performing complex statistical simulations. This is especially pronounced in the field of artificial intelligence and machine learning, where the training and fine-tuning of models require substantial computational capacity.
Federated learning, an approach to machine learning, further propels this demand by enabling the training of algorithms across multiple decentralized devices or servers, thereby necessitating significant distributed computer power.
You've reached the finale! We hope you're as excited as we are about the numerous possibilities that the AstraAI Layer 1 Network will unlock for its users!
On the next page, you can dive into our data marketplace!
Validator Dynamics
Validators are crucial to the AstraiVM ecosystem, responsible for processing transactions, creating blocks, and maintaining the blockchain's overall health. Their eligibility and selection are contingent upon the amount of ATAI staked, with higher stakes improving their chances of earning more rewards.
Delegation System
AstraiVM facilitates a delegated staking system, allowing token holders to delegate their stakes to validators, thus participating in the network's security and earning potential indirectly. This system democratizes the earning process, enabling smaller stakeholders to benefit from the network's growth. The longer the users delegate, the higher their rewards will be when claiming.
At the heart of AstraiVM's reward distribution lies the Emission Balancer, a sophisticated algorithm designed to ensure fair and sustainable reward allocation among validators and delegators.
Dynamic APR and Reward Calculation
The Emission Balancer dynamically adjusts the Annual Percentage Rate (APR), taking into account the total number of active validators and the aggregate staked amount. This ensures that the rewards remain sustainable and proportional to each participant's contribution. Validator rewards are computed based on their staked amount, stake duration, and their performance in validating transactions.
Stake Tracking and Management
Stakes in AstraiVM are meticulously tracked, with each staking event—be it registrating validator for staking, withdrawing validator, delegating to validators, or undelegation—prompting an update in the system. This event-driven model ensures that the total staked amount and individual validator stakes are always current, allowing for accurate reward computations.
Minting of New ATAI
The Emission Balancer is responsible for minting new ATAI tokens, adhering to predetermined emission schedules and caps. This minting process is directly tied to the validation of new blocks, with freshly minted tokens being distributed as rewards to active validators and delegators based on the calculated reward distribution.
Fee Distribution Mechanism
Transaction fees collected by AstraiVM are also managed by the Emission Balancer. A portion of these fees is redistributed as rewards, adding an additional incentive layer for network participants. The distribution follows the same equitable principles, ensuring validators and delegators receive fees proportional to their contributions.
Validator: Represents a node that has staked ATAI tokens to participate in the network's consensus mechanism. Validators earn rewards based on their staked amount and the delegations they receive.
EmissionAccount: Holds the unclaimed balance of ATAI tokens that are to be distributed as rewards.
EpochTracker: Manages the epochs, which are time periods in which rewards are calculated and distributed. An epoch's length determines how frequently rewards are calculated.
Initialization
Upon initialization, the Emission Balancer sets up with the total supply, maximum supply of ATAI tokens, and the emission account details. It also establishes a map to track validators and their information.
Staking and Delegation
Validators can stake ATAI tokens to participate in the network, and users can delegate their tokens to validators. The Emission Balancer records and updates these stakes and delegations, adjusting the total staked amount accordingly.
Reward Calculation
Rewards are calculated based on the Annual Percentage Rate (APR), the total staked amount, and individual validator contributions. The APR can adjust based on the number of validators, ensuring a balance between incentivizing participation and maintaining a sustainable reward rate.
APR Adjustment
The APR adjusts inversely with the number of validators beyond a base count, ensuring that as more validators join, the rewards are balanced to prevent inflation.
Rewards Per Epoch
At the end of each epoch, the total rewards are calculated based on the APR and the total staked amount. These rewards are then distributed among validators and delegators according to their contributions.
Reward Distribution
Rewards are distributed at the end of each epoch. Validators and delegators can claim their accumulated rewards. The distribution takes into account the delegation fee rate set by validators, which determines the split between validator earnings and delegator rewards.
Fee Distribution
Transaction fees are collected and distributed alongside rewards. A portion of the fees goes to the emission account, and the rest is distributed among validators and delegators, similar to reward distribution.
Withdrawals and Claims
Validators and delegators can withdraw their staked tokens and unclaimed rewards. The Emission Balancer handles these transactions, updating the total staked amount and validator statuses accordingly.
Block Height and Timestamps
The Emission Balancer relies on block height and timestamps to manage epochs and reward distributions. Each block's acceptance into the chain triggers checks against epoch lengths and distribution schedules.
Validator States
Validators have active and inactive states, determined by their stake start and end times. Only active validators participate in reward distributions.
Delegator Rewards Calculation
Delegator rewards are calculated based on the amount they have staked with a validator, the duration of the stake, and the rewards allocated for delegators in each epoch.
Efficient Reward Calculation
To optimize performance, the Emission Balancer calculates rewards per unit staked, reducing the need for iterative calculations across all validators and delegators.
Dynamic APR: Adjusts based on validator count to ensure a balanced reward system.
Epoch-Based Rewards: Facilitates predictable and regular reward distributions.
Delegation Support: Allows users to delegate tokens to validators, participating indirectly in the consensus mechanism.
Transparent Reward Distribution: Ensures fairness in distributing rewards based on stake contributions.
Scalability: Designed to handle a growing number of validators and delegators efficiently.
The mind map covers the following aspects of the Emission Balancer:
Initialization: Setting up with total and maximum supply, along with a validators map.
Staking and Delegation: Recording stakes and updating delegations.
Reward Calculation: Based on APR and stake contributions.
Reward Distribution: Occurring at the end of each epoch and allowing claims of rewards.
Fee Distribution: Handling transaction fees and the emission account.
Withdrawals and Claims: Managing the withdrawal of staked tokens and claiming of unclaimed rewards.
The Component diagram includes the following classes and their relationships:
EmissionBalancer: The main class managing various operations such as initialization, reward calculations, stake management, and interactions with the blockchain VM.
Validator: Represents a validator in the system with attributes such as NodeID, PublicKey, StakedAmount, and related reward and delegation information.
EmissionAccount: Holds the unclaimed balance of ATAI tokens to be distributed as rewards.
EpochTracker: Manages epochs for reward calculation and distribution.
AstraiVM: Interacts with the underlying blockchain VM for state management, validator information, and block data.
The Emission Balancer plays a pivotal role in the Astrai network's staking ecosystem, ensuring fair reward distributions and incentivizing network participation. Its design considerations for scalability, efficiency, and fairness make it a foundational component for maintaining the network's health and sustainability.