IOSG Ventures: LLM Empowering Blockchain, Opening a New Era of On-Chain Experience
Author: Yiping, IOSG Ventures
Introduction
With the booming development of large language models (LLMs), we have seen many projects integrating artificial intelligence (AI) and blockchain. The combination of LLM and blockchain is becoming more and more common, and we are witnessing the opportunity for AI to re-merge with blockchain, with zero-knowledge machine learning (ZKML) being one notable example.
Artificial intelligence and blockchain are two transformative technologies with fundamental differences. AI requires powerful computational capabilities, often provided by centralized data centers. On the other hand, blockchain offers decentralized computing and privacy protection, but performs poorly when it comes to large-scale computing and storage tasks. We are still exploring and researching the best practices for integrating AI and blockchain, and we will also introduce some project cases that combine "AI + blockchain" in the future.
Source: IOSG Ventures
This research report is divided into two parts, and this article focuses on the application of LLM in the encryption field, as well as the exploration of implementation strategies.
What is LLM?
LLM (large language model) is a computerized language model consisting of an artificial neural network with a large number of parameters (usually billions). These models are trained on a large amount of unlabeled text.
Around 2018, the emergence of LLM completely changed the research on natural language processing. Unlike the previous approaches that required training specific supervised models for specific tasks, LLM as a universal model performs well on various tasks. Its capabilities and applications include:
Understanding and summarizing text: LLM can understand and summarize a large amount of human language and text data. They can extract key information and generate concise summaries.
Generating new content: LLM has the ability to generate content based on text. By providing prompts to the model, it can answer questions, generate new text, summaries, or sentiment analysis.
Translation: LLM can be used for translation between different languages. They utilize deep learning algorithms and neural networks to understand the context and relationships between vocabulary.
Predicting and generating text: LLM can predict and generate text based on contextual backgrounds, similar to human-generated content, including songs, poems, stories, marketing materials, and more.
In various fields of application: large-scale language models have wide applicability in natural language processing tasks. They are used in conversational AI, chatbots, healthcare, software development, search engines, tutoring, writing tools, and many other fields.
The advantages of LLM include its ability to understand a large amount of data, perform various language-related tasks, and customize results according to user needs.
Common applications of large-scale language models
Due to its outstanding natural language understanding abilities, LLM has great potential, and developers mainly focus on the following two aspects:
Providing accurate and up-to-date answers to users based on a large amount of contextual data and content
Completing specific tasks given by users through the use of different agents and tools
It is these two aspects that have led to a proliferation of LLM applications such as chatting with XX, chatting with PDFs, chatting with documents, and chatting with academic papers.
Subsequently, people have attempted to integrate LLM with various data sources. Developers have successfully integrated platforms such as Github, Notion, and some note-taking software with LLM.
To overcome the inherent limitations of LLM, different tools have been incorporated into the system. The first such tool is a search engine, which provides LLM with the ability to access the latest knowledge. Further advancements will integrate tools like WolframAlpha, Google Suites, and Etherscan with large-scale language models.
Architecture of LLM Apps
The diagram below outlines the process of LLM applications when responding to user queries: Firstly, relevant data sources are transformed into embedding vectors and stored in a vector database. The LLM adapter uses the user query and similarity search to find relevant contexts from the vector database. The relevant contexts are put into Prompts and sent to LLM. LLM executes these Prompts and generates answers using tools. Sometimes, LLM may fine-tune on specific datasets to improve accuracy and reduce costs.
The workflow of LLM applications can be roughly divided into three main stages:
Data preparation and embedding: This stage involves preserving confidential information (such as project memos) for future access. Typically, files are split and processed using embedding models, saved in a special type of database called a vector database.
Prompt formulation and extraction: When a user submits a search request (in this case, searching for project information), the software creates a series of prompts to input into the language model. The final prompt typically includes prompt templates hardcoded by software developers as effective output examples for few-shot learning, as well as any necessary data obtained from external APIs and relevant files extracted from vector databases.
Prompt execution and inference: After the prompts are completed, they are provided to pre-existing language models for inference, which may include proprietary model APIs, open-source models, or individually fine-tuned models. At this stage, some developers may also integrate operating systems (such as logging, caching, and verification) into the system.
Introducing LLM to the Encryption Field
While the encryption field (Web3) has some applications similar to Web2, developing excellent LLM applications in the encryption field requires extra caution.
The encryption ecosystem is unique, with its own culture, data, and fusion. LLM fine-tuned on these encryption-constrained datasets can provide superior results at relatively low costs. Although rich data is available, there is a notable lack of open datasets on platforms like HuggingFace. Currently, there is only one dataset related to smart contracts, containing 113,000 smart contracts.
Developers also face the challenge of integrating different tools into LLM. These tools, unlike those used in Web2, empower LLM with access to transaction-related data, interaction with decentralized applications (Dapps), and the ability to execute transactions. So far, we have not found any integration of Dapps in Langchain.
Although developing high-quality encrypted LLM applications may require additional investment, LLM is naturally suited for the encryption field. This field provides rich, clean, and structured data. Coupled with the concise nature of Solidity code, this makes generating functional code with LLM easier.
In the following sections, we will discuss eight potential directions in which LLM can help the blockchain field, such as:
Integrating built-in AI/LLM capabilities into blockchain
Analyzing transaction records using LLM
Identifying potential bots using LLM
Writing code using LLM
Reading code using LLM
Assisting communities using LLM
Tracking markets using LLM
Analyzing projects using LLM
Stay tuned!
