Skip to content

AI Integration into Decentraland

EnactedGrant Request

Proposal Details

Author0xedf4…0bad
PublishedJan 17, 2024 11:07
Voting beginsJan 17, 2024 11:07
Voting endsJan 31, 2024 11:07
Snapshot#5aca038

Grant size

24,000 DAI

Tier

Higher Tier

Beneficiary

0x64ac…CE57

Description

This grant is aimed to bring AI technology to Decentraland through Open Source tools. Obviously, AI & LLMs (Large Language Models) have changed the world in the last year. We hear from all that we will not interact with the world in the old manner. That's why we need to stay updated in this field in Decentraland. This proposal outlines an advanced AI integration system with applications in virtual environments and language processing. It includes a customizable AI character, multilingual interaction capabilities, and diverse backend setups compatible with various LLMs like OpenAI and Open Source Models. Key tools such as MemGPT and AutoGen enhance AI character detailing and interaction. The system's use cases are versatile, covering virtual guides in Decentraland for navigation and information, interactive storytelling and roleplay, AI companions for social interaction, innovative educational modules, and AI-driven customer service representatives, decorations, quests, audio & speech features. Progress from the last proposal: 1. [Chat-bot for Golf Craft Game: Deployed in Genesis City](https://decentraland.org/play/?position=55%2C-42) 2. [Chat-bot for Decentral Game ICE Poker](https://twitter.com/TonyMo74/status/1745380483389956509) 3. [Multilanguage Q&A](https://twitter.com/TonyMo74/status/1746759684705632643) 4. [Image generation integration](https://twitter.com/TonyMo74/status/1740682713332637925) 5. [Retweet by LangChain](https://twitter.com/LangChainAI/status/1746569216239808885)(a widely used library for building AI/LLM applications) - They highlighted that it is a completely new interface for AI. These tweets received 55K impressions! Many likes, views, etc. Several developers (non-DCL related) asked me in DMs how to build on Decentraland + AI. This brings in completely new people to the board. Planned features (all open source): - Frontend library for AI in SDK7 (includes NPC Characters, Shapes for Image Generations, prompts for music, voice, text generation). - Backend server with abilities to use proprietary API (OpenAI, Claude2) and Local Open Sourced models. - Backend for running local models (ollama) and tools (like MemGPT, LangChain, etc). - Comprehensive documentation on how to run it. Features available: - Chat Bot NPC knowledgeable about specific projects, by uploading PDFs, links, websites. - Multilanguage Q&A Characters. - Image Generation. - Voice Generation (new level of interactivity). - Music Generation. - AI role play NPC - predefined stories and behavior by creators. - First experiments of integrating Web3 functionality to AI, including NPCs with their own Ethereum addresses for transactions.

7 Comments

palewinFeb 15, 2024

Yesterday we had the onboarding meeting with the grantee. We reviewed the roadmap and the duration of the project. The team is already working, and their first month of development is very advanced. We will check their public Roadmap monthly, which is created here: https://dcl-dao.notion.site/d907b77f5c2149859888bf0636d90b2c?v=f431639082f44bb5b1c05513e9ae989b&pvs=74 Regards and thank you!

daoFeb 14, 2024

AI Integration into Decentraland This proposal has been ENACTED by a DAO Committee Member (0xfb1afa4dc069ffb47b19dbee196045d508fcd5a2) Vesting Contract Addresses: 0x1dc90bf74bf7c3d98fb1b1ae98b5e72a73dbc89d

daoJan 31, 2024

AI Integration into Decentraland This proposal is now in status: PASSED. Voting Results: * Yes 77% 6,584,446 VP (58 votes) * No 22% 1,929,465 VP (9 votes) * Abstain 1% 42,132 VP (4 votes)

CanessaJan 19, 2024

I'd love to see this integrated with Doki's NPC avatar project. :call_me_hand:

DedHeadJJan 18, 2024

![image|690x388](upload://yZAH6mXbUTzAU3hLnQkIIpNG4tt.jpeg) I voted to Abstain prior because I didn't fully understand the technical aspect of the project. Now you have provided "easier to understand" data, examples and information that a non-technical person can understand. Thank you and I will vote YES this time. I am very excited about this project and thank you for bringing it forward. Jenn

daxJan 17, 2024

i was a little skeptical about this at first, but after talking to tony and understanding the proposal better, and that this is more like an open-source framework which allows users to use any LLM with the various tools implemented here, I think this a great addition for a relatively small sum. I also really love that you can upload PDFs to train your own bots to talk about your scene / game / business / etc - it's a small feature that's going to save a lot of time and also encourage better documentation and that feels like a big plus for me!

PunkPinkJan 17, 2024

As many of you already know my opinion, I think that integrating AI into the platform seems essential to me. Firstly, because we are updating the platform to these new technologies, preventing us from being left behind, and at the same time improving the functions we currently have. On the other hand, it is a great claim to attract new users. We have all realized that AI is on everyone's lips and I don't think we should be left behind in this race.