TL;DR

We built an onchain game with AI-powered NPCs. Sign up for early access now 🤠.

Get early access

The Slack message came through to me like almost all of them do—while I was in the middle of something important. Normally, I would have waited until I could read it and then respond. However, I immediately stopped what I was doing and responded. The message was from our Head of Design, Marge. She asked me if I wanted to join her and our Head of Developer Relations, Steve, on a hacking team for EthGlobal Online. Pinata’s entire history is built around hackathons—born from a hackathon, launching new products at hackathons, and supporting hackathons—so, of course, I was in.

The Pinata team was fresh off wins at EthNYC, and we knew we wanted to build something completely fresh, in an entirely different direction. We started bouncing ideas off each other, but an idea Steve and I had been talking about for weeks kept coming back to us. Spurred on by a comment from our CEO weeks ago, we had been trying to figure out how we could apply ERC-6551 (tokenbound accounts) and AI. For the hackathon, we decided we’d make a game where NPCs were the main characters. In doing so, we’d be able to showcase the power of blockchain, tokenbound accounts, and AI.

Cosmic Cowboys was born

With the rough idea that we wanted to build an old-school text adventure-style game, we started working on our story. Eventually, we crafted a game story that would power everything else. Our NPCs would be space cowboys who launched into space to mine a giant lithium rock that had crashed into our galaxy. Only, they didn’t make it to the rock. They crash-landed on Jupiter’s moon Ganymede. There, they had to use what supplies they had to build a settlement and survive.

Starting with a story was critical to creating the logic and functionality for the game. Knowing that the game would be on-chain, we worked within the constraints of smart contract logic while combining that with a server that could manage all of our AI interactions. From the game story, space cowboys were given a goal (amass the most credits on Ganymede) and they were provided game logic (i.e. you can only buy and sell supplies at the supply depot). At least, they were given these things on paper. We still had to create the characters, launch them into space, and figure out how to make it all work on-chain.

Bringing the characters to life

To make the game world fun and more immersive, we wanted every character to have their own backstory and image. Using OpenAI’s API, we prompted the model with the game’s backstory and asked it to create 20 NPCs with varying backstories that could all be tied into the game’s main backstory. We also added a request to return a prompt that could be used in Midjourney to generate character images.

We learned a couple of interesting things from this process:

1. Diversity has to be intentional
2. It’s hard to generate good image prompts from a prompt

OpenAI is great at creating static and similar characters, but bad at creating diversity. Part of this could have been because the phrase “space cowboy” may have thrown the model off. We wanted a diverse game, so we had to manually tweak character names and backstories to achieve this. The image prompts were basically useless. I’m sure there’s a way to craft solid image prompts from an OpenAI chat completion prompt, but we didn’t have the time to figure it out. So, we manually prompted Midjourney for each of our 20 NPCs, making use of their unique backstories and their names.

As we worked on the game, we deployed multiple contracts after testing, but the original set of characters remained the same. You can see them here.

We had our characters, their names, backstories, and their avatars. Now, we needed to truly bring them to life. We did this by turning them into NFTs with tokenbound accounts so that each NPC could own items in the game.

To the blockchain…and beyond

We leaned on Steve Simkins to handle the smart contract duties for our game. He quickly put together a custom ERC-721 contract, a custom ERC-20 contract, a custom ERC-6551 contract, and a custom registry contract to support our 6551 contract. He also wrapped all of this in a large operator contract that would support the core of our game logic.

Here’s a tl;dr of how it all works:

The operator contract accepted function calls from the admin wallet we created. The admin wallet minted and owned all NPC NFTs (721). The admin wallet then created a tokenbound account for each NPC (6551). Finally, the admin wallet minted the appropriate starting game items into each NPC’s tokenbound wallet.

This is a simplification of what was a lot of work in a short amount of time. Steve did an incredible job on this. If you’d like to see the hackathon version of the project’s codebase, it’s all here.

With financially capable NPCs, we now needed to bring those NPCs to life.

Bringing our NPCs to life

Our game world would be just as empty and boring as most large online gaming worlds are when they first start out if we didn’t give our NPCs some life. For the hackathon, we wanted to prove it could fun on its own to interact with NPCs while they also made decisions on their own without player input.

To do this, we turned to OpenAI’s chat completions. With some careful prompt crafting, we were able to provide enough details about each NPC’s current game state, the game’s backstory, and conversation history to make the NPCs feel real. We created a cron job that would run every hour and prompted the NPCs to each make a game decision based on their objectives and their current state. Through this process, we learned something interesting about OpenAI’s function calling API.

While we could provide functions that mapped to on-chain actions the NPCs could take to the API, we could not ensure the AI response would call one of those functions. For the sake of chatting with the NPCs, this worked great as you’ll see later. However, for the hourly decision-making process, we wanted NPCs to ALWAYS make a decision. We solved this by modifying our prompt and not using the function calling API. Instead, we fed the functions directly into the prompt with descriptions of what they did and asked the model to always select a function. From that selection response, we would trigger the on-chain action.

As mentioned above, we also needed to make our NPCs talk. To do this, we immediately chose XMTP. XMTP is a secure messaging protocol designed to work with messages sent wallet-to-wallet. Since each NPC has a tokenbound account, we decided to have players chat directly with an NPC through XMTP.

As with the decision-making prompt, we used OpenAI to handle chat prompts between the player and NPC. We fed the NPC’s name, backstory, and game state into the prompt and let it use that to help respond to the player. However, we took things a step further by allowing players to potentially influence NPC decisions. Even though the NPCs are autonomous, they respond to suggestions just like sentient beings. For this, we turned back to OpenAI’s function calling.

Because we didn’t want an NPC to always call a function, the unpredictable nature of whether the prompt would result in a function call or not made the experience feel more like you were talking to a real person. When a function call was executed, we would inform the player so that they could take some pride in influencing an NPC.

Again, like with the smart contracts, this is a simplified description of anything that went into bringing our NPCs to life. If you’re interested, the code from the hackathon is here.

The first and last part of the project was creating an amazing interface for interaction with these NPCs.

Terminally online

We turned to Marge, who has a history of designing projects that win hackathons, to take the entire concept and unify it through a UI that was fun and intuitive. We talked about various directions, but we ultimately landed on the concept of a government computer terminal specially designed to communicate across the galaxy through satellites.

With this design direction, Marge created a terminal-first interface that allowed players to chat with NPCs through a command prompt. Each NPCs information was presented and a player could choose which NPC to “engage” with. Once engaged, a player could send encrypted messages to the NPC. We also created an activity feed to showcase the decisions the NPC has previously made.

The game was fun, but it was missing something. We didn’t have a good way of seeing information at a glance. So, we created a leaderboard and a map view. The leaderboard was a simple view that sorted the NPCs based on how many credits they had in their wallet. It was a ton of fun watching the NPCs jockey for position by making their own on-chain decisions.

Since there were multiple locations an NPC could visit, the map view gave players a peek into where the NPCs were. Through this view, we learned something really interesting about our NPCs and their ability to understand the game logic.

After a couple of days, almost every NPC was at the supply depot. This makes logical sense. The supply depot is where all the action happens. But it was powerful seeing the NPCs arrive at this conclusion on their own.

Putting it all together

Impressively, the team managed to finish the entire app with enough time to spend almost a full week testing before submitting it to judges at EthGlobal. Through this testing, we recognized how fun the game was in its current state. We shared it with friends and family and watched how they interacted so that we could continue to make the game better. But at its core, it was working and it was ready.

When the hackathon began, we were going to optimize for sponsor prizes, but as we fell in love with the idea, we leaned into functionality that made sense and an experience that was great. So, we ignored the prize money and just built it for the people who would be playing our game. This strategy paid off in the end.

When submissions closed, we froze our code base and allowed ourselves a moment to breathe.

…And then just as quickly, it was time to present to judges.

We knew how we felt about the project, but we didn’t know what to expect from judges and others. The response was overwhelming. We made it past asynchronous judging, presented live to two judges and then made it to the finals as one of EthGlobals winners. Watching the reaction from judges, people watching the live demos in the finals, and on social media, it was clear that we had built something special.

In a little over two weeks, we went from idea to winning project at EthGlobal Online’s largest event ever.

Why does all of this matter?

When I was first learning to code, I leaned into writing video games. I would build simple games that my kids could play. I wrote games in JavaScript, C#, and Swift. Every one of them was simple and yet my kids wanted something more. They wanted a multiplayer world. And they aren’t alone. Many game devs want this, but building a multiplayer world isn’t just hard, it suffers from the cold start problem.

If you build a vast empty world, players are unlikely to stick around. The engine that powers our game proves that you can launch a brand new game populated with rich and interesting life that can keep players engaged.

Beyond the cold start problem, the game extends a time-honored tradition of game modding. From the original Wolfenstein 3D and Doom to Minecraft and Roblox today, gamers love to mod games. Our game creates a new model for modding. The backend is powered by a universal state machine (the blockchain). Any interface can be built on top of the existing game or future iterations of the game anytime. If you prefer an immersive 3D world of Cosmic Cowboys, you could build that and leverage all of the same NPC autonomy, game logic, and game state.

Finally, by leveraging IPFS for tracking the decision-making of NPCs, we can build better models in the future. We can expose the data to others who want to use it to create their own characters and worlds.

All of this is leading to an open world of development and gaming that aligns perfectly with what we’ve always believed was IPFS’s biggest strength.

What’s next

We plan to continue building Cosmic Cowboys and exploring the concepts of NPCs in game worlds that are powered by blockchain and AI. We’ve already begun work on an official release of the game that will have improved gameplay mechanics such as:

• Playable characters
• More urgency
• Leaderboards powered by more complex formulas
• And more…

We’re getting ready to ship invites for alpha and eventually beta access to the updated full game. If you want on the list, you can add your email at the below link.

Get early access

We’re also in the process of building a repeatable engine to help game developers leverage all of the same power we did when building this game. We’re excited to dive not just into the world of blockchain gaming but into a new world of intelligent NPCs that can make game development and gameplay richer.