Back to blog
How to Build an NFT Indexer with Ponder and Pinata
When it comes to building onchain apps, you can generally get away with querying the blockchain directly with a tool like Viem, but eventually you will encounter gaps. For example, we recently built CONCEALMINT as a tool for people to create and share private file NFTs, and one of the main pieces necessary was listing all the NFTs that user owned. That sounds like a simple task, but when you look at the functions on the smart contract, you start to realize it gets pretty messy. Your typical ERC721 contract will have the functions balanceOf, which will give you the total number of NFTs owned by an address, but not the tokenId of those NFTs. It also has ownerOf which takes in a tokenId and returns an address, but yet again you’re still stuck with the possibility of mapping over every NFT and adding way too much computation on your client. For CONCEALMINT we also wanted to have easy access to image links using our Pinata Gateway CDN plus image optimizations.
The solution here is an indexer: a server that listens for smart contract events and records the data in a database. Setting up an indexer manually can be a daunting task, however the folks over at ponder.sh make it delightfully simple and easy to build your own indexer and API to go along with it, all in one neat package. That’s exactly what we used for our app, and today we’ll show you how you can set one up yourself!
Setup
Before you start building this indexer, you’re going to need a few things, so let’s go over those real quick.
Pinata
First thing you’ll want to grab is a free Pinata account, which should only take a minute or two to sign up. It’s gonna come in handy for two pieces:
- Storing media and metadata on IPFS when minting NFTs
- Fetching content from IPFS and optimizing it using a Dedicated Gateway
Once you sign up for an account just follow these instructions to get your Pinata API key and your Gateway domain!
Deploy a Contract
Before we even started building CONCEALMINT we needed to deploy a simple ERC721 contract for our NFTs, so if you want a starting point check out the open source repo here. It uses Foundry and Open Zeppelin contract templates, and if you’re not exactly sure how to use it then check out this tutorial; it will also show you how to make a web app to mint them!
One last thing we would recommend here is to verify your contract before going to the next step. This is really easy to do if you get an Etherscan API key, as all you need to do after deploying a contract is running the following forge command:
forge v CONTRACT_ADDRESS src/Concealmint.sol:Concealmint -e ETHERSCAN_API_KEY --rpc-url RPC_URL
Ponder
Now comes the fun part where we start building our indexer! Ponder.sh makes it easy to launch a server that will track the events of a given contract and record your selected pieces of data into a database. In order for it to work, it needs a few things:
- Contract Address - This would be the contract address that you deployed.
- Contract ABI - ABI stands for Application Binary Interface, and it acts as a blueprint of how our code can be compiled and interact with the compiled byte code living on the blockchain. This is generated when you build and deploy a contract with Foundry.
- RPC URL - An EVM RPC URL is a node provider that allows you to subscribe and interact with a blockchain. It has some public RPCs built in, however for the speeds you would be using, it might be better to get one from a provider like Alchemy.
If you followed the last step of verifying the contract, almost all of this will be taken care of for you! First, you’ll want to look up your contract on a blockchain explorer. For instance, I used Base Sepolia so I’ll use https://sepolia.basescan.org. When I search for my contract address - 0x9B046Ca68dD4c7F7fC36A98DEdecf01C8EE904f7 - I get this link: https://sepolia.basescan.org/address/0x9B046Ca68dD4c7F7fC36A98DEdecf01C8EE904f7 . Then, in my terminal using pnpm, I can run this command:
pnpm create ponder --etherscan <https://sepolia.basescan.org/address/0x9B046Ca68dD4c7F7fC36A98DEdecf01C8EE904f7>
From there, you can give your project a name, and Ponder will take care of the rest. Once it’s complete, you’ll want to open the project in your text editor. Let’s take a look at the project structure:
├── abis
│ └── ConcealmintAbi.ts
├── package.json
├── pnpm-lock.yaml
├── ponder-env.d.ts
├── ponder.config.ts
├── ponder.schema.ts
├── src
│ └── Concealmint.ts
└── tsconfig.json
By using the --etherscan shortcut, Ponder has automatically put together our contract address, network/RPC info, and our ABI! It’s a really smooth, quick start that takes care of some of the heavy work.
One last item of housekeeping is our environment variables. Create a new file called .env.local and put in the following values:
PONDER_RPC_URL_84532= # This is optional, but we would recommend a provider like Alchemy or similar
CONTRACT_ADDRESS= # The contract addresss of your NFT collection
PINATA_JWT= # Your Pinata API Key JWT
GATEWAY_URL= # Your Pinata Dedicated Gateway domain in the format example.mypinata.cloud
Indexing
Now we can start adjusting and writing the code for our indexer. First up is some utils to help with our data fetching and writing to the database. One of those is Pinata for creating image links, so in the terminal install the IPFS SDK:
pnpm i pinata-web3
Then make a new folder in the root of the project called utils and inside it make a file called pinata.ts. In there, we’ll export an instance of our IPFS SDK:
import { PinataSDK } from "pinata-web3";
export const pinata = new PinataSDK({
pinataJwt: process.env.PINATA_JWT,
pinataGateway: process.env.GATEWAY_URL,
});
Next, we’re going to use Viem, which is already part of the dependencies for Ponder. Make another file in utils called viem.ts.
import { createPublicClient, http } from "viem";
import { baseSepolia } from "viem/chains";
export const publicClient = createPublicClient({
chain: baseSepolia,
transport: http(process.env.PONDER_RPC_URL_84532),
});
Finally, we’ll make one more file called types.ts with a type for our NFTs. This was defined by the metadata we used for CONCEALMINT NFTs, so adjust accordingly to your own.
export type NFT = {
token_id: string;
image: string;
name: string;
description: string;
file: string;
external_url: string;
};
Another thing we’re going to tweak is our ponder.config.ts file in the root of the project. Most of this will already be filled out, and the only thing we’re going to change is adding a pollingInterval of 2_000 which means it will listen to the blockchain at a rate of ones per every two seconds. This will help prevent your RPC bill from blowing up, or hitting a rate limit with a public RPC!
import { createConfig } from "ponder";
import { http } from "viem";
import { ConcealmintAbi } from "./abis/ConcealmintAbi";
export default createConfig({
networks: {
baseSepolia: {
chainId: 84532,
transport: http(process.env.PONDER_RPC_URL_84532),
pollingInterval: 2_000, // add this
},
},
contracts: {
Concealmint: {
abi: ConcealmintAbi,
address: process.env.CONTRACT_ADDRESS as `0x`,
network: "baseSepolia",
startBlock: 19169268,
},
},
});
There’s another Ponder file we need to edit, and that’s the ponder.schema.ts. This file will declare the structure of our database tables and how data is recorded. Copy and paste the schema we have below into your own schema file:
import { onchainTable } from "ponder";
export const account = onchainTable("account", (t) => ({
address: t.hex().primaryKey(),
}));
export const token = onchainTable("token", (t) => ({
id: t.bigint().primaryKey(),
owner: t.hex(),
token_uri: t.text(),
metadata: t.json(),
images: t.json(),
}));
export const transferEvent = onchainTable("transfer_event", (t) => ({
id: t.text().primaryKey(),
timestamp: t.integer().notNull(),
from: t.hex().notNull(),
to: t.hex().notNull(),
token: t.bigint().notNull(),
}));
Here, we just have three tables:
account- This will simply hold accounts that have interacted with our contracttoken- This will record info about our NFTs, including data like the owner, metadata, and image links that we’ll put together soontransfer_event- This will hold all the transfer events occurring on the contract with data we can use later, if necessary.
Moving on to our actual indexing (finally!). When you ran the initialization of the project, it should have generated a file under src/ContractName.ts , which will be named whatever your contract is named. Ours is Concealmint.ts, so let’s open that up and delete everything inside of it. Then put in the following code:
import { ponder } from "ponder:registry";
import schema from "ponder:schema";
import { publicClient } from "../utils/viem";
import { ConcealmintAbi } from "../abis/ConcealmintAbi"; // Update this according to your ABI file
import { pinata } from "../utils/pinata";
import type { NFT } from "../utils/types";
ponder.on("Concealmint:MetadataUpdate", async ({ event, context }) => {
const tokenURI = await publicClient.readContract({
address: process.env.CONTRACT_ADDRESS as `0x`,
abi: ConcealmintAbi,
functionName: "tokenURI",
args: [event.args._tokenId],
});
const { data: nftData } = await pinata.gateways.get(tokenURI as string);
const nft = nftData as unknown as NFT;
const baseImageURL = await pinata.gateways.convert(nft.image);
// Create or update a Token.
await context.db
.insert(schema.token)
.values({
id: event.args._tokenId,
token_uri: tokenURI,
metadata: {
name: nft.name,
description: nft.description,
external_url: nft.external_url,
image: nft.image,
file: nft.file,
},
images: {
original: baseImageURL,
thumbnail: `${baseImageURL}?img-width=200`,
sm: `${baseImageURL}?img-width=400`,
md: `${baseImageURL}?img-width=800`,
lg: `${baseImageURL}?img-width=1200`,
},
})
.onConflictDoUpdate({
// Update all fields except 'id' and 'owner'
token_uri: tokenURI,
metadata: {
name: nft.name,
description: nft.description,
external_url: nft.external_url,
image: nft.image,
file: nft.file,
},
images: {
original: baseImageURL,
thumbnail: `${baseImageURL}?img-width=200`,
sm: `${baseImageURL}?img-width=400`,
md: `${baseImageURL}?img-width=800`,
lg: `${baseImageURL}?img-width=1200`,
},
});
});
At the very top, we import a few pieces of code we’ll need. One that’s very important is our contract ABI. This should be already generated in the abi folder, so be sure to adjust to whatever your ABI variable is named. Then we import our schema, Pinata and Viem clients, and our type. The next thing you’ll notice is a function called ponder.on(), and this is going to be our listener on the contract that will be looking for the event passed in. In order for it to understand what events exists, it will use the contract ABI, which has all of that info included. Then, similar to a webhook, we can operate and run functions inside when the event is emitted from the contract!
The first event we track here is our "MetadataUpdate", which occurs when someone mints an NFT and the token URI is set. The event only emits one piece of information which is the token ID, and that doesn’t directly give us the information we need. So, to resolve that, we’re going to use our publicClient to read the tokenURI of the event.args._tokenId. This will return an IPFS URI like the one below:
ipfs://bafkreiaijh5mb3mntg32vmqgf5v4u3bijzl67pnqprxmupd3passg4xyhq
While this CID on its own doesn’t provide much info, we can use our pinata IPFS SDK instance to get the data inside.
const { data: nftData } = await pinata.gateways.get(tokenURI as string);
const nft = nftData as unknown as NFT;
Now we have access to all our NFT data! One of the pieces inside is going to be an image url which is also a CID. We want to save not only this CID, but some ready to use image urls as well. To make this happen we’ll create a base url that we can edit with image optimizations in our indexing. The IPFS SDK provides a handy method that will convert the IPFS reference into a Gateway URL with our Dedicated Gateway in our config.
const baseImageURL = await pinata.gateways.convert(nft.image);
This is where it gets fun. The time has come to start writing to our database, and Ponder makes this easy for us!
// Create or update a Token.
await context.db
.insert(schema.token)
.values({
id: event.args._tokenId,
token_uri: tokenURI,
metadata: {
name: nft.name,
description: nft.description,
external_url: nft.external_url,
image: nft.image,
file: nft.file,
},
images: {
original: baseImageURL,
thumbnail: `${baseImageURL}?img-width=200`,
sm: `${baseImageURL}?img-width=400`,
md: `${baseImageURL}?img-width=800`,
lg: `${baseImageURL}?img-width=1200`,
},
})
.onConflictDoUpdate({
// Update all fields except 'id' and 'owner'
token_uri: tokenURI,
metadata: {
name: nft.name,
description: nft.description,
external_url: nft.external_url,
image: nft.image,
file: nft.file,
},
images: {
original: baseImageURL,
thumbnail: `${baseImageURL}?img-width=200`,
sm: `${baseImageURL}?img-width=400`,
md: `${baseImageURL}?img-width=800`,
lg: `${baseImageURL}?img-width=1200`,
},
});
The context parameter from our ponder.on() will give us access to our Ponder database. With it, we can insert values into our tables we previous declared in the ponder.schema.ts. If you remember, we had a token table, so we’ll insert into that one. Inside of it, we’ll add all of our fields that we declared as part of our schema and our NFT type. The metadata and images are just JSON that we can make whatever we want, so for our NFTs, I’ve provided the raw metadata we grabbed from the tokenURI, then I also made several images that have different sizes ranging from the original to thumbnail to large sizes; all we had to do is add the image optimization queries! We also have a onClonflictUpdate that will handle the case where an NFT might already exist.
To get the info about who is the owner of these NFTs, we need to listen to the "Transfer" event as well. Add another function below the one we just made that will do this.
ponder.on("Concealmint:Transfer", async ({ event, context }) => {
// Create an Account for the sender, or update the balance if it already exists.
await context.db
.insert(schema.account)
.values({ address: event.args.from })
.onConflictDoNothing();
// Create an Account for the recipient, or update the balance if it already exists.
await context.db
.insert(schema.account)
.values({ address: event.args.to })
.onConflictDoNothing();
const tokenURI = await publicClient.readContract({
address: process.env.CONTRACT_ADDRESS as `0x`,
abi: ConcealmintAbi,
functionName: "tokenURI",
args: [event.args.tokenId],
});
const { data: nftData } = await pinata.gateways.get(tokenURI as string);
const nft = nftData as unknown as NFT;
const baseImageURL = await pinata.gateways.convert(nft.image);
// Create or update a Token.
await context.db
.insert(schema.token)
.values({
id: event.args.tokenId,
owner: event.args.to,
token_uri: tokenURI,
metadata: {
name: nft.name,
description: nft.description,
external_url: nft.external_url,
image: nft.image,
file: nft.file,
},
images: {
original: baseImageURL,
thumbnail: `${baseImageURL}?img-width=200`,
sm: `${baseImageURL}?img-width=400`,
md: `${baseImageURL}?img-width=800`,
lg: `${baseImageURL}?img-width=1200`,
},
})
.onConflictDoUpdate({ owner: event.args.to });
// Create a TransferEvent.
await context.db.insert(schema.transferEvent).values({
id: event.log.id,
from: event.args.from,
to: event.args.to,
token: event.args.tokenId,
timestamp: Number(event.block.timestamp),
});
});
This does almost the exact same thing as our MetadataUpdate event, except we specifically target the info revolving around who owns the NFT. This will handle the cases where a user mints the NFT or transfers it to another account. Just like that, our indexer is all set!
API
We’ve got our indexer and our database, but in order to use it with another client, we need an API. Thankfully, Ponder has us covered there too! Hono is built into Ponder and is a breeze to setup. First, make a folder inside of src called api, then add a index.ts file to it. Then put the following code inside it:
import { ponder } from "ponder:registry";
import { eq, desc } from "ponder";
import { token } from "ponder:schema";
ponder.get("/", (c) => {
return c.text("Hello, world!");
});
ponder.get("/nft", async (c) => {
// Handle auth here
const address = c.req.query("address");
if (address) {
const nftData = await c.db
.select()
.from(token)
.where(eq(token.owner, address))
.orderBy(desc(token.id));
const safeNfts = nftData.map((nft) => ({
...nft,
id: String(nft.id),
}));
return c.json({ nfts: safeNfts }, 200);
}
const nftData = await c.db.select().from(token).orderBy(desc(token.id));
const safeNfts = nftData.map((nft) => ({
...nft,
id: String(nft.id),
}));
return c.json({ nfts: safeNfts }, 200);
});
ponder.get("/nft/:id", async (c) => {
// Handle auth here
const tokenId = c.req.param("id");
const nftData = await c.db.select().from(token).where(eq(token.id, tokenId));
const safeNfts = nftData.map((nft) => ({
...nft,
id: String(nft.id),
}));
if (safeNfts.length === 0) {
return c.json({ error: "NFT not Found" }, 400);
}
return c.json(safeNfts[0], 200);
});
Inside this file, we’ll import ponder, which is our Hono instance here. We’ll also import our schema and some database helpers. Then it’s a simple matter of making our endpoints! The first one will simply select all of our NFTs from the token table and sort them by token.id and return them. There’s also an optional query of address where we can only return the NFTs that are owned by a given address. The next endpoint is similar, except it uses a path parameter of id so we can find a particular NFT. That’s it! You’ll notice in comments I recommend protecting these endpoints with and authentication mechanism, but I left them out here for simplicity.
Now it’s time to test it out. 👀 In the terminal, run the following:
pnpm dev
This is going to spin up the indexer and backfill the contract events from when the first block is set in the ponder.config.ts file. Once it has all the events filled, it will continue to listen for them, and it also means the API will be ready to test. The default port is 42069, so if you open up http://localhost:42069 in your browser, then you should get the “hello world” greeting. If you have NFTs minted, then you can try using http://localhost:42069/nft and get something like this!
{
"nfts": [
{
"id": "1",
"owner": "0x4a290f18c35bbfe97b2557cf765de9387726de39",
"token_uri": "ipfs://bafkreideky4ghk5zawfwn3pnxozferlpyquglxw62rprz57raivgxmy65u",
"metadata": {
"name": "guess",
"description": "this is a clue",
"external_url": "<https://warpcast.com/m-j-r.eth>",
"image": "ipfs://bafkreib2nsjrl7ysprz4eaqmzkd7gmuti4viddgszj6wjnp6y7dalkl7aa",
"file": "bafkreicihwndxrhwhzacr46wjepruiixdmdrtso5tluetsr62q3yhl3ucq"
},
"images": {
"original": "<https://dweb.mypinata.cloud/ipfs/bafkreib2nsjrl7ysprz4eaqmzkd7gmuti4viddgszj6wjnp6y7dalkl7aa>",
"thumbnail": "<https://dweb.mypinata.cloud/ipfs/bafkreib2nsjrl7ysprz4eaqmzkd7gmuti4viddgszj6wjnp6y7dalkl7aa?img-width=200>",
"sm": "<https://dweb.mypinata.cloud/ipfs/bafkreib2nsjrl7ysprz4eaqmzkd7gmuti4viddgszj6wjnp6y7dalkl7aa?img-width=400>",
"md": "<https://dweb.mypinata.cloud/ipfs/bafkreib2nsjrl7ysprz4eaqmzkd7gmuti4viddgszj6wjnp6y7dalkl7aa?img-width=800>",
"lg": "<https://dweb.mypinata.cloud/ipfs/bafkreib2nsjrl7ysprz4eaqmzkd7gmuti4viddgszj6wjnp6y7dalkl7aa?img-width=1200>"
}
]
}
}
If you’d like to see the final code of the indexer we built for CONCEALMINT, check it out here!
Wrapping Up
There’s plenty more we could do here to make our API even more powerful, like adding more chains and contracts, or use Gateway Access Controls to access IPFS content outside your account to build a marketplace index — the possibilities are endless!
Pinata is here to make your IPFS needs simple, so don’t wait to try it out and sign up today; we can’t wait to see what you build!