Back to blog
Backing Up Arbitrum Event Data to IPFS using Ponder and Pinata
As blockchains evolve and gain popularity they always encounter the issue of scale. Each one has its own solution to the problem, and some chains are born as an effort to help scale Ethereum. One of those chains is Arbitrum, an optimistic rollup that’s fast, cheap, and boasts some special features. Despite the progress of blockchains and scaling them, there is still the problem of space. Everything that happens onchain is recorded to the ledger, and over time that causes the entire blockchain to grow into a massive database that becomes harder to store on nodes. One solution that has been discussed here before is storing event data on IPFS so it can be pruned and save space. If you’re not familiar, events are logs of data emitted by smart contracts and can be used for all sorts of helpful applications like tracking who transferred a token or an NFT to another person.
In this post we’ll be going over the same topic, but using a much more specific toolchain that’s used for indexing data. With it you could store and backup your own contract event data and proposes how you could rebuild a database of events using that backup data on IPFS. To demonstrate we’ve build Pinnie’s Guestbook, a simple smart contract and app deployed to Arbitrum One that let’s people leave a message + a gif. This won’t necessarily be an end to end tutorial but will show you all the pieces and how backing up events to IPFS is easy. Let’s check it out!
Contract Setup
Let’s take a quick look at the smart contract for our guestbook app.
// SPDX-License-Identifier: MIT
pragma solidity ^0.8.20;
/**
* @title Guestbook
* @dev A simple onchain guestbook where users can sign with their wallet and leave a message
*/
contract Guestbook {
// Event emitted when a new guestbook entry is created
event NewEntry(address indexed signer, string message, string imageUrl, uint256 timestamp);
// Optional: track total number of entries
uint256 public totalEntries;
/**
* @dev Sign the guestbook with a message
* @param message The message to leave in the guestbook
* @param imageUrl Image reference to leave in the guestbook
*/
function signGuestbook(string calldata message, string calldata imageUrl) external {
// Emit the event with signer address, message, and timestamp
emit NewEntry(msg.sender, message, imageUrl, block.timestamp);
// Increment total entries
totalEntries += 1;
}
/**
* @dev Get the total number of entries in the guestbook
* @return The total number of entries
*/
function getEntryCount() external view returns (uint256) {
return totalEntries;
}
}
The guestbook is made of a few simple pieces
- An event called
NewEntrywhich logs who made the message, the message itself, an image URL, and a timestamp - A state to track the
totalEntries - Our main function
signGuestbookwhich takes in themessageandimageUrland emits theNewEntryevent and increments thetotalEntries - A
getEntryCountfunction to read the total number of posts
Nothing too special, but the power is that we can index those events and build a history onchain of who signed the guestbook. By using events we can avoid storing that data within the contract which would get more and more expensive as the guestbook entries grew.
Indexer Setup
With our contract deployed to Arbitrum One and accessible to write events to it we can now setup our indexer. We’ll be using Ponder for this example as it is fairly easy to use and flexible with how you can handle the data. Let’s take a look at the main indexing file Guestbook.ts.
import { ponder } from "ponder:registry";
import schema from "ponder:schema";
import { pinata } from "./pinata";
import { fetchWeb3BioProfile, extractProfileInfo } from "./web3bio";
ponder.on("Guestbook:NewEntry", async ({ event, context }) => {
const { signer, message, imageUrl, timestamp } = event.args;
// Generate a unique ID for this entry
const id = `${event.transaction.hash}-${event.log.logIndex}`;
// Fetch Web3.bio profile data
const profiles = await fetchWeb3BioProfile(signer);
const profileInfo = extractProfileInfo(profiles);
// Store or update account information using the proper upsert pattern
await context.db
.insert(schema.account)
.values({
address: signer,
farcasterName: profileInfo.farcasterName,
farcasterDisplayName: profileInfo.farcasterDisplayName,
farcasterAvatar: profileInfo.farcasterAvatar,
farcasterDescription: profileInfo.farcasterDescription,
farcasterFollowers: profileInfo.farcasterFollowers,
ensName: profileInfo.ensName,
lensHandle: profileInfo.lensHandle,
lastUpdated: Number(timestamp),
})
.onConflictDoUpdate(() => ({
farcasterName: profileInfo.farcasterName,
farcasterDisplayName: profileInfo.farcasterDisplayName,
farcasterAvatar: profileInfo.farcasterAvatar,
farcasterDescription: profileInfo.farcasterDescription,
farcasterFollowers: profileInfo.farcasterFollowers,
ensName: profileInfo.ensName,
lensHandle: profileInfo.lensHandle,
lastUpdated: Number(timestamp),
}));
// Store the guestbook entry
await context.db.insert(schema.guestbookEntry).values({
id,
signer,
message,
imageUrl,
timestamp: Number(timestamp),
accountId: signer,
});
function stringifyWithBigInt(value: any) {
return JSON.stringify(value, (_, v) =>
typeof v === 'bigint' ? `${v}n` : v
);
}
const serializedEvent = stringifyWithBigInt(event);
const blob = new Blob([serializedEvent])
const file = new File([blob], "event.json", { type: "application/json" })
const store = await pinata.upload.public.file(file)
.keyvalues({
id,
signer,
message,
imageUrl,
timestamp: Number(timestamp).toString(),
accountId: signer,
})
.group("9003e1ad-b0d9-45b2-baed-7baae19781a3")
// Used to parse event.json to restore BigInt
// function parse(text) {
// return JSON.parse(text, (_, value) => {
// if (typeof value === 'string') {
// const m = value.match(/(-?\\d+)n/);
// if (m && m[0] === value) {
// value = BigInt(m[1]);
// }
// }
// return value;
// });
// }
console.log("Event stored: ", store.cid)
});
There’s a fair bit of code here to handle our database functionality, but let’s break it down piece by piece. The main function being run here is the following:
ponder.on("Guestbook:NewEntry", async ({ event, context }) => {
// processing...
})
Ponder will use an RPC to listen to the contract we have deployed for any events that match NewEntry just as we saw in our contract code just a moment ago. Any time that event is fired then it will run the code inside, giving us access to the event data as well as the Ponder context which includes some other important information. The next piece is what we do with that initial data.
const { signer, message, imageUrl, timestamp } = event.args;
// Generate a unique ID for this entry
const id = `${event.transaction.hash}-${event.log.logIndex}`;
// Fetch Web3.bio profile data
const profiles = await fetchWeb3BioProfile(signer);
const profileInfo = extractProfileInfo(profiles);
// Store or update account information using the proper upsert pattern
await context.db
.insert(schema.account)
.values({
address: signer,
farcasterName: profileInfo.farcasterName,
farcasterDisplayName: profileInfo.farcasterDisplayName,
farcasterAvatar: profileInfo.farcasterAvatar,
farcasterDescription: profileInfo.farcasterDescription,
farcasterFollowers: profileInfo.farcasterFollowers,
ensName: profileInfo.ensName,
lensHandle: profileInfo.lensHandle,
lastUpdated: Number(timestamp),
})
.onConflictDoUpdate(() => ({
farcasterName: profileInfo.farcasterName,
farcasterDisplayName: profileInfo.farcasterDisplayName,
farcasterAvatar: profileInfo.farcasterAvatar,
farcasterDescription: profileInfo.farcasterDescription,
farcasterFollowers: profileInfo.farcasterFollowers,
ensName: profileInfo.ensName,
lensHandle: profileInfo.lensHandle,
lastUpdated: Number(timestamp),
}));
// Store the guestbook entry
await context.db.insert(schema.guestbookEntry).values({
id,
signer,
message,
imageUrl,
timestamp: Number(timestamp),
accountId: signer,
});
Here we parse the event.args which is the data we passed into the event from our contract. With it we can build some pretty cool info by using Web3Bio to fetch social data with the wallet address. Once we have that user info and the event data we can insert it into the database that comes with Ponder, which we can use to make API calls to later and query the data. Now let’s look at backing up that data.
function stringifyWithBigInt(value: any) {
return JSON.stringify(value, (_, v) =>
typeof v === 'bigint' ? `${v}n` : v
);
}
const serializedEvent = stringifyWithBigInt(event);
const blob = new Blob([serializedEvent])
const file = new File([blob], "event.json", { type: "application/json" })
const store = await pinata.upload.public.file(file)
.keyvalues({
id,
signer,
message,
imageUrl,
timestamp: Number(timestamp).toString(),
accountId: signer,
})
.group("9003e1ad-b0d9-45b2-baed-7baae19781a3")
// Used to parse event.json to restore BigInt
// function parse(text) {
// return JSON.parse(text, (_, value) => {
// if (typeof value === 'string') {
// const m = value.match(/(-?\\d+)n/);
// if (m && m[0] === value) {
// value = BigInt(m[1]);
// }
// }
// return value;
// });
// }
At the top we have a function that handles a problem as old as time; handling BigInt data types in JSON 🥲 There’s a lot of ways you can go about this, including storing the files in different formats since IPFS supports any kind of file. Anyway you slice it, you can feed that data into a File object and then upload it to IPFS through Pinata in just a few lines of code. For convenience we also stored some keyvalues that we can use to query and find data using Pinata’s API. We’ll also store it to a group to make it easier to manage the files in our account. At the end we have an example on how you could restore the backup data that’s been modified to handle BigInts.
In our example here we have both a database which is “hot” and we can query easily, but we also have backups on IPFS which are open, accessible, and could be used to rebuild our database if we wanted to. Ponder includes an API through Hono where we can return the events stored.
app.get("/entries", async (c) => {
const entries = await db
.select({
id: schema.guestbookEntry.id,
signer: schema.guestbookEntry.signer,
message: schema.guestbookEntry.message,
imageUrl: schema.guestbookEntry.imageUrl,
timestamp: schema.guestbookEntry.timestamp,
farcasterName: schema.account.farcasterName,
farcasterDisplayName: schema.account.farcasterDisplayName,
farcasterAvatar: schema.account.farcasterAvatar,
ensName: schema.account.ensName,
lensHandle: schema.account.lensHandle,
})
.from(schema.guestbookEntry)
.leftJoin(schema.account, eq(schema.guestbookEntry.signer, schema.account.address))
.orderBy(desc(schema.guestbookEntry.timestamp))
return c.json(entries);
});
Super simple but makes for a great little app!
Building the Guestbook App
For the app we chose a simple Vite + React client, and almost all of it happens inside the App.tsx file.
import { useState, useEffect, FormEvent, useCallback } from "react"
import { Card, CardContent, CardDescription, CardFooter, CardHeader, CardTitle } from "./components/ui/card";
import { Input } from "./components/ui/input";
import { Button } from "./components/ui/button";
import { NotebookPenIcon } from "lucide-react";
import { BaseError, useAccount, useConnect, useDisconnect, useWaitForTransactionReceipt, useWriteContract} from 'wagmi';
import { abi, CONTRACT_ADDRESS } from './lib/contract';
import { arbitrum } from "viem/chains";
import { Avatar, AvatarFallback, AvatarImage } from "./components/ui/avatar";
import { GifSearch } from "./components/GifSearch";
import { toast } from "sonner"
import pinnie from "./assets/pinnie.svg"
interface GuestbookEntry {
id: string;
signer: string;
message: string;
imageUrl: string;
timestamp: number;
farcasterName?: string;
farcasterDisplayName?: string;
farcasterAvatar?: string;
ensName?: string;
lensHandle?: string;
}
function App() {
const [entries, setEntries] = useState<GuestbookEntry[]>([]);
const [isLoading, setIsLoading] = useState<boolean>(true);
const [message, setMessage] = useState<string>('');
const [imageUrl, setImageUrl] = useState<string>('');
const [isSubmitting, setIsSubmitting] = useState(false);
// Wallet connection states
const { address, isConnected, } = useAccount();
const { connect, connectors } = useConnect();
const { disconnect } = useDisconnect();
// Contract interaction states
const { data: hash, error, isPending, writeContract } = useWriteContract();
const { isLoading: isConfirming, isSuccess: isConfirmed } =
useWaitForTransactionReceipt({
hash,
});
async function connectWallet(){
connect({ connector: connectors[0], chainId: arbitrum.id})
}
const fetchEntries = useCallback(async () => {
setIsLoading(true);
try {
const response = await fetch(`${import.meta.env.VITE_SERVER_URL}/entries`);
const data = await response.json();
setEntries(data);
} catch (error) {
console.error('Error fetching entries:', error);
toast.error("Failed to load guestbook entries");
} finally {
setIsLoading(false);
}
}, []);
useEffect(() => {
if (hash && isConfirming) {
toast.loading("Waiting for transaction confirmation...", {
id: "tx-confirming",
});
}
}, [hash, isConfirming]);
useEffect(() => {
if (isConfirmed && hash) {
toast.dismiss("tx-confirming"); // Dismiss the loading toast
toast.success("Your message has been added to the guestbook!");
// After confirmation, fetch the updated entries once
setTimeout(() => {
fetchEntries();
setIsSubmitting(false);
}, 2000); // Wait 2 seconds for indexer to process
}
}, [isConfirmed, hash, fetchEntries]);
useEffect(() => {
if (isPending) {
setIsSubmitting(true);
}
}, [isPending]);
useEffect(() => {
if (error) {
toast.error("Transaction Error", {
description: (error as BaseError).shortMessage || error.message,
});
}
}, [error]);
useEffect(() => {
fetchEntries()
},[fetchEntries])
async function submitEntry(e: FormEvent) {
e.preventDefault();
if (!message || !isConnected) return;
try {
const submittedMessage = message;
const submittedImageUrl = imageUrl;
writeContract({
address: CONTRACT_ADDRESS,
abi: abi,
functionName: 'signGuestbook',
args: [submittedMessage, submittedImageUrl],
});
setMessage('');
setImageUrl('');
} catch (error) {
console.error('Error submitting entry:', error);
toast.error("Failed to submit your guestbook entry");
setIsSubmitting(false);
}
}
const getDisplayName = (entry: GuestbookEntry) => {
if (entry.farcasterName) return entry.farcasterName;
if (entry.farcasterDisplayName) return entry.farcasterDisplayName;
if (entry.ensName) return entry.ensName;
return `${entry.signer.substring(0, 6)}...${entry.signer.substring(entry.signer.length - 4)}`;
}
return (
<div className="mx-auto px-4 py-8 max-w-xl">
<img className="w-24 mx-auto" src={pinnie} alt="Pinnie" />
<h1 className="sm:text-5xl text-4xl font-[900] mb-6 text-center">Pinnie's Guestbook</h1>
{connectors.length > 0 ? (
<>
<div className="mb-4 text-center">
{isConnected ? (
<div className="flex flex-col items-center gap-2">
<p className="text-sm font-mono p-2">
{address?.substring(0, 6)}...{address?.substring(address.length - 4)}
</p>
<Button
variant="outline"
size="sm"
onClick={() => disconnect()}
>
Disconnect
</Button>
</div>
) : (
<Button
onClick={connectWallet}
>
Connect Wallet
</Button>
)}
</div>
<Card className="mb-8">
<form onSubmit={submitEntry}>
<CardHeader className="mb-4">
<CardTitle>Leave a message</CardTitle>
<CardDescription>Share your thoughts with others</CardDescription>
</CardHeader>
<CardContent className="space-y-4">
<div>
<Input
placeholder="Your message..."
value={message}
onChange={(e) => setMessage(e.target.value)}
className="w-full"
required
disabled={isSubmitting}
/>
</div>
<div>
<GifSearch
onSelect={(url) => setImageUrl(url)}
selectedGif={imageUrl || null}
/>
</div>
</CardContent>
<CardFooter className="flex justify-end my-4">
<Button
type="submit"
disabled={!message || !isConnected || isPending || isConfirming || isSubmitting}
>
{isPending || isConfirming ? 'Processing...' : 'Sign Guestbook'}
</Button>
</CardFooter>
</form>
</Card>
</>
) : (
<div className="text-center text-muted-foreground mb-4 text-sm">
<p>Leave a message by visiting inside Farcaster or connecting with a wallet enabled browser!</p>
</div>
)}
<div className="space-y-4 relative">
{isLoading ? (
<div className="flex items-center justify-center py-8">
<NotebookPenIcon className="animate-bounce h-8 w-8" />
</div>
) : entries.length === 0 ? (
<p className="text-center py-4">No entries yet. Be the first to sign!</p>
) : (
// Just render the confirmed entries
entries.map((entry) => (
<Card key={entry.id} className="mb-4">
<CardHeader>
<div className="flex items-center gap-3">
<Avatar>
{entry.farcasterAvatar ? (
<AvatarImage src={entry.farcasterAvatar} alt={getDisplayName(entry)} />
) : (
<AvatarFallback>
{getDisplayName(entry).substring(0, 2).toUpperCase()}
</AvatarFallback>
)}
</Avatar>
<div>
<CardTitle className="text-lg">
{getDisplayName(entry)}
</CardTitle>
<CardDescription>
{new Date(entry.timestamp * 1000).toLocaleString()}
</CardDescription>
</div>
</div>
</CardHeader>
<CardContent>
<p>{entry.message}</p>
{entry.imageUrl && (
<div className="mt-4">
<img
src={entry.imageUrl}
alt="Entry image"
className="max-h-64 rounded-md mx-auto"
onError={(e) => e.currentTarget.style.display = 'none'}
/>
</div>
)}
</CardContent>
</Card>
))
)}
</div>
</div>
);
}
export default App
Lot of code there, but in short it does the following:
- Fetches from our server API to get the entries and map them out as cards
- Allows users to connect their wallet
- Users can write a message with their wallet by interacting with the contract on Arbitrum One
- After submitting the indexer will get the new event and store it in the database
As a result we have a pretty clean app that includes onchain social data from Farcaster! (Works as a miniapp too by the way)
If you’re interested to check out these apps we’ve got the repos linked below which are all open sourced!
PinataCloud
PinataCloud
PinataCloud
Wrapping Up
As blockchains scale and more data needs to be preserved, we should start thinking of ways to make it more accessible and verifiable. If we followed this same flow but backed up to a centralized S3 bucket then it’s no longer publicly accessible, verifiable, or persistent. With IPFS you get all those benefits, where every member of the community can have access to that event data as well as help pin it to keep it accessible and decentralized. Pinata helps make pinning at scale possible, and makes it effortless for teams to implement. Give it a shot today!