Will JSON Work As a Database?

The idea of JSON as a database isn’t new. NoSQL databases are effectively just this. However, JSON in its simplest form is a lightweight file that can be hosted anywhere. It doesn’t need a database service provider. Any storage provider will work.

The idea of JSON as your main data store is a powerful one for single-player apps, especially if those apps are local-first. Many local-first apps use the browser’s built-in IndexedDB to store data and persist state. IndexedDB is essentially a JSON document in the browser. This means that, when it’s time to sync your local state with a remote server so that you can access your data on another device, sending a JSON file is dead simple.

An Example

Let’s look at a quick example of how you can load data from IndexedDB using a library called Dexie and write it to Pinata using the Pinata SDK as a single JSON upload.

// Initialize the Dexie database
const db = new Dexie("myDatabase");

// Define the schema for the object stores
db.version(1).stores({
  users: "id, name",
  orders: "id, user_id, total"
});

// Load all data into a single JSON object
async function syncToPinata() {
  const allData = {};

  // Use Dexie's table method to access the object stores
  const storeNames = db.tables.map(table => table.name);

  // Fetch data from each store
  for (const storeName of storeNames) {
    allData[storeName] = await db.table(storeName).toArray();
  }

	//  Send the data to Pinata as a JSON file
  await pinata.upload.json(allData)
}

syncToPinata();

In just a few lines of code, you have a minimal sync service. All thanks to JSON. But how do you load the data from a remote source and populate the local IndexedDB? It’s just as easy.

const db = new Dexie("myDatabase");

// Define the schema
db.version(1).stores({
  users: 'id, name',
  orders: 'id, user_id, total'
});

async function loadFromRemote() {
	const remoteData = await pinata.gateways.get("CID") // CID is the identifier for your remote file
	try {
    // Iterate over each key in the JSON object
    for (const [storeName, data] of Object.entries(remoteData)) {
      // Insert the data into the corresponding object store
      await db.table(storeName).bulkPut(data);
    }
    console.log('Database populated successfully!');
  } catch (error) {
    console.error('Error populating database:', error);
  }
}

loadFromRemote()

This is a simple seeding or remote sync script that will populate the new client with the data needed to run the app. All with a single request for a JSON file stored privately with Pinata.

This is a powerful use case for JSON, but we’re left with one final question. How can we get the correct remote file when populating or syncing the database? There are a variety of options, but the simplest might be to put your synced JSON into a group using Pinata’s Groups feature.

const upload = await pinata.upload
  .json(allData)
  .group(GROUP_ID)

Then, you can list the files in your group and grab the most recent one:

const files = await pinata.files
  .list()
  .limit(10)
  
const groupedFiles = files.filter(f => f.group_id === GROUP_ID)
const newestData = groupedFiles[0]
const cid = newestData.cid

With that CID, you can load the JSON, like in the earlier example, and your local app state will be synced with the remote server state.

Going Further

This is a very high-level look at what you can do with JSON as a database. If you want to dive deeper, you should look into libraries that allow you to query your JSON data. Dexie does a great job of this if your data is in IndexedDB, but if you want to query in-memory on the JSON returned from Pinata, for example, you might look into the following libraries:

• json-query
• JSONata
• GROQ-JS
• Jora

Conclusion

JSON is a powerful data format, but it’s also a lightweight option for storage and data replication. It can be used as a primary database in some cases, as a syncing tool, or for populating local-first apps.

If you’re ready to get started with JSON as a database (or for any other use, really), Pinata makes it easy to upload, protect, and retrieve your files.