While you’re using IPFS for your crypto application, you often have to use more traditional solutions for other parts of your stack. With Pinata’s Private IPFS solution, you can use the content addressability of IPFS while keeping data private for at least one part of that stack—logging.

Logging is a critical part of any production application, but it can get expensive fast. However, when you look at the core of what logging is, it seems surprisingly like file storage. You might log errors or important data as your app is being used. These data points are generally text or JSON format, perfect for file storage. What if we built a logging solution that used your existing Pinata account but sent the logs to Private IPFS? And what if those logs were every bit as queryable as more traditional logging providers?

We’re not going to be building a full application in this tutorial, so the structure is going to be a little different. How you handle logging and the places you choose to capture logs are a decision you have to make based on your needs. Instead, what we’ll do is write the code that will send your logs to Private IPFS on Pinata in a way you can query and make use of.

Getting Started

For this tutorial, you’ll need a free Pinata account. You can sign up here. You’re already writing code and building crypto apps, so I assume you have a code editor and, well, code where you’ll do the logging.

For this project, I’m going to write my code in JavaScript, but will also include cURL versions to make it easier for you to convert it to the programming language of your choice. You’ll need a Pinata API key, though. You can generate that by logging in and going to the API Keys page. We’ll need the JWT that is created.

Logging Errors

For logging errors, you would likely create a function dedicated to that in your code. In JavaScript, that might look like:

const logError = async (userData, error, function) = > {
	const fullError = new Blob([JSON.stringify(error)], { type: 'application/json' });
	
	const form = new FormData();
	form.append("name", Date.now());
	form.append("keyvalues", JSON.stringify({
		userId: userData.id, 
		errorCode: error.statusCode, 
		function: function
	}));
	form.append("file", fullError)
	
	const options = {
  method: 'POST',
  headers: {
    Authorization: 'Bearer <PINATA JWT>' 
  },
  body: form
};
	
	await fetch(`https://uploads.pinata.cloud/v3/file`, options)
}

Here’s the cURL version:

curl --request POST \\
  --url <https://uploads.pinata.cloud/v3/file> \\
  --header "Authorization: Bearer <PINATA JWT>" \\
  --form "name=$(date +%s)" \\
  --form "keyvalues={\\"userId\\":\\"<user-id>\\",\\"errorCode\\":\\"<error-code>\\",\\"function\\":\\"<function-name>\\"}" \\
  --form "[email protected]"

This function uploads the raw error object but it also includes key-value metadata which makes it easier to query later. For example, if you wanted to find all the errors that have happened with a single function, you could write a function like:

const fetchErrorsByFunction = async(function) => {
	const res = await fetch(`https://api.pinata.cloud/v3/files?metadata[function]=${function}`, {
		method: "GET",
		headers: {
	    Authorization: 'Bearer <PINATA JWT>' 
	  }
	}
	
	const functionErrors = await res.json()
}

And here’s the cURL version of that:

curl --request GET \\
  --url '<https://api.pinata.cloud/v3/files?metadata[function]=><function>' \\
  --header "Authorization: Bearer <PINATA JWT>"

This will get your a paginated list of errors for the function you specified. Of course, “function” could be API route or whatever you want to provide when you’re logging. You can combine queries as well. So, if you want to query for all errors for a specific user and a specific function, you can combine your query parameters like this:

<https://api.pinata.cloud/v3/files?metadata[function]=${function}&metadata[userId]=${userId}>

This is simple methodology can get you as much or as little data as you need. By querying this, you get data points on error frequency, functions, users, whatever. But you also get the raw error. The GET request will return CIDs for each error. You can load the actual error and inspect it whenever you want by using your Pinata Gateway.

Logging Messages

To log arbitrary messages, you can take a similar approach.

const logMessage = async (userData, message, source) = > {
	const fullError = new Blob([message], { type: 'text/plain' });
	
	const form = new FormData();
	form.append("name", Date.now());
	form.append("keyvalues", JSON.stringify({
		userId: userData.id, 
		source: source
	}));
	form.append("file", fullError)
	
	const options = {
  method: 'POST',
  headers: {
    Authorization: 'Bearer <PINATA JWT>' 
  },
  body: form
};
	
	await fetch(`https://uploads.pinata.cloud/v3/file`, options)
}

Here’s the cURL version:

curl --request POST \\
  --url <https://uploads.pinata.cloud/v3/file> \\
  --header "Authorization: Bearer <PINATA JWT>" \\
  --form "name=$(date +%s)" \\
  --form "keyvalues={\\"userId\\":\\"<user-id>\\",\\"source\\":\\"<source>\\"}" \\
  --form "file=@-;filename=message.txt" <<< "<message>"

In the message example, we are simplifying things a bit. The message is a string—could be stringified JSON or an actual text string. We are capturing the source of the message generation as well as the user id. This all works similarly to capturing errors.

You can query these messages the same way. One thing you might want to do to differentiate between errors and messages is add another key-value pair like this:

JSON.stringify({
		userId: userData.id, 
		source: source, 
		type: "message"
	})

Or:

JSON.stringify({
		userId: userData.id, 
		function: function, 
		type: "error"
	})

But with these two functions, you can capture your apps logs using Private IPFS.

Benefits of Private IPFS

The benefits of this approach is you are already (hopefully) using Pinata for your public IPFS files, but it also includes built-in de-duplication. Errors happen over and over again. You don’t need to store a file for each error unless there’s additional metadata you want to include.

Private IPFS, when exposed over a peer-to-peer connection with other permissioned nodes can allow error logging on edge devices as well. Peer to peer connections to Private IPFS are available by custom agreement, so contact us if you’re interested.

Conclusion

Logging solutions are often expensive and force you to add another tool to your stack. If you’re building in crypto and using IPFS, why not use Private IPFS for your logging solution. There are many ways to extend to simple examples in this tutorial to make for a robust and practical logging system.

Happy private pinning!