Generate files CID without connecting to node

[jacob-jarick]

Greetings, I have been testing richard schneider's librarys as Id like to generate a CID that matches the IPFS clients CID without connecting to an IPFS node and storing the file (it will serve as a reference only),

I have tried net-ipfs-http-client with success but it does seem to connect to a remote HTTP api http://ipv4.fiddler:5001 if no URL is provided (I may be mistaken).

Below is my working code that uses the embedded daemon which works but as mentioned I do not want a IPFS daemon running on the same server that this C# code is running.

using Ipfs.CoreApi;
using Ipfs.Engine;

var ipfs = new IpfsEngine("this is not a secure pass phrase".ToCharArray());

for (int i = 0; i < 255; i++)
{
  var options = new AddFileOptions { OnlyHash = true };
  var fsn = await ipfs.FileSystem.AddTextAsync("hello world " + i.ToString(), options);
  Console.WriteLine((string)fsn.Id);
}

The code below shows I have had success generating a multihash but I cannot figure out how to wrap the file in UnixFS.
Ideally Id add an additional step and get the expected CID: Qmf412jQZiuVUtdgnB36FXFX7xg5V6KEbSJ4dpQuhkLyfD

byte[] prefix = { 0x12, 0x20 }; // 0x12 == sha256, 0x20 == size (always same size for sha256)
const string data = "hello world";

// multihash native calculation.
var BinData = Encoding.UTF8.GetBytes(data);
var sha256 = SHA256.Create();
var hash = sha256.ComputeHash(BinData);
var combined = prefix.Concat(hash).ToArray();
string result = Base58.Encode(combined);
Console.WriteLine("result: " + result);

Thanks

[Arlodotexe]

If you need to start using IPFS in your apps today, I would recommend using the downloader and bootstrapper in the OwlCore.Kubo library until a WASI implementation of IPFS is available.

Bringing net-ipfs-engine up to par with Kubo would take an extraordinary amount of time and effort that we (the community) aren't able to provide right now.

---

As for wrapping in UnixFS objects, it looks like under both Go and JS, they've created separate packages for this:

• https://github.com/ipfs/go-unixfs
• https://github.com/ipfs/js-ipfs-unixfs

Because interacting with UnixFS is an implementation detail rather than an API or model, code for this would be in net-ipfs-engine, instead of net-ipfs-core or net-ipfs-http-client.

We don't have a package for this, but you might be able to borrow code from here to build what you need without running an IPFS daemon.

[Arlodotexe]

The easy way to generate a Multihash/Cid from a byte array:

var multihash = MultiHash.ComputeHash(bytes);

This type has an implicit conversion between MultiHash and Cid, so simply use one in place of the other:

// Create multihash from byte array
var multihash = MultiHash.ComputeHash(bytes);

// Implicit cast to Cid
MethodThatNeedsCID(multihash);

void MethodThatNeedsCID(Cid contentId)
{
}

[jacob-jarick]

I have tried casting the output of MultiHash.ComputeHash(bytes) to Cid, I get the same result as if I didnt cast it.

  string test1 = "hello world";
  var bytes = Encoding.ASCII.GetBytes(test1);

  var options = new AddFileOptions { OnlyHash = true };
  var fsn = await ipfs.FileSystem.AddTextAsync(test1, options);
  Console.WriteLine("'" + test1 + "' ipfs.FileSystem.AddTextAsync = " + (string)fsn.Id);

  Cid cid = (Cid) MultiHash.ComputeHash(bytes);
  Console.WriteLine("'" + test1 + "MultiHash.ComputeHash casted to Cid = " + cid.ToString());

  var multihash = MultiHash.ComputeHash(bytes);
  Console.WriteLine("'" + test1 + "' MultiHash.ComputeHash = " + multihash.ToString());

output:

'hello world' ipfs.FileSystem.AddTextAsync = Qmf412jQZiuVUtdgnB36FXFX7xg5V6KEbSJ4dpQuhkLyfD
'hello world' MultiHash.ComputeHash casted to Cid = QmaozNR7DZHQK1ZcU9p7QdrshMvXqWK6gpu5rmrkPdT3L4
'hello world' MultiHash.ComputeHash = QmaozNR7DZHQK1ZcU9p7QdrshMvXqWK6gpu5rmrkPdT3L4

the ipfs.FileSystem.AddTextAsync output is as expected, it matches what the desktop IPFS client shows for a textfile with the contents "hello world"

[Arlodotexe]

I have tried casting the output of MultiHash.ComputeHash(bytes) to Cid, I get the same result as if I didnt cast it.

That's because Console.WriteLine is implicitly converting it to a string. Calling .ToString(), casting to a string, or concatenating with a string are expected to have the same output.

A Cid and a MultiHash will output the same value when converted to a string.

---

the ipfs.FileSystem.AddTextAsync output is as expected, it matches what the desktop IPFS client shows for a textfile with the contents "hello world"

That's odd 🤔. Maybe one of the default settings in Kubo has changed since this was last updated by @richardschneider. I'll get back to you on this one.

[Arlodotexe]

I imagine this is because extremely small content can be inlined into a single UnixFS block, where larger content requires splitting the content into multiple blocks.

The way each content is split into blocks, and in turn the way that the CID is generated, is dictated entirely by the IPFS implementation being used. This is why I suggested to borrow code from here to build what you need without running an IPFS daemon.

There's no guarantee the CID will match what you get with Kubo/Iroh/Helia, unless the UnixFS implementation used to chunk content also matches.

That in mind, I would recommend matching (or just using) the implementation you've chosen to create the CID you're after.
If you're using Kubo, a good option is to use the bootstrapper in the OwlCore.Kubo library until a WASI implementation of IPFS is available to us.