mkfs.btrfs: support fs-verity in `--rootdir`

[allisonkarlitskaya]

When creating a filesystem with mkfs.btrfs --rootdir, it would be nice to check if the incoming files have fs-verity enabled on them, and if so, to enable it in the created filesystem.

It's possible to use FS_IOC_READ_VERITY_METADATA to copy the data out of the running kernel, avoiding the need to redo all the calculations (although it may be useful to have this code anyway, to share with fs-verity verification in btrfs check).

[adam900710]

Again, I do not think we should depend any feature on the host fs of rootdir.
For things like fs block size can easily screw up things.

If you want to implement a per-inode verify feature, please make it into the --inode-flags feature, and generate the verity data by our own.

[allisonkarlitskaya]

I agree that the questions about block size are interesting here. I worry about that too: if things change here, you're in a lot of trouble. Here's why I don't think that this is a practical issue though: the whole point of enabling fs-verity on a file is that you'll measure the file at some point in the future. If you're measuring the file, you're going to expect it to have some given value. If the block size changed, the computed digest would also change, and then the measurement would fail. In the usecase I'm dealing with (and I don't think it's an atypical one) we even name the files by their fs-verity digest in a content-addressed sort of scheme.

So for me, if the required fs-verity block size of the source and target file systems were different, it would actually be more appropriate to simply fail...

There are also some significant barriers to using --inode-flags here... the existing flags "nodatacow", "nodatasum" are simple flags, but fs-verity can be done in a lot of different ways (as noted, different block sizes, but also choice of hash algorithm, salt, etc.).

It also means that we need to know something about the original filesystem layout when we go to create the filesystem tree. My ultimate goal here is to support creating system images for composefs systems, but in order to do that, I need to know that the /sysroot/composefs/objects/ path has fs-verity enabled on all of its files, but no files anywhere else have fs-verity enabled. That happens to be true today, but I can imagine it might change. bootc, for example, also (sometimes) enables fs-verity on a top-level config file which is used as a marker. We might even have a situation where one of the files in the objects/ directory have fs-verity and others don't, which would mean we'd rather need a mechanism sort of like "--inode-flags-from-file" or so where each line describes a single path, but based on my reading of the existing implementation, having a large number of these could lead to problems with algorithmic complexity...

Of course, the --inode-flags approach makes it possible to build images on systems that don't have native fs-verity support, or only have support for a blocksize that's not compatible with the one we want to use in the final image. This is not an insignificant advantage....

[adam900710]

There are also some significant barriers to using --inode-flags here... the existing flags "nodatacow", "nodatasum" are simple flags, but fs-verity can be done in a lot of different ways (as noted, different block sizes, but also choice of hash algorithm, salt, etc.).

That's easy, just extra verity specified flags. It's fine to require extra options for certain features.
You can just add extra options like --verity-options XXX=YYY,ZZZ=WWW for native verity metadata generation.
But I know this is definitely going to make things much more complex, especially if you want to use different verity options for different files.

Just curious, do you have a case where all verity enabled files have different algo/blocksize/salt/sign?
If so, then I guess it means the --verity-options will also need a path.

With the complexity to specify the verity options for each file, it indeed makes sense to use the native verity data from the host fs.

In that case, I think something like --inode-flags verity-passthrough:PATH may be what you want.

This also brings the extra ability to error out if the host fs doesn't have verity enabled.
(But you still need to do all the extra checks, to make sure we can re-use the verity metadata from the host fs).

For the complexity of each --inode-flags option, we're definitely going to enhance it in the future, instead of blindly re-compare the full path, build some hash or pure (dev_t, ino) pair as the index.
But for now, I think we do not need to over-consider this, especially if you're only specifying a dozen of --inode-flags.

Would this be an acceptable solution?

[allisonkarlitskaya]

For me I guess the most obvious thing to implement is --inode-flags verity-passthrough:/ by default...

I've been thinking as well that it might be interesting to be able to create a filesystem from a stream: mke2fs supports tar (although it can't read fs-verity flags from that, obviously)... it would be really cool to have some sort of a file format that mkfs.btrfs could take on stdin with the file contents (plus fs-verity info for each file), producing the image from that...

[adam900710]

For me I guess the most obvious thing to implement is --inode-flags verity-passthrough:/ by default...

Not a generic solution, as fs-verity is a per-inode feature, just like all the btrfs flags, that users can specify for specified file.
And remember verity feature doesn't get inherited for child inodes (that there is no verity for directory right?), thus even with that, it will only get verify for the root dir, which doesn't make any sense.

I've been thinking as well that it might be interesting to be able to create a filesystem from a stream: mke2fs supports tar (although it can't read fs-verity flags from that, obviously)... it would be really cool to have some sort of a file format that mkfs.btrfs could take on stdin with the file contents (plus fs-verity info for each file), producing the image from that...

That sounds like a reasonable feature, I can explore the idea and implement something like --rootdir-config option for either a stream or file.