Skip to content

layer: clarify attributes for implied directories #970

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

layer: clarify attributes for implied directories #970

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions layer.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,20 @@ Where supported, MUST include file attributes for Additions and Modifications in

[Sparse files](https://en.wikipedia.org/wiki/Sparse_file) SHOULD NOT be used because they lack consistent support across tar implementations.

#### Implied Directories

As the tar format describes directory hierarchies using a flat datastructure, it is possible to have so-called "implied directories" where not all parent directories implied by an entries' path in the archive have their own entry.
Copy link

@cgwalters cgwalters Jun 3, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note it's more subtle than this, as there's actually two different cases:

  • A parent directory is missing from a specific tar layer, but is present in a "parent" (prior) layer in an "image" (layer stack)
  • A parent directory is missing entirely from the final merged layer stack

It happens to be today that popular runtimes like docker/podman unpack layers literally to disk and then just stitch them together with overlayfs at runtime - and because of this implementation they must create such missing parent directories.

But that's just one implementation model, and not the only one. We're debating this in containers/composefs-rs#132 but basically an entirely different way to do this is to have an object store, and instead of creating an overlayfs stack, one uses reflinks (or composefs) to pre-compute the final merged filesystem tree in a single directory. (Or pre-compute multiple squashed ones, it's flexible)

I would actually like the spec to make it possible for a conforming implementation to error on these "implied directories" in the case where they're actually missing completely in the final image.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But for the case where they are missing, for implementations that want to accept such images (for compatibility, which makes sense), it does make sense to me to specify here a recommendation for what implementations should do.


When applying a layer, implementations MUST create any parent directories implied by an entries' path, even if it is otherwise absent from the archive. Attributes of the created parent directories MUST be set as follows:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this first MUST is uncontroversial, but the second MUST is potentially sticky and perhaps needs to be SHOULD?

To state that another way, are we confident that all existing implementations are currently complying with this second MUST? (I'm reasonably confident they are complying with the first one, because it's kind of unavoidable.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Friendly ping @neersighted 🙇

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that the attribute list needs to be a SHOULD, if only because requiring empty xattrs can run into issues (for instance, SELinux labels and NFSv4 xattrs both have weird behaviours in this respect).


* `mtime` is set to the Unix epoch (`0`)
* `uid` is set to the `0`
* `gid` is set to the `0`
* `mode` is set to `0755`
* `xattrs` are empty
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This cannot be MUST because some xattrs are auto-set by the kernel or need to be set in order for the system to work properly (SELinux labels for instance). I agree with @tianon that this should be a SHOULD.


Layer authors SHOULD ensure directory entries are fully present for all directory hierarchies in their layers, as previous versions of this specification did not specify this behavior and results may be implementation defined.

#### Hardlinks

* Hardlinks are a [POSIX concept](https://pubs.opengroup.org/onlinepubs/9699919799/functions/link.html) for having one or more directory entries for the same file on the same device.
Expand Down