Option to share parent directories with the container?

[oconnor663]

I often find myself running into the limitation described in the README:

path dependencies (in Cargo.toml) that point outside the Cargo project won't work because cross use docker containers only mounts the Cargo project so the container doesn't have access to the rest of the filesystem.

I have a project that looks like this:

.
├── b3sum
│   ├── Cargo.toml
│   └── ...
├── c
│   ├── [a bunch of C source files...]
│   ├── blake3_c_rust_bindings
│   │   ├── Cargo.toml
│   │   └── ...
│   └── test.py
├── Cargo.toml
├── reference_impl
│   ├── Cargo.toml
│   └── ...
├── src
│   ├── ...
├── tests
│   └── ...
└── test_vectors
    ├── Cargo.toml
    └── ...

The details don't super duper matter, but for example, the b3sum crate depends on the root crate, and everything wants to depend on reference_impl for testing. And maybe this is a little weird :) but the blake3_c_rust_bindings builds and links the C source files that are in its parent directory, so that cargo test can exercise them.

Currently I only cross test the root crate, because of the path limitation. Every so often I think about reworking this entire thing so that I could cross test all the different components. But I haven't thought of a natural way to do it. Today it occurred to me that all these problems could go away if there was a flag that let me do something like "share the entire project directory with the container, and then run Cargo in a subdirectory." In fact, literally running Cargo in the root directory and adding in e.g. --manifest-path=./b3sum/Cargo.toml seems to almost work, but I run into this error:

    Updating crates.io index
error: failed to write /project/b3sum/Cargo.lock

Caused by:
  failed to open: /project/b3sum/Cargo.lock

Caused by:
  Read-only file system (os error 30)

I'm going to keep playing with this and see if I can find a workaround. In the meantime, I wanted to ask whether something like this had been considered before. Is there a known workaround? Is there a particular reason cross can't support something like this? Is my project layout an abomination? :)

[oconnor663]

Ok, it seems like running cargo generate-lockfile (or anything else with the same side effect) is enough to get past that error. After that I run into some strange errors that I'm trying to debug now. (I see a lot of ELF@0@8: not found printed out by cargo test, and I don't currently know what could be printing that. I'll try to get a smaller repro before I dump a bunch of project-specific details in here.)

A possible work-around is to create a Cargo.toml workspace in the top-most directory:

[workspace]
members = [
  "b3sum",
  "c/blake3_c_rust_bindings",
  "reference_impl",
  "test_vectors"
]

Than you can run cross from that directory:

cross test --package blake3_c_rust_bindings --target armv7-unknown-linux-gnueabihf
cross build --release --package blake3_c_rust_bindings --target armv7-unknown-linux-gnueabihf

I'm not sure this will work for everyone, but it did for me, so I thought I might share :)

[oconnor663]

Coming back to this, the --manifest-path approach seems to work better than it did before. The generate-lockfile workaround is no longer needed. I still see ELF...not found errors specifically on my tests that try to shell out to a built binary, but it's not surprising to me that those are problematic. (In that project, cross test --manifest-path=b3sum/Cargo.toml --target=aarch64-unknown-linux-gnu. Doesn't seem to be specific to the target, as long as it's different from the host.)

I haven't tried the workspace approach yet, in part because of some bad experiences I had in the past with workspaces. (Nothing to do with Cross. But I found that it was hard to test no_std within a workspace, because intra-workspace dependencies meant the std feature was always on...I think.) But that looks like a good alternative for a lot of cases too.

[Emilgardis]

#443 should also help, it's not well documented though

[oconnor663]

@Emilgardis could you give an example of using volumes to allow a path like ../Cargo.toml to resolve?

[Alexhuszagh]

@oconnor663 I've created a sample repository showing how to do this:

The README explains how to mount specific volumes based on environment variables for the given project structure, and then how to build the project. Maybe this should be added to cross_toml.md?

[Emilgardis]

do note though, if your project structure is like that, you should look into https://doc.rust-lang.org/book/ch14-03-cargo-workspaces.html, but that doesn't help for cross right now.

We have #684 in the pipeline, which will make workspaces (and path dependencies) work seamlessly, without having to use volumes as shown in the example

[Alexhuszagh]

#684 is merged supporting workspaces, and we also use the same mount location as on the host for the project if any additional volumes are mounted. This means if you mounted the parent directory of the project, you can access all data from the parent directory while running cross. This means that the project will always work, as will manifest paths and workspaces.