Tracking Issue for Packages as (optional) namespaces #122349

oli-obk · 2024-03-11T21:57:24Z

This is a tracking issue for the RFC "3243" (rust-lang/rfcs#3243).

crates.io tracking issue: Tracking Issue for Packages as (optional) namespaces crates.io#8292
cargo tracking issue: Tracking Issue for Packages as (optional) namespaces cargo#13576
docs.rs tracking issue: Tracking Issue for Packages as (optional) namespaces docs.rs#2457

About tracking issues

Tracking issues are used to record the overall progress of implementation.
They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions.
A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature.
Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.

Steps

Implement the RFC (can anyone write up mentoring
instructions?)
Adjust documentation (see instructions on rustc-dev-guide)
Formatting for new syntax has been added to the Style Guide (nightly-style-procedure)
Stabilization PR (see instructions on rustc-dev-guide)

Unresolved Questions

How exactly should the Cargo.toml lib.name key work in this world, and how does that integrate with --extern and -L and sysroots?
Decide on and document precise name conflict rules for the compiler (RFC: Packages as (optional) namespaces rfcs#3243 (comment))

Third-parties, like Linux distributions, will need to decide how to encode
cargo package names in their distribution package names according to their
individual rules.
Compared to existing ecosystems with namespaces that they package, the only new
wrinkle is that there can be 0-1 namespace levels.

Implementation history

The text was updated successfully, but these errors were encountered:

epage · 2024-03-12T15:41:37Z

Started a zulip thread exploring if there is any UX work we can do for rustdoc / docs.rs to help users.

epage · 2024-03-17T00:39:38Z

Basic cargo support is in as of rust-lang/cargo#13591 (submodule update: #122575)

I created a cargo::issue_13576 lib and then next part to resolve in the implementation appears to be in rustc:

cargo-features = ["open-namespaces"]

[package]
name = "cargo::issue_13576"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]

    Finished dev [unoptimized + debuginfo] target(s) in 0.07s
     Running `/home/epage/src/personal/cargo/target/debug/cargo -Zscript check -vvv`
    Checking cargo::issue_13576 v0.1.0 (/home/epage/src/personal/dump/cargo-13576)
     Running `CARGO=/home/epage/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/cargo CARGO_CRATE_NAME='cargo::issue_13576' CARGO_MANIFEST_DIR=/home/epage/src/personal/dump/cargo-13576 CARGO_PKG_AUTHORS='' CARGO_PKG_DESCRIPTION='' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME='cargo::issue_13576' CARGO_PKG_README='' CARGO_PKG_REPOSITORY='' CARGO_PKG_RUST_VERSION='' CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 CARGO_RUSTC_CURRENT_DIR=/home/epage/src/personal/dump/cargo-13576 LD_LIBRARY_PATH='/home/epage/src/personal/dump/cargo-13576/target/debug/deps:/home/epage/src/personal/cargo/target/debug/build/curl-sys-2b2243103868cb3e/out/build:/home/epage/src/personal/cargo/target/debug/build/libgit2-sys-71ef3ff041cafba4/out/build:/home/epage/src/personal/cargo/target/debug/build/libnghttp2-sys-5d0ed877a94b784a/out/i/lib:/home/epage/src/personal/cargo/target/debug/build/libsqlite3-sys-5134432fb6771f3a/out:/home/epage/src/personal/cargo/target/debug/build/libssh2-sys-f4299e26a5b2bcb1/out/build:/home/epage/src/personal/cargo/target/debug/deps:/home/epage/src/personal/cargo/target/debug:/home/epage/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib:/home/epage/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib' /home/epage/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc --crate-name 'cargo::issue_13576' --edition=2021 src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata -C embed-bitcode=no -C debuginfo=2 -C metadata=5d93f16ea53919c5 -C extra-filename=-5d93f16ea53919c5 --out-dir /home/epage/src/personal/dump/cargo-13576/target/debug/deps -C incremental=/home/epage/src/personal/dump/cargo-13576/target/debug/incremental -L dependency=/home/epage/src/personal/dump/cargo-13576/target/debug/deps`
error: invalid character `':'` in crate name: `cargo::issue_13576`
  |
  = help: you can either pass `--crate-name` on the command line or add `#![crate_name="…"]` to set the crate name

error: could not compile `cargo::issue_13576` (lib) due to 1 previous error

Caused by:
  process didn't exit successfully: `CARGO=/home/epage/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/cargo CARGO_CRATE_NAME='cargo::issue_13576' CARGO_MANIFEST_DIR=/home/epage/src/personal/dump/cargo-13576 CARGO_PKG_AUTHORS='' CARGO_PKG_DESCRIPTION='' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME='cargo::issue_13576' CARGO_PKG_README='' CARGO_PKG_REPOSITORY='' CARGO_PKG_RUST_VERSION='' CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 CARGO_RUSTC_CURRENT_DIR=/home/epage/src/personal/dump/cargo-13576 LD_LIBRARY_PATH='/home/epage/src/personal/dump/cargo-13576/target/debug/deps:/home/epage/src/personal/cargo/target/debug/build/curl-sys-2b2243103868cb3e/out/build:/home/epage/src/personal/cargo/target/debug/build/libgit2-sys-71ef3ff041cafba4/out/build:/home/epage/src/personal/cargo/target/debug/build/libnghttp2-sys-5d0ed877a94b784a/out/i/lib:/home/epage/src/personal/cargo/target/debug/build/libsqlite3-sys-5134432fb6771f3a/out:/home/epage/src/personal/cargo/target/debug/build/libssh2-sys-f4299e26a5b2bcb1/out/build:/home/epage/src/personal/cargo/target/debug/deps:/home/epage/src/personal/cargo/target/debug:/home/epage/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/lib:/home/epage/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/lib' /home/epage/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc --crate-name 'cargo::issue_13576' --edition=2021 src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata -C embed-bitcode=no -C debuginfo=2 -C metadata=5d93f16ea53919c5 -C extra-filename=-5d93f16ea53919c5 --out-dir /home/epage/src/personal/dump/cargo-13576/target/debug/deps -C incremental=/home/epage/src/personal/dump/cargo-13576/target/debug/incremental -L dependency=/home/epage/src/personal/dump/cargo-13576/target/debug/deps` (exit status: 1)

The implementation will need to watch for any files/directories that are created that include :: in the name as that is unsupported on Windows.

epage · 2024-03-18T18:46:44Z

@Nemo157

My understanding from the RFC was that the library name was meant to just be bar, that would avoid the rustc issue you noted

The autogenerated lib.name key for such a crate will just be bar, the leaf crate name, and the expectation is that to use such crates one must use --extern foo::bar=bar.rlib syntax. There may be some better things possible here, perhaps foo_bar can be used here.
—https://rust-lang.github.io/rfcs/3243-packages-as-optional-namespaces.html#reference-level-explanation

(I'd assume this also applies to the autogenerated bin.name for src/main.rs)

(centralizing the conversation)

epage · 2024-03-18T18:47:07Z

That gets complicated as cargo assumes the crate-name and extern-name are the same (and both pull from <target>.name) and I suspect that runs deeper than that in both programs.

Nemo157 · 2024-03-18T18:53:42Z

Deeper than rustc's assumption that a crate-name is an identifier? (IIRC pre-1.0 - actually made it into the crate name, so there was extern crate "foo-bar" as foo_bar; syntax, but that was removed from rustc and normalization to an identifier was moved to cargo).

epage · 2024-03-18T19:09:29Z

I can't speak to that or most of the rustc side. If someone familiar enough with both wants to put forward a working proposal for how both sides can work, I'd love to look over it and see what the trade offs are.

epage · 2024-03-25T20:57:12Z

Thinking about this some more, some routes we can go

Allow :: in crate names, breaking rustc's assumption that its an identifier
Translate before <target>.name but this breaks cargo's assumption that <target>.name is the name callers use to access the item
Translate from <target>.name (with ::) to an identifier when doing --extern calls

Some things we need to keep in mind

Rustdoc tests should be accessed with the :: name. I've seen rustdoc do enough weird stuff that I hold no assumptions that its behaving like a normal compilation process.
Ideally, user errors would be in terms of the name they set, not a name used internally for bookkeeping. If the crate name doesn't show up much in errors, maybe this isn't a big deal.

hydroperx · 2024-07-21T12:24:27Z

Thinking about this some more, some routes we can go

Allow :: in crate names, breaking rustc's assumption that its an identifier

Translate before <target>.name but this breaks cargo's assumption that <target>.name is the name callers use to access the item

Translate from <target>.name (with ::) to an identifier when doing --extern calls

Some things we need to keep in mind

Rustdoc tests should be accessed with the :: name. I've seen rustdoc do enough weird stuff that I hold no assumptions that its behaving like a normal compilation process.

Ideally, user errors would be in terms of the name they set, not a name used internally for bookkeeping. If the crate name doesn't show up much in errors, maybe this isn't a big deal.

Can you intercept file system calls to a crate's filename (whether for the crate's download or for the crate's target artifacts) on file open? (Where, in Windows, :: could translate to -- as suggested earlier)

Manishearth · 2024-09-11T18:47:48Z

Can you intercept file system calls to a crate's filename (whether for the crate's download or for the crate's target artifacts) on file open?

I'd prefer we don't use hacks like this.

Nemo157 · 2025-02-14T14:07:01Z

Since I don't recall ever seeing it mentioned, another unresolved question seems to be multi-registry handling:

[dependencies]
"foo::bar" = "1"
"foo::baz" = { version = "1", registry = "custom" }

epage · 2025-02-14T15:12:00Z

@Nemo157 I assume that is a question more for rust-lang/cargo#13576 as this tracking issue is just for the rustc side.

This was discussed during the RFC without a definitive answer at rust-lang/rfcs#3243 (comment) but I was leaning towards leaving all restrictions for what can participate in a namespace to crates.io

Issue rust-lang#122349

Also refactors some of the crate name parsing code and adds unit tests Issue rust-lang#122349 Co-authored-by: León Orell Valerian Liehr <[email protected]>

Add unstable parsing of `--extern foo::bar=libbar.rlib` command line options This is a tiny step towards implementing the rustc side of support for implementing packages as optional namespaces (rust-lang#122349). We add support for parsing command line options like `--extern foo::bar=libbar.rlib` when the `-Z namespaced-crates` option is present. We don't do anything further with them. The next step is to plumb this down to the name resolver. This PR also generally refactors the extern argument parsing code and adds some unit tests to make it clear what forms should be accepted with and without the flag. cc `@epage` `@ehuss`

Add unstable parsing of `--extern foo::bar=libbar.rlib` command line options This is a tiny step towards implementing the rustc side of support for implementing packages as optional namespaces (rust-lang#122349). We add support for parsing command line options like `--extern foo::bar=libbar.rlib` when the `-Z namespaced-crates` option is present. We don't do anything further with them. The next step is to plumb this down to the name resolver. This PR also generally refactors the extern argument parsing code and adds some unit tests to make it clear what forms should be accepted with and without the flag. cc ``@epage`` ``@ehuss``

Add unstable parsing of `--extern foo::bar=libbar.rlib` command line options This is a tiny step towards implementing the rustc side of support for implementing packages as optional namespaces (rust-lang#122349). We add support for parsing command line options like `--extern foo::bar=libbar.rlib` when the `-Z namespaced-crates` option is present. We don't do anything further with them. The next step is to plumb this down to the name resolver. This PR also generally refactors the extern argument parsing code and adds some unit tests to make it clear what forms should be accepted with and without the flag. cc ```@epage``` ```@ehuss```

Rollup merge of rust-lang#139647 - eholk:package-namespace, r=fmease Add unstable parsing of `--extern foo::bar=libbar.rlib` command line options This is a tiny step towards implementing the rustc side of support for implementing packages as optional namespaces (rust-lang#122349). We add support for parsing command line options like `--extern foo::bar=libbar.rlib` when the `-Z namespaced-crates` option is present. We don't do anything further with them. The next step is to plumb this down to the name resolver. This PR also generally refactors the extern argument parsing code and adds some unit tests to make it clear what forms should be accepted with and without the flag. cc ```@epage``` ```@ehuss```

Also refactors some of the crate name parsing code and adds unit tests Issue rust-lang#122349 Co-authored-by: León Orell Valerian Liehr <[email protected]>

petrochenkov · 2025-04-26T14:23:20Z

Can I say "No Go" for this proposal from the compiler side?
I have briefly seen it in the RFC stage, but didn't realize it required language changes, but now I have reviewed #140271.

The chosen syntax deliberately introduces a quite nasty ambiguity into name resolution, which is significantly worse than some similar cases that we already have for backward compatibility, and which are considered unfortunate.
In particular, we can no longer resolve names like first in first::second::third in isolation, we need to consider the following segments in their path as well, this is not something we previously had.
#140271 works this around by breaking the parenting relation instead and "planting" a foo from crate my_api::foo as a reexport into the crate root of the different crate my_api if it's present, even if it's not actually there.

The core motivation in the RFC 3243 is social - allowing organizations to reserve some names and users to trust those names, it's not about some missing capabilities in the language.
So I'd prefer for the org::my_lib -> org_my_lib normalization to happen at crates.io or/and cargo level, without ever getting to rustc.

Manishearth · 2025-04-26T15:15:46Z

@petrochenkov I suggest having the compiler team discuss this and weigh tradeoffs then. We did solicit feedback from the compiler team and got some sign off when writing this rfc.

I'm not convinced of the implementation difficulty, the list of known crates is small and the crate filtering can be done early, presumably? I think there are some other potential implementation concerns but this doesn't feel like a major one.

Also note that the RFC deliberately only introduces a single level of nesting. I don't think the problem is too bad for multiple layers either.

This might not have been in the RFC itself but generally the idea was that if you have foo and foo::bar crates, any bar module in foo can be assumed to not exist by the compoler: this is something crate owners should not do (unless it's just a reexport).

workingjubilee · 2025-04-28T04:31:39Z

This might not have been in the RFC itself but generally the idea was that if you have foo and foo::bar crates, any bar module in foo can be assumed to not exist by the compoler: this is something crate owners should not do (unless it's just a reexport).

Perhaps we should specify this in some way and a predetermined resolution should be picked? ...I could see "fail the build catastrophically" being the preferred solution, here. Or is this an ambiguity that continues to be a severe problem even if we make it functionally impossible that the ambiguity ever slices two ways?

epage · 2025-04-28T14:15:10Z

The core motivation in the RFC 3243 is social - allowing organizations to reserve some names and users to trust those names, it's not about some missing capabilities in the language.

This is not accurate. While it originally started as a registry namespacing feature, with the new syntax it shifted towards being an API design feature: "(partially) open (API) namespaces". Because of this, I've shifted the language of the tracking issues to that term even though we didn't officially rename the RFC.

For instance, Python users had hacked-in support for this for years and they officially recognized it and added support for it. This design behaves most like Python in that it only allows you to add extend a namespace with sub-namespaces. Python requires the top-level package to exclusively be a namespace package while we allow it to be a regular package. We also add publish-control to our own registry (other registries and other sources of code can do their own thing). Languages like C++ put no limits on how open namespaces are.

heaths · 2025-04-28T15:32:57Z

.NET (NuGet), JS (NPM), Java (Maven), Go (URIs), et. al. all have scopes built into their packaging systems.

NuGet is an interesting comparison because their registered namespace separator is . - the same character to separate namespace elements like :: that was proposed for Rust. But in their cases, it's nuget.org itself that handles the reservations. For example, my team owns the Azure. root namespace. No one else can publish a package starting with that prefix (after the prefix support was added and our root registered). But it's still part of the package name, which tends to match the namespace.

Alternatively, if resolving :: as part of the path is problematic, why not choose one of the other separators that was suggested? When I read through the history of the RFC a while back, there were some good suggestions and it was a little surprising that the Rust path separator was chosen because it seemed problematic to differentiate with a regular path.

oli-obk · 2025-04-28T16:56:17Z

Tracking issues are used to record the overall progress of implementation.
They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions.
A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature.
Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.

oli-obk added the C-tracking-issue Category: An issue tracking the progress of sth. like the implementation of an RFC label Mar 11, 2024

rust-lang locked as off-topic and limited conversation to collaborators Mar 11, 2024

oli-obk added T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-crates-io Relevant to the crates.io team, which will review and decide on the PR/issue. labels Mar 11, 2024

ehuss added the T-cargo Relevant to the cargo team, which will review and decide on the PR/issue. label Mar 11, 2024

This was referenced Mar 12, 2024

Tracking Issue for Packages as (optional) namespaces rust-lang/cargo#13576

Open

Tracking Issue for Packages as (optional) namespaces rust-lang/crates.io#8292

Open

syphar mentioned this issue Mar 13, 2024

Tracking Issue for Packages as (optional) namespaces rust-lang/docs.rs#2457

Open

epage mentioned this issue Mar 15, 2024

RFC: Packages as (optional) namespaces rust-lang/rfcs#3243

Merged

This comment has been minimized.

Sign in to view

rust-lang unlocked this conversation Apr 15, 2024

kpreid mentioned this issue Oct 4, 2024

RFC: Nested Cargo packages rust-lang/rfcs#3452

Open

tisonkun mentioned this issue Dec 28, 2024

feat: Add 'open-namespaces' feature rust-lang/cargo#13591

Merged

eholk added a commit to eholk/rust that referenced this issue Apr 10, 2025

Add unstable parse for foo::bar extern command line arguments

942ab5d

Issue rust-lang#122349

eholk mentioned this issue Apr 10, 2025

Add unstable parsing of --extern foo::bar=libbar.rlib command line options #139647

Merged

eholk added a commit to eholk/rust that referenced this issue Apr 14, 2025

Add unstable parse for foo::bar extern command line arguments

4fcec5c

Issue rust-lang#122349

eholk added a commit to eholk/rust that referenced this issue Apr 14, 2025

Add unstable parse for foo::bar extern command line arguments

4445857

Issue rust-lang#122349

eholk added a commit to eholk/rust that referenced this issue Apr 15, 2025

Add unstable foo::bar extern command line arguments

f35c85f

Also refactors some of the crate name parsing code and adds unit tests Issue rust-lang#122349 Co-authored-by: León Orell Valerian Liehr <[email protected]>

Emilgardis mentioned this issue Apr 16, 2025

proposal: separate crates for peripherals rust-embedded/svd2rust#902

Open

petrochenkov mentioned this issue Apr 26, 2025

Add compiler support for namespaced crates #140271

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tracking Issue for Packages as (optional) namespaces #122349

Tracking Issue for Packages as (optional) namespaces #122349

oli-obk commented Mar 11, 2024 •

edited by epage

Loading

epage commented Mar 12, 2024

epage commented Mar 17, 2024 •

edited

Loading

epage commented Mar 18, 2024

epage commented Mar 18, 2024

Nemo157 commented Mar 18, 2024

epage commented Mar 18, 2024

epage commented Mar 25, 2024 •

edited

Loading

This comment has been minimized.

This comment has been minimized.

hydroperx commented Jul 21, 2024 •

edited

Loading

Manishearth commented Sep 11, 2024

Nemo157 commented Feb 14, 2025

epage commented Feb 14, 2025

petrochenkov commented Apr 26, 2025

Manishearth commented Apr 26, 2025

workingjubilee commented Apr 28, 2025

epage commented Apr 28, 2025

heaths commented Apr 28, 2025

oli-obk commented Apr 28, 2025

Tracking Issue for Packages as (optional) namespaces #122349

Tracking Issue for Packages as (optional) namespaces #122349

Comments

oli-obk commented Mar 11, 2024 • edited by epage Loading

About tracking issues

Steps

Unresolved Questions

Implementation history

epage commented Mar 12, 2024

epage commented Mar 17, 2024 • edited Loading

epage commented Mar 18, 2024

epage commented Mar 18, 2024

Nemo157 commented Mar 18, 2024

epage commented Mar 18, 2024

epage commented Mar 25, 2024 • edited Loading

This comment has been minimized.

This comment has been minimized.

hydroperx commented Jul 21, 2024 • edited Loading

Manishearth commented Sep 11, 2024

Nemo157 commented Feb 14, 2025

epage commented Feb 14, 2025

petrochenkov commented Apr 26, 2025

Manishearth commented Apr 26, 2025

workingjubilee commented Apr 28, 2025

epage commented Apr 28, 2025

heaths commented Apr 28, 2025

oli-obk commented Apr 28, 2025

oli-obk commented Mar 11, 2024 •

edited by epage

Loading

epage commented Mar 17, 2024 •

edited

Loading

epage commented Mar 25, 2024 •

edited

Loading

hydroperx commented Jul 21, 2024 •

edited

Loading