Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Turbo prune does not respect yarn v1 lock file protocols split entries #8849

Closed
1 task done
Netail opened this issue Jul 26, 2024 · 5 comments · Fixed by #9627
Closed
1 task done

Turbo prune does not respect yarn v1 lock file protocols split entries #8849

Netail opened this issue Jul 26, 2024 · 5 comments · Fixed by #9627
Labels
area: prune turbo prune kind: bug Something isn't working

Comments

@Netail
Copy link

Netail commented Jul 26, 2024 •
edited
Loading

Verify canary release

  • I verified that the issue exists in the latest Turborepo canary release.

Link to code that reproduces this issue

https://github.com/Netail/repro-yarn-lock

What package manager are you using / does the bug impact?

Yarn v1

What operating system are you using?

Mac

Which canary version will you have in your reproduction?

2.0.11

Describe the Bug

Pruned yarn.lock results in merging npm: prefixed versions & non-prefixed versions

"string-width-cjs@npm:string-width@^4.2.0", [email protected], string-width@^4.1.0, string-width@^4.2.0, string-width@^4.2.3:
  version "4.2.3"
  resolved "https://registry.yarnpkg.com/string-width/-/string-width-4.2.3.tgz#269c7117d27b05ad2e536830a8ec895ef9c6d010"
  integrity sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==
  dependencies:
    emoji-regex "^8.0.0"
    is-fullwidth-code-point "^3.0.0"
    strip-ansi "^6.0.1"

Expected Behavior

Split yarn.lock entries starting with the npm: prefix

"string-width-cjs@npm:string-width@^4.2.0":
  version "4.2.3"
  resolved "https://registry.yarnpkg.com/string-width/-/string-width-4.2.3.tgz#269c7117d27b05ad2e536830a8ec895ef9c6d010"
  integrity sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==
  dependencies:
    emoji-regex "^8.0.0"
    is-fullwidth-code-point "^3.0.0"
    strip-ansi "^6.0.1"

string-width@^4.1.0, string-width@^4.2.0, string-width@^4.2.3:
  version "4.2.3"
  resolved "https://registry.yarnpkg.com/string-width/-/string-width-4.2.3.tgz#269c7117d27b05ad2e536830a8ec895ef9c6d010"
  integrity sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==
  dependencies:
    emoji-regex "^8.0.0"
    is-fullwidth-code-point "^3.0.0"
    strip-ansi "^6.0.1"

To Reproduce

  1. Have the split yarn.lock entries in your original yarn.lock
  2. Prune repo
  3. Check pruned yarn.lock
  4. See that they are merged again

Additional context

This issue got fixed in Yarn 1.22.22 with yarnpkg/yarn#9023.

A full thread on the issue; yargs/cliui#138
@Netail Netail added kind: bug Something isn't working needs: triage New issues get this label. Remove it after triage owned-by: turborepo labels Jul 26, 2024
@Netail Netail changed the title Turbo prune does not respect yarn v1 lock file Turbo prune does not respect yarn v1 lock file npm: protocol split Jul 26, 2024
@Netail Netail changed the title Turbo prune does not respect yarn v1 lock file npm: protocol split Turbo prune does not respect yarn v1 lock file npm: protocol split entries Jul 26, 2024
@anthonyshew anthonyshew added area: prune turbo prune and removed needs: triage New issues get this label. Remove it after triage labels Jul 31, 2024
@Netail
Copy link
Author

Netail commented Aug 7, 2024 •
edited
Loading

Important parts in the repro;

package.json (versions defined)
https://github.com/Netail/repro-yarn-lock/blob/main/apps/web/package.json#L5-L8

yarn.lock (how the yarn.lock entries also should look like after being pruned)
https://github.com/Netail/repro-yarn-lock/blob/main/yarn.lock#L25-L41

Currently blocking us from optimally using prune in our CI as we have to delete the pruned yarn.lock and generate a new yarn.lock based on the pruned repo. Slowing the pipeline a bit :(

@Netail
Copy link
Author

Netail commented Sep 20, 2024 •
edited
Loading

The same for theyarn:, github:, git protocols. (And file:, portal: & link:, but those are not even copied over, related to: #4105).

Basically all version fields starting with a protocol prefix should end up as separate entries in the yarn.lock

@Netail Netail changed the title Turbo prune does not respect yarn v1 lock file npm: protocol split entries Turbo prune does not respect yarn v1 lock file protocols split entries Sep 20, 2024
@Netail
Copy link
Author

Netail commented Sep 25, 2024 •
edited
Loading

@chris-olszewski are you able to help me with this issue? yarn.lock entries with a protocol shouldn't be merged together (neither with one without a protocol) Probably something with;

turborepo/crates/turborepo-lockfiles/src/yarn1/mod.rs

Lines 148 to 155 in 6d4e655

const PROTOCOLS: &[&str] = ["", "npm:", "file:", "workspace:", "yarn:"].as_slice();
fn possible_keys<'a>(name: &'a str, version: &'a str) -> impl Iterator<Item = String> + 'a {
PROTOCOLS
.iter()
.copied()
.map(move |protocol| format!("{name}@{protocol}{version}"))
}

Where it tries to check if anything matches with a protocol

@Netail Netail mentioned this issue Oct 29, 2024
Closed
@chris-olszewski chris-olszewski mentioned this issue Dec 17, 2024
Merged
@chris-olszewski
Copy link
Member

@Netail Sorry for the delay on fixing this. This ended up being an issue just with our implementation of Yarn's serialization logic not matching a post yarnpkg/yarn#9023 yarn version. #9627 should provide the fix for this by no longer collapsing entries for descriptors that have different package names.

@Netail
Copy link
Author

Netail commented Dec 17, 2024 •
edited
Loading

Awesome, thanks! No worries on the delay. Tried to take a stab at it too with #9349, but apparently that was not the issue xd

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area: prune turbo prune kind: bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(prune): fix Yarn1 entries getting merged erroneously vercel/turborepo
3 participants
@chris-olszewski @Netail @anthonyshew