why are all platform binaries installed within node_modules?
#59821
Replies: 3 comments 3 replies
-
|
So (and in reference to my edit above), considering next is doing this on purpose to put binaries in place for all possible archictures, we are left with:
|
Beta Was this translation helpful? Give feedback.
-
|
I took a crack at the
const os = require("node:os")
module.exports = {
hooks: {
readPackage: (pkg, _context) => {
if (pkg.name === "next") {
Object.keys(pkg.optionalDependencies)
.filter(
(optionalDependency) =>
optionalDependency.startsWith("@next/swc-") &&
!optionalDependency.startsWith(`@next/swc-${os.platform()}-${os.arch()}`),
)
.forEach((optionalDependency) => {
console.log(`Omitting optional dependency "${optionalDependency}"`)
delete pkg.optionalDependencies[optionalDependency]
delete pkg.dependencies[optionalDependency]
})
}
return pkg
},
},
}Validation# generate a next app
pnpm create next-app test-reduced-platform-binaries \
--typescript \
--tailwind false \
--app \
--src-dir \
--eslint \
--import-alias "@/*"
cd test-reduced-platform-binaries
# get disk usage of node_modules
du -sh node_modules # 1.3G
# write a `.pnpmfile.cjs` to remove downstream `@next/swc-{platform}-{arch}*`
# `dependencies` + `optionalDependencies`
echo '
const os = require("node:os")
module.exports = {
hooks: {
readPackage: (pkg, _context) => {
if (pkg.name === "next") {
Object.keys(pkg.optionalDependencies)
.filter(
(optionalDependency) =>
optionalDependency.startsWith("@next/swc-") &&
!optionalDependency.startsWith(`@next/swc-${os.platform()}-${os.arch()}`),
)
.forEach((optionalDependency) => {
console.log(`Omitting optional dependency "${optionalDependency}"`)
delete pkg.optionalDependencies[optionalDependency]
delete pkg.dependencies[optionalDependency]
})
}
return pkg
},
},
}
' > .pnpmfile.cjs
# regenerate the dependency tree
rm -rf node_modules pnpm-lock.yaml
pnpm install
# get disk usage of node_modules again
du -sh node_modules # 302M
# still works
pnpm lint
pnpm dev
pnpm build && pnpm start |
Beta Was this translation helpful? Give feedback.
-
|
I tried this out and did get into issues with it - the dependencies for the other platforms won't be stored into the lockfile at all, so once you push into a CI provider running a linux container rather than a local darwin (macos) platform, you'll probably fail to run the build |
Beta Was this translation helpful? Give feedback.
-
|
Yeah this is a huge pain, if anyone finds a solve that would be really nice |
Beta Was this translation helpful? Give feedback.
-
|
Create a .pnpmfile.cjs in your project root: // .pnpmfile.cjs module.exports = { regenerate installrm -rf node_modules pnpm-lock.yaml This approach reduced node_modules from ~1.3G → ~300M in practice. |
Beta Was this translation helpful? Give feedback.
-
|
This is problem still exists and the above solution didn't fix the problem. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Summary
I've created a new next app, and was surprised to observe the local environment's
node_modulesto be over 1 GB. I am glad that optimized prod output with standalone mode is quite trim, but CI and developer machines suffer here. \My team creates corporate guidance around Next.js use, and this a heavy installation cost for users on a capped internet connection, remote users with low internet speeds, or users with very tight disk space. Does anyone know if this is intentional behavior, something that has changed, something that can be optimized, etc?
Re-creation
Info
Size
Research
At one point, @timneutkens stated only the supported version should be installed
#29653 (comment)
Why are all of these platform-specific dependencies installed?
They seem to be direct dependencies now.
(EDIT!: It was pointed out to me that
npm viewwill display the optional dependencies here too - and you can confirm the lockfile representation correctly specifies optionalDependencies - like in https://github.com/vercel/examples/blob/main/starter/next-news/pnpm-lock.yaml#L2192-L2201)(END EDIT)
@felixmosh looked into this at some point too via #33398 (reply in thread)
Moreover, the contributing docs explicitly mention using
optionalDependenciestoo:Is that up to date? Did this change out from under the team at some point? The
package.jsonseems to be programmatically generated, as I don't see any of these swc binaries in https://github.com/vercel/next.js/blob/canary/packages/next/package.json and their inclusion at the end of the list above hints to be they are added perhaps in memory and pnpm never has a chance to save / reorder them.Additional information
No response
Example
No response
Beta Was this translation helpful? Give feedback.
All reactions