[BUG] NPM >= 10.8.0 causes NPM installation failures

[kirrg001]

Is there an existing issue for this?

• I have searched the existing issues

This issue exists in the latest npm version

• I am using the latest npm

Current Behavior

Hi!

The problem started with NPM 10.8.0.
We are using latest Node v18 version.

With NPM 10.8.0 our dependencies fail to install. We are seeing this warning before the complete installation fails:

npm warn cleanup Failed to remove some directories [
npm warn cleanup   [
npm warn cleanup     '/artifacts/node_modules/node-rdkafka',
npm warn cleanup     [Error: ENOTEMPTY: directory not empty, rmdir '/artifacts/node_modules/node-rdkafka/deps/librdkafka'] {
npm warn cleanup       errno: -39,
npm warn cleanup       code: 'ENOTEMPTY',
npm warn cleanup       syscall: 'rmdir',
npm warn cleanup       path: '/artifacts/node_modules/node-rdkafka/deps/librdkafka'
npm warn cleanup     }
npm warn cleanup   ],
npm warn cleanup   [
npm warn cleanup     '/artifacts/node_modules/kafka-avro/node_modules/node-rdkafka',
npm warn cleanup     [Error: ENOTEMPTY: directory not empty, rmdir '/artifacts/node_modules/kafka-avro/node_modules/node-rdkafka/deps/librdkafka'] {
npm warn cleanup       errno: -39,
npm warn cleanup       code: 'ENOTEMPTY',
npm warn cleanup       syscall: 'rmdir',
npm warn cleanup       path: '/artifacts/node_modules/kafka-avro/node_modules/node-rdkafka/deps/librdkafka'
npm warn cleanup     }
npm warn cleanup   ],
npm warn cleanup   [
npm warn cleanup     '/artifacts/node_modules',
npm warn cleanup     [Error: ENOTEMPTY: directory not empty, rmdir '/artifacts/node_modules/kafka-avro/node_modules/node-rdkafka/deps/librdkafka'] {
npm warn cleanup       errno: -39,
npm warn cleanup       code: 'ENOTEMPTY',
npm warn cleanup       syscall: 'rmdir',
npm warn cleanup       path: '/artifacts/node_modules/kafka-avro/node_modules/node-rdkafka/deps/librdkafka'
npm warn cleanup     }
npm warn cleanup   ],
npm warn cleanup   [
npm warn cleanup     '/artifacts/node_modules/kafka-avro',
npm warn cleanup     [Error: ENOTEMPTY: directory not empty, rmdir '/artifacts/node_modules/kafka-avro/node_modules/node-rdkafka/deps/librdkafka'] {
npm warn cleanup       errno: -39,
npm warn cleanup       code: 'ENOTEMPTY',
npm warn cleanup       syscall: 'rmdir',
npm warn cleanup       path: '/artifacts/node_modules/kafka-avro/node_modules/node-rdkafka/deps/librdkafka'
npm warn cleanup     }
npm warn cleanup   ]

Follow by an extraction error

npm error gyp verb extracted file from tarball node-v18.20.7/include/node/v8-platform.h
npm error gyp verb extracted file from tarball node-v18.20.7/include/node/v8-local-handle.h
npm error gyp verb extracted file from tarball node-v18.20.7/include/node/uv.h
npm error gyp verb extracted file from tarball node-v18.20.7/include/node/v8-script.h
....

Finally resulting in:

npm error /root/.cache/node-gyp/18.20.7/include/node/v8-object.h:19:7: note: forward declaration of 'class v8::Function'
npm error    19 | class Function;
npm error       |       ^~~~~~~~
npm error ../../nan/nan_maybe_43_inl.h: In function 'Nan::MaybeLocal<v8::Object> Nan::NewInstance(v8::Local<v8::Function>, int, v8::Local<v8::Value>*)':
npm error ../../nan/nan_maybe_43_inl.h:53:24: error: invalid use of incomplete type 'class v8::Function'
npm error    53 |   return scope.Escape(h->NewInstance(isolate->GetCurrentContext(), argc, argv)
npm error gyp ERR! build error 
npm error gyp ERR! stack Error: `make` failed with exit code: 2
npm error gyp ERR! stack at ChildProcess.<anonymous> (/artifacts/node_modules/node-gyp/lib/build.js:216:23)
npm error gyp ERR! System Linux 5.4.0-193-generic
npm error gyp ERR! command "/usr/local/bin/node" "/artifacts/node_modules/.bin/node-gyp" "rebuild"
npm error gyp ERR! cwd /artifacts/node_modules/event-loop-stats
npm error gyp ERR! node -v v18.20.7
npm error gyp ERR! node-gyp -v v11.1.0
npm error gyp ERR! not ok
npm verbose cwd /artifacts
npm verbose os Linux 5.4.0-193-generic
npm verbose node v18.20.7
npm verbose npm  v10.8.0

No issues with NPM 10.7.0.

It seems not caused by node-gyp.

Expected Behavior

No failures.

Steps To Reproduce

Currently I cannot reproduce this error in a Docker container.

Environment

Using node: v18.20.4
Using npm: >= 10.8.0
Architecture: x64
Python3 version: Python 3.11.2
Make version: GNU Make 4.3
GCC version: gcc (Debian 12.2.0-14) 12.2.0

[milaninfy]

@kirrg001 Can you please provide more details like, reproduction steps, package.json, lock file complete logs etc ?

[kirrg001]

Hi!

So far I could not reproduce it in a Docker container.

package.json files:

• https://github.com/instana/nodejs/blob/main/package.json
• https://github.com/instana/nodejs/tree/main/packages (sub packages)

lock file complete logs

How to provide this file? Thanks!

[milaninfy]

You can run your install with log levels --loglevel silly flag to get the logs and at the end of execution you can also get the log file. checkout https://docs.npmjs.com/cli/v11/using-npm/logging

It's very difficult to track the issue without reproduction steps. Try to replicate the same on latest npm for your node version.

[milaninfy]

This is my output of running npm install on the package.json and the repo link you provided with

~/workarea/rep/test $ node -v
v22.14.0
~/workarea/rep/test $ npm -v
11.2.0


~/workarea/rep/test $ git clone https://github.com/instana/nodejs.git
Cloning into 'nodejs'...
remote: Enumerating objects: 60680, done.
remote: Counting objects: 100% (864/864), done.
remote: Compressing objects: 100% (363/363), done.
remote: Total 60680 (delta 635), reused 507 (delta 501), pack-reused 59816 (from 3)
Receiving objects: 100% (60680/60680), 188.44 MiB | 26.56 MiB/s, done.
Resolving deltas: 100% (46172/46172), done.

~/workarea/rep/test $ cd nodejs 

~/workarea/rep/test/nodejs $ npm install
npm warn Unknown project config "clidriver". This will stop working in the next major version of npm.
npm warn deprecated @npmcli/[email protected]: This functionality has been moved to @npmcli/fs
npm warn deprecated [email protected]: Package no longer supported. Contact Support at https://www.npmjs.com/support for more info.
npm warn deprecated [email protected]: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm warn deprecated [email protected]: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.
npm warn deprecated [email protected]: Please upgrade to latest, formidable@v2 or formidable@v3! Check these notes: https://bit.ly/2ZEqIau
npm warn deprecated [email protected]: The `apollo-server-plugin-base` package is part of Apollo Server v2 and v3, which are now deprecated (end-of-life October 22nd 2023 and October 22nd 2024, respectively). This package's functionality is now found in the `@apollo/server` package. See https://www.apollographql.com/docs/apollo-server/previous-versions/ for more details.
npm warn deprecated [email protected]: The `apollo-datasource` package is part of Apollo Server v2 and v3, which are now deprecated (end-of-life October 22nd 2023 and October 22nd 2024, respectively). See https://www.apollographql.com/docs/apollo-server/previous-versions/ for more details.
npm warn deprecated [email protected]: The `apollo-server-types` package is part of Apollo Server v2 and v3, which are now deprecated (end-of-life October 22nd 2023 and October 22nd 2024, respectively). This package's functionality is now found in the `@apollo/server` package. See https://www.apollographql.com/docs/apollo-server/previous-versions/ for more details.
npm warn deprecated [email protected]: The `apollo-server-errors` package is part of Apollo Server v2 and v3, which are now deprecated (end-of-life October 22nd 2023 and October 22nd 2024, respectively). This package's functionality is now found in the `@apollo/server` package. See https://www.apollographql.com/docs/apollo-server/previous-versions/ for more details.
npm warn deprecated [email protected]: The `apollo-server-env` package is part of Apollo Server v2 and v3, which are now deprecated (end-of-life October 22nd 2023 and October 22nd 2024, respectively). This package's functionality is now found in the `@apollo/utils.fetcher` package. See https://www.apollographql.com/docs/apollo-server/previous-versions/ for more details.
npm warn deprecated [email protected]: The `apollo-reporting-protobuf` package is part of Apollo Server v2 and v3, which are now deprecated (end-of-life October 22nd 2023 and October 22nd 2024, respectively). This package's functionality is now found in the `@apollo/usage-reporting-protobuf` package. See https://www.apollographql.com/docs/apollo-server/previous-versions/ for more details.
npm warn deprecated @types/[email protected]: mongodb provides its own types. @types/mongodb is no longer needed.
npm warn deprecated [email protected]: Critical security vulnerability fixed in v0.21.1. For more information, see https://github.com/axios/axios/pull/3410

> postinstall
> chmod +x ./node_modules/husky/lib/bin.js && husky install && lerna run node_modules:exists

husky - Git hooks installed
lerna notice cli v8.1.8

   ✔  @instana/metrics-util:node_modules:exists (400ms)
   ✔  @instana/autoprofile:node_modules:exists (401ms)
   ✔  @instana/serverless:node_modules:exists (402ms)
   ✔  @instana/google-cloud-run:node_modules:exists (512ms)
   ✔  @instana/aws-fargate:node_modules:exists (514ms)
   ✔  @instana/serverless-collector:node_modules:exists (515ms)
   ✔  @instana/aws-lambda:node_modules:exists (516ms)
   ✔  @instana/azure-container-services:node_modules:exists (518ms)
   ✔  instana-aws-lambda-auto-wrap:node_modules:exists (303ms)

————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————

 Lerna (powered by Nx)   Successfully ran target node_modules:exists for 9 projects (1s)


added 4162 packages, and audited 4178 packages in 2m

312 packages are looking for funding
  run `npm fund` for details

55 vulnerabilities (5 low, 25 moderate, 21 high, 4 critical)

To address issues that do not require attention, run:
  npm audit fix

To address all issues possible (including breaking changes), run:
  npm audit fix --force

Some issues need review, and may require choosing
a different dependency.

Run `npm audit` for details.

[milaninfy]

Closing due to age. If this is still a problem please feel free to reopen this issue, or create a new issue w/ steps to reproduce.