Skip to content

chore: speedup builds for binary and prepackage #6022

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 23 commits into from
Jun 27, 2025

Conversation

uinstinct
Copy link
Contributor

@uinstinct uinstinct commented Jun 6, 2025

Description

Using child processes and parallel executions, this PR accelerates installations, copying and downloads. In most cases, the performance improvement is more than 2x.

  • put installNodeModuleInTempDirAndCopyToCurrent into install-copy-nodemodule.js for copying lancedb targets on parallel in child process
  • install node modules for gui and vscode on parallel using child process npm-install.js
  • install and copy sqlite and esbuild on parallel using child process download-copy-sqlite-esbuild.js
  • bundle binaries and download sqlite and ripgrep in parallel in child processes bundle-binary.js
  • used nodejs http dowload over curl (http is 2-3x faster than curl)
  • generate and copy config yaml schema in a child process so that npm installations can be done on parallel generate-copy-config.js
  • refactoring of few functions to avoid duplication
  • when running "launch extension" using the vscode debug panel, prepackage will skip installing and downloading dependencies

performance improvements from my machine:

previous now
npm run build (binary) 65 seconds 15 seconds
npm run prepackage (extension/vscode) 22 seconds 10 seconds
install dependencies script 139 seconds 88 seconds

CON-2267

Checklist

  • [] I've read the contributing guide
  • [] The relevant docs, if any, have been updated or created
  • [] The relevant tests, if any, have been updated or created

Screenshots

[ For visual changes, include screenshots. Screen recordings are particularly helpful, and appreciated! ]

Tests

[ What tests were added or updated to ensure the changes work as expected? ]

Copy link

cubic-dev-ai bot commented Jun 6, 2025

Your cubic subscription is currently inactive. Please reactivate your subscription to receive AI reviews and use cubic.

Copy link

netlify bot commented Jun 6, 2025

Deploy Preview for continuedev ready!

Name Link
🔨 Latest commit a6dbe94
🔍 Latest deploy log https://app.netlify.com/projects/continuedev/deploys/685e15ee3df18a00082c4fb3
😎 Deploy Preview https://deploy-preview-6022--continuedev.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@uinstinct uinstinct changed the title speedup builds for binary and prepackage chore: speedup builds for binary and prepackage Jun 6, 2025
@uinstinct uinstinct marked this pull request as ready for review June 6, 2025 17:51
@uinstinct uinstinct requested a review from a team as a code owner June 6, 2025 17:51
@uinstinct uinstinct requested review from tomasz-stefaniak and removed request for a team June 6, 2025 17:51
@dosubot dosubot bot added the size:XXL This PR changes 1000+ lines, ignoring generated files. label Jun 6, 2025
Copy link

recurseml bot commented Jun 13, 2025

⚠️ Only 5 files will be analyzed due to processing limits.

if (msg.error) {
reject();
}
resolve();
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the copySqlite function, the promise is resolved unconditionally after the error check. If msg.error is true, the code will still call resolve() right after calling reject(), leading to an inconsistent promise state. The resolve() call should be in an else block to ensure mutual exclusion with the error case.


React with 👍 to tell me that this comment was useful, or 👎 if not (and I'll stop posting more comments like this in the future)

if (msg.error) {
reject();
}
resolve();
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to the copySqlite function, the copyEsbuild function has the same promise resolution issue where resolve() is called unconditionally after the error check. The resolve() call should be in an else block to prevent resolving after an error has occurred.


React with 👍 to tell me that this comment was useful, or 👎 if not (and I'll stop posting more comments like this in the future)

Copy link

recurseml bot commented Jun 13, 2025

😱 Found 2 issues. Time to roll up your sleeves! 😱

@uinstinct
Copy link
Contributor Author

normal.extension.launch.mp4
faster.extension.launch.mp4

Copy link
Contributor

@sestinj sestinj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested this on my machine, inspected most of the code to be sure it is really just a refactor + parallelization, had multiple LLM review sessions, and tested building a version of the extension. All worked. This is great. We should keep an eye on it though, because anything in the build process has potential to cause very serious problems

@github-project-automation github-project-automation bot moved this from Todo to In Progress in Issues and PRs Jun 27, 2025
@sestinj sestinj merged commit 78426cc into continuedev:main Jun 27, 2025
36 checks passed
@github-project-automation github-project-automation bot moved this from In Progress to Done in Issues and PRs Jun 27, 2025
@github-actions github-actions bot locked and limited conversation to collaborators Jun 27, 2025
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Jun 27, 2025
@uinstinct uinstinct deleted the speedup-builds branch June 27, 2025 04:22
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
lgtm This PR has been approved by a maintainer size:XXL This PR changes 1000+ lines, ignoring generated files.
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

2 participants