Skip to content

Add global state storage size logging with threshold filter #3810

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

KJ7LNW
Copy link
Collaborator

@KJ7LNW KJ7LNW commented May 22, 2025

Context

This PR adds a utility function to log global state storage sizes with a configurable threshold filter (default 10KB) to help users identify large items.

Implementation

  • Created a new utility file src/utils/storageUtils.ts with:
    • formatBytes function to convert byte sizes to human-readable format
    • logGlobalStorageSize function with threshold parameter (default 10KB)
  • Updated extension.ts to use the new utility functions
  • Added comprehensive tests for the formatBytes function
  • Changed logging to use console.log instead of outputChannel

How to Test

  • Run the extension and check the console output
  • Verify that only storage items larger than 10KB are shown
  • Verify that total size and number of skipped items are displayed

Fixes #3809


Important

Adds utility to log global state storage sizes with a threshold filter and updates logging behavior in the extension.

  • Behavior:
    • Adds logGlobalStorageSize in storageUtils.ts to log global state items exceeding a size threshold (default 10KB).
    • Updates extension.ts to use logGlobalStorageSize during activation.
    • Changes logging from outputChannel to console.log in extension.ts.
  • Utilities:
    • Adds formatBytes in storageUtils.ts to convert byte sizes to human-readable format.
  • Testing:
    • Adds tests for formatBytes in storageUtils.test.ts.

This description was created by Ellipsis for 2f7ab2d. You can customize this summary. It will automatically update as commits are pushed.

Adds a utility function to log global state storage sizes with a configurable
threshold filter (default 10KB) to help users identify large items.

This provides visibility into which global state items are consuming significant
storage space, making it easier to diagnose performance issues.

Fixes: #3809
Signed-off-by: Eric Wheeler <[email protected]>
@KJ7LNW KJ7LNW force-pushed the storage-size-threshold branch from 2f7ab2d to bcb4268 Compare May 22, 2025 02:05
@KJ7LNW
Copy link
Collaborator Author

KJ7LNW commented May 22, 2025

@cte @mrubens @hannesrudolph what does knip test?

It looks like the intention is to prevent files that are not used, but this new file is definitely used (imported by extension.ts) and I cannot figure out how to fix this error:

Run npm run knip
  npm run knip
  shell: /usr/bin/bash -e {0}
  env:
    NODE_VERSION: 20.18.1

> [email protected] knip
> knip --include files

src/utils/storageUtils.ts  
Error: Process completed with exit code 1.

@webpro
Copy link

webpro commented May 22, 2025

First of all, apologies for my creepy unsolicited entrance. Creator of Knip here. Sometimes I see projects using Knip and this caught my eye. Feel free to ignore me! But I noticed that this is a monorepo and the root "." workspace could be better configured like so:

{
	"$schema": "https://unpkg.com/knip@latest/schema.json",
	"workspaces": {
		".": {
			"entry": ["src/extension.ts", "src/workers/countTokens.ts", "src/**/__tests__/**/*{b,B}enchmark.ts"],
			"project": ["src/**/*.ts"]
		},
		"webview-ui": {
			"entry": ["src/index.tsx"],
			"project": ["src/**/*.{ts,tsx}"]
		}
	},
	"ignore": ["**/__mocks__/**"]
}

This way, there's no need to ignore as much! In the next release of Knip you'll be able to remove that "**/__mocks__/**" from ignore as well (and they'll automatically be added as entry files by the Jest and Vitest plugins).

Have a great day ☀️

@KJ7LNW
Copy link
Collaborator Author

KJ7LNW commented May 22, 2025

First of all, apologies for my creepy unsolicited entrance. Creator of Knip here. Sometimes I see projects using Knip and this caught my eye. Feel free to ignore me! But I noticed that this is a monorepo and the root "." workspace could be better configured like so:

{
	"$schema": "https://unpkg.com/knip@latest/schema.json",
	"workspaces": {
		".": {
			"entry": ["src/extension.ts", "src/workers/countTokens.ts", "src/**/__tests__/**/*{b,B}enchmark.ts"],
			"project": ["src/**/*.ts"]
		},
		"webview-ui": {
			"entry": ["src/index.tsx"],
			"project": ["src/**/*.{ts,tsx}"]
		}
	},
	"ignore": ["**/__mocks__/**"]
}

This way, there's no need to ignore as much! In the next release of Knip you'll be able to remove that "**/__mocks__/**" from ignore as well (and they'll automatically be added as entry files by the Jest and Vitest plugins).

Have a great day ☀️

@webpro, Thank you for dropping in!

Do you understand what causes the following? As far as I can tell, src/utils/storageUtils.ts is being used by this PR but for some reason is it is getting triggered:

Run npm run knip
  npm run knip
  shell: /usr/bin/bash -e {0}
  env:
    NODE_VERSION: 20.18.1

> [email protected] knip
> knip --include files

src/utils/storageUtils.ts  
Error: Process completed with exit code 1.

@hannesrudolph hannesrudolph moved this from New to PR [Pre Approval Review] in Roo Code Roadmap May 22, 2025
@webpro
Copy link

webpro commented May 23, 2025

In a monorepo, the top-level entry and project options are not effective, because these should be in the root workspace config at workspaces["."]. Use --debug and look for "Included workspace configs" to see the config Knip uses.

Knip could def need some more config validation, etc. but we're not there yet.

Btw the ignore option in the config I posted can be removed entirely after installing [email protected] (or later).

@KJ7LNW
Copy link
Collaborator Author

KJ7LNW commented May 23, 2025

In a monorepo, the top-level entry and project options are not effective, because these should be in the root workspace config at workspaces["."]. Use --debug and look for "Included workspace configs" to see the config Knip uses.

Knip could def need some more config validation, etc. but we're not there yet.

Btw the ignore option in the config I posted can be removed entirely after installing [email protected] (or later).

thank you I am going to reference this in a new issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: PR [Pre Approval Review]
Development

Successfully merging this pull request may close these issues.

Global state storage size information not available to users
2 participants