Skip to content

Add global state storage size logging with threshold filter #3810

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 18 additions & 3 deletions src/extension.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ import { DIFF_VIEW_URI_SCHEME } from "./integrations/editor/DiffViewProvider"
import { TerminalRegistry } from "./integrations/terminal/TerminalRegistry"
import { McpServerManager } from "./services/mcp/McpServerManager"
import { telemetryService } from "./services/telemetry/TelemetryService"
import { logGlobalStorageSize } from "./utils/storageUtils"
import { API } from "./exports/api"
import { migrateSettings } from "./utils/migrateSettings"
import { formatLanguage } from "./shared/language"
Expand All @@ -43,12 +44,23 @@ import { initializeI18n } from "./i18n"
*/

let outputChannel: vscode.OutputChannel
let extensionContext: vscode.ExtensionContext
let _extensionContext: vscode.ExtensionContext | undefined

/**
* Returns the extension context.
* @throws Error if the extension context is not available.
*/
export function getExtensionContext(): vscode.ExtensionContext {
if (!_extensionContext) {
throw new Error("Extension context is not available.")
}
return _extensionContext
}

// This method is called when your extension is activated.
// Your extension is activated the very first time the command is executed.
export async function activate(context: vscode.ExtensionContext) {
extensionContext = context
_extensionContext = context
outputChannel = vscode.window.createOutputChannel(Package.outputChannel)
context.subscriptions.push(outputChannel)
outputChannel.appendLine(`${Package.name} extension activated`)
Expand Down Expand Up @@ -130,6 +142,9 @@ export async function activate(context: vscode.ExtensionContext) {
const socketPath = process.env.ROO_CODE_IPC_SOCKET_PATH
const enableLogging = typeof socketPath === "string"

// Log global storage items larger than 10KB
logGlobalStorageSize(10 * 1024)

// Watch the core files and automatically reload the extension host
const enableCoreAutoReload = process.env?.NODE_ENV === "development"
if (enableCoreAutoReload) {
Expand All @@ -150,7 +165,7 @@ export async function activate(context: vscode.ExtensionContext) {
// This method is called when your extension is deactivated.
export async function deactivate() {
outputChannel.appendLine(`${Package.name} extension deactivated`)
await McpServerManager.cleanup(extensionContext)
await McpServerManager.cleanup(getExtensionContext())
telemetryService.shutdown()
TerminalRegistry.cleanup()
}
73 changes: 73 additions & 0 deletions src/utils/storageUtils.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
import { getExtensionContext } from "../extension"

/**
* Formats a number of bytes into a human-readable string with units (Bytes, KB, MB, GB, etc.).
*
* @param bytes The number of bytes.
* @param decimals The number of decimal places to include (default is 2).
* @returns A string representing the formatted bytes.
*/
export function formatBytes(bytes: number, decimals = 2): string {
if (bytes === 0) {
return "0 Bytes"
}
const k = 1024
const dm = decimals < 0 ? 0 : decimals
const sizes = ["Bytes", "KB", "MB", "GB", "TB", "PB", "EB", "ZB", "YB"]
const i = Math.floor(Math.log(bytes) / Math.log(k))
return parseFloat((bytes / Math.pow(k, i)).toFixed(dm)) + " " + sizes[i]
}

/**
* Logs the estimated sizes of items in the VS Code global state to the console.
* Only logs items that exceed the specified minimum size threshold.
*
* @param minSizeBytes The minimum size in bytes to log (default: 10KB)
*/
export function logGlobalStorageSize(minSizeBytes: number = 10 * 1024): void {
const context = getExtensionContext()
try {
console.log(`[Roo Code] Global State Storage Estimates (items > ${formatBytes(minSizeBytes)}):`)
const globalStateKeys = context.globalState.keys()
const stateSizes: { key: string; size: number }[] = []
let totalSize = 0
let itemsSkipped = 0

for (const key of globalStateKeys) {
const value = context.globalState.get(key)
try {
const valueString = JSON.stringify(value)
const size = valueString.length
totalSize += size

if (size >= minSizeBytes) {
stateSizes.push({ key, size })
} else {
itemsSkipped++
}
} catch (e) {
// Handle cases where value might not be stringifiable
stateSizes.push({ key, size: -1 }) // Indicate an error or unmeasurable size
console.log(` - ${key}: (Error calculating size)`)
}
}

stateSizes.sort((a, b) => b.size - a.size)

stateSizes.forEach((item) => {
if (item.size === -1) {
// Already logged error
} else if (item.size === undefined) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the loop where items are logged, there's a check for item.size === undefined. If context.globalState.get(key) actually returns undefined, JSON.stringify(undefined) produces the string "undefined", so size would be 9. Could we clarify how explicitly undefined values retrieved from globalState should be handled or reported here? For instance, if a key exists but its value is undefined, should that be logged differently than an error in size calculation?

Copy link
Collaborator Author

@KJ7LNW KJ7LNW May 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting point.

That code is correct, however, as it should measure len(json).toString() because the size-estimate assumption is that the global state backend storage size is stored as JSON text, so that is the size of the element. (Even if vscode's global state does not store on disk as JSON, it still correctly provides the magnitude of key sizes and the sort order would still be correct as well, and that is all we care about here.)

In any event, the result of a size being only 9 is rather moot: the entire point of this is to provide inspection of large global state sizes, because when the global state gets too big (ie, megabytes) it causes problems, hence PR #3785 and possibly related issues about memory crashes.

This pull request is a result of inspecting global state storage for #3785 to generally see where the problems lie: I thought it could be useful in general for logging Roo startup, to keep an eye on things, and also give users a way to report information like this if troubleshooting is necessary (ie, paste your console logs into this ticket)---however I did not think it should be part of #3785 since that is getting big and I want to keep it focused.

console.log(` - ${item.key}: (undefined value)`)
} else {
console.log(` - ${item.key}: ${formatBytes(item.size)}`)
}
})

console.log(` Total size of all items: ${formatBytes(totalSize)}`)
console.log(` Items below threshold (${itemsSkipped}): not shown`)
console.log("---")
} catch (e: any) {
console.log(`Error displaying global state sizes: ${e.message}`)
}
}
Loading