Load storages in the storage Worker in the standalone app - #1323
Merged
Conversation
A caller has to tell the storage Worker where sqlite3.wasm is, and knowing that means resolving a file inside node_modules. Wrap it in a `?url` import the bundler rewrites to whatever it emitted, so an app can hand over the URL without reaching into the dependency layout itself. The suffix is understood by Vite, hence the module declaration; the VS Code build resolves the same asset its own way.
Selecting a storage file blocked the page: sqlite-wasm initialization, the queries, and for a Journal file the whole decode, split, parse and replay all ran on the caller's thread, which is the UI thread. The app stopped responding until it finished. The file is now read with File.arrayBuffer() and its buffer transferred to the storage Worker. The provider owns the session rather than holding whatever storage it was handed: it keeps at most one at a time, refuses a second file while one is open, closes on unmount, and tracks loading and error state so the UI can show both. A load whose session was replaced while it ran closes the storage it just opened rather than publishing it. Which Worker to start is injected, because a Vite build and a VS Code Webview start one differently. The Webview still parses on the UI thread and hands the result to setStorage(), which is why that entry point survives. Its format detection moved into vscode_entry.tsx, where it belongs until that side is moved over as well: left in the provider, it would keep pulling the SQLite backend into the standalone bundle, which is exactly what this change gets rid of.
A storage request that failed used to leave the page as it was, since nothing caught it and nothing rendered an error. The study list and the study page now survive a rejected request, ignore a response that arrives after they stopped caring about it, and report the failure to a snackbar, which is the first time any of these errors are visible. The loader is hidden once a storage is open, so a second file cannot quietly replace the first, and the study list gains a button to close the current one.
Dropping an unrelated file opened it as a storage with nothing in it: the loader disappeared, no study appeared, and nothing said why. Three files did that. A JSON or text file is not SQLite, so it was read as a Journal, where a line that cannot be read is collected as a warning rather than raised. That is what keeps a partially written Journal usable, but it also makes a file of the wrong format read as one with no records. Count the lines that were understood as records and reject a file that yielded none. A Journal storage whose studies were all deleted still has records, so it stays openable. An empty file is neither format, and creating a storage does leave one behind, so name that case instead of opening it. A SQLite database that Optuna never wrote deserializes like any other and only failed on the first query, from where the study list could not tell it apart from a storage with no study. Check for the tables the queries need while opening. All three checks are in the Worker, next to the format detection they belong to.
The button that closed the current storage was an unlabelled icon: it said neither what was loaded nor what pressing it would do. Show the file name in the app bar instead, as a chip whose delete icon closes it and brings the loader back. The name comes from the File the loader read, so it is only there for a storage the standalone app opened itself.
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Contributor License Agreement
Reference Issues/PRs
Follow-up of #1322, which added the storage Worker and its client without any caller. This is the first caller.
What does this implement/fix? Explain your changes.
Selecting a storage file in the standalone app blocked the page. sqlite-wasm initialization,
sqlite3_deserialize()and every query ran on the UI thread, and a Journal file was decoded, split, JSON-parsed and replayed there as well, so nothing responded until parsing finished.The app now reads the file with
File.arrayBuffer()and transfers the buffer to the storage Worker, which detects the format, opens the backend and answers queries. The UI thread only sends requests and renders the results.Session handling
StorageProviderowns the session instead of holding whatever storage it was handed:Failures are visible
A rejected storage request used to leave the page as it was, since nothing caught it and nothing rendered it. The study list and the study page now survive one, ignore a response that arrives after they stopped caring about it, and report it to a snackbar. This is the first time any of these errors are visible.
Bundle
The UI bundle no longer contains the SQLite backend: it is only reachable from the Worker entry.
vite buildbefore and after:The VS Code Webview is unchanged
Starting the Worker in a Webview needs assets that the Webview build does not emit yet, so
vscode_entry.tsxkeeps parsing on the UI thread and hands the result to the provider throughsetStorage().The format detection moved from the provider into that entry point: left where it was, it would keep pulling the SQLite backend into the standalone bundle. Both go away in the follow-up that moves the Webview over.