Data Explorer: Add support for opening .xlsx files #7366
Conversation
DuckDB v1.2 launched native support for reading `.xlsx` files with the `excel` extension. This commit bumps our DuckDB WASM dependency to pick up this change, and wires up `.xlsx` as a supported extension for the data explorer. In my testing, this was *incredibly* sensitive to the exact version of DuckDB WASM and requires nightly builds of the `excel` extension. (Some upstream discussion is here: duckdb/duckdb-wasm#1956.) I'd understand if we want to hold off on this until WASM support for `read_xlsx()` feels more solidified. As a secondary issue: we simply can't read all real-world Excel files, and there's no way at present to expose some of the knobs that `read_xlsx()` supports [0] (e.g. header detection, sheet selection) with the existing data explorer UI. So in many cases this path will simply fail, silently, and leave users with a blank data explorer. Addresses #4202. [0]: https://duckdb.org/docs/stable/extensions/excel.html#reading-xlsx-files Signed-off-by: Aaron Jacobs <[email protected]>
|
E2E Tests 🚀 |
| @@ -1570,6 +1570,12 @@ export class DataExplorerRpcHandler { | |||
| const query = `CREATE OR REPLACE TABLE ${catalogName} AS | |||
| SELECT * FROM parquet_scan('${virtualPath}');`; | |||
| await this.db.runQuery(query); | |||
| } else if (baseExt === '.xlsx') { | |||
| await this.db.runQuery('INSTALL excel FROM core_nightly;'); | |||
There was a problem hiding this comment.
Since this "phones home" to load the extension from duckdb.org (http://nightly-extensions.duckdb.org, see https://duckdb.org/docs/stable/extensions/installing_extensions.html), it might fail in some environments where Positron isn't able to access the public internet, but that seems like an acceptable caveat until we sort out how to bundle supported DuckDB extensions with Positron to avoid installing from the internet.
There was a problem hiding this comment.
Yeah, good callout.
| @@ -32,7 +32,7 @@ | |||
| "vsce": "^2.11.0" | |||
| }, | |||
| "dependencies": { | |||
| "@duckdb/duckdb-wasm": "1.29.0", | |||
| "@duckdb/duckdb-wasm": "1.29.1-dev68.0", | |||
There was a problem hiding this comment.
Depending on a dev version here feels a bit icky to me, however the last non-dev release of duckdb-wasm, 1.29.0, was October 2024, so the next release may not be for a while. I can ask the duckdb folks what they think about shipping a dev version to production to see. I also just learned about https://duckdb.org/docs/stable/clients/node_neo/overview.html which would enable us to get off of wasm altogether which might be a preferable path
DuckDB v1.2 launched native support for reading
.xlsxfiles with theexcelextension. This commit bumps our DuckDB WASM dependency to pick up this change, and wires up.xlsxas a supported extension for the data explorer.In my testing, this was incredibly sensitive to the exact version of DuckDB WASM and requires nightly builds of the
excelextension. (Some upstream discussion is here: duckdb/duckdb-wasm#1956.) I'd understand if we want to hold off on this until WASM support forread_xlsx()feels more solidified.As a secondary issue: we simply can't read all real-world Excel files, and there's no way at present to expose some of the knobs that
read_xlsx()supports (e.g. header detection, sheet selection) with the existing data explorer UI. So in many cases this path will simply fail, silently, and leave users with a blank data explorer.Addresses #4202.
Release Notes
New Features
.xlsxfiles (Open excel or other data files from file explorer #4202).Bug Fixes
QA Notes
Add automated tests with sample
.xlsxfiles.