Skip to content

Convert XLS cell values when importing a Databook - #665

Merged
claudep merged 1 commit into
jazzband:masterfrom
vineethsaivs:fix/xls-databook-cell-conversion
Jul 25, 2026
Merged

Convert XLS cell values when importing a Databook#665
claudep merged 1 commit into
jazzband:masterfrom
vineethsaivs:fix/xls-databook-cell-conversion

Conversation

@vineethsaivs

Copy link
Copy Markdown
Contributor

The bug

XLSFormat.import_set reads cells through a cell_value helper that turns Excel date serials into datetime objects and error codes into strings like #N/A. XLSFormat.import_book re-implemented the row loop with raw sheet.row_values(i) and never got that helper, so the same file returns different values depending on whether you load it as a Dataset or a Databook.

Using the fixtures already in the test suite:

dates.xls    'birth_date'
   Dataset  -> datetime.datetime(2015, 4, 12, 0, 0)
   Databook -> 42106.0

errors.xls   'not available (formula)'
   Dataset  -> '#N/A'
   Databook -> 42

That 42 is the symptom from #202 ("Tablib import Excel NA() values as 42"), which was fixed on the import_set path only; it still reproduces today through Databook.

XLS is the only spreadsheet format where the two paths differ. Both XLSX and ODS delegate import_set and import_book to a shared import_sheet, so they cannot drift. The existing test_xls_date_import and test_xls_import_with_errors assert the converted values, but only via Dataset, and the one XLS Databook test only checks a sheet title, which is why this went unnoticed.

The fix

Extract an import_sheet classmethod and call it from both entry points, mirroring the structure _xlsx.py and _ods.py already use. The cell conversion becomes a read_cell classmethod, matching the read_cell in _ods.py, with the workbook datemode passed in explicitly rather than captured from an enclosing scope (sheet.book is not reliable under xlrd's on_demand).

Header handling is deliberately unchanged. import_set reads headers with raw row_values today and the shared helper keeps doing that, so Databook now matches Dataset exactly rather than changing both.

Tests

Two regression tests load the existing dates.xls and errors.xls fixtures as a Databook and assert the same values the Dataset tests already assert. Both fail before this change (42106.0 != datetime.datetime(2015, 4, 12, 0, 0)) and pass after.

$ pytest tests/ -q
182 passed

Full suite passes before and after; the count goes from 180 to 182 with the two new tests. Added myself to AUTHORS per the contributing guide.

XLSFormat.import_book re-implemented the row loop with raw sheet.row_values(),
so the same file loaded as a Databook returned Excel date serials and raw error
codes where a Dataset returned datetime objects and strings like #N/A.

Extract an import_sheet classmethod, mirroring the XLSX and ODS formats, and
call it from both import_set and import_book so the two paths cannot drift
again. The cell conversion moves into a read_cell classmethod, matching the ODS
format, with the workbook datemode passed in explicitly.
@codecov

codecov Bot commented Jul 20, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.47%. Comparing base (234f434) to head (4189fab).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #665      +/-   ##
==========================================
+ Coverage   93.42%   93.47%   +0.05%     
==========================================
  Files          29       29              
  Lines        3298     3308      +10     
==========================================
+ Hits         3081     3092      +11     
+ Misses        217      216       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@vineethsaivs

Copy link
Copy Markdown
Contributor Author

The only red check here is label: "Label error. Requires at least 1 of: changelog: Added, changelog: Changed, ...". I do not have rights to label the PR myself, so could a maintainer add changelog: Fixed? Everything else (tests, codecov) is green.

Recap: Databook import goes through the sheet rows directly, so XLS cell values are never passed through the same conversion the single-Dataset path applies. Dates, times and booleans therefore come back as raw floats/ints in a Databook but as proper Python objects in a Dataset from the same file. The fix routes both paths through the same conversion, with a regression test covering the mismatch.

@claudep claudep added the changelog: Fixed For any bug fixes label Jul 25, 2026

@claudep claudep left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks for this fix!

@claudep
claudep merged commit 27dffd5 into jazzband:master Jul 25, 2026
23 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog: Fixed For any bug fixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants