v0.3.8

Release 0.3.8

Bookokrat now supports DJVU

This release adds pure-Rust DJVU support to my terminal EPUB / PDF book reader.

Couple users asked for the feature and this turned into small weekend exercise in binary parsing, fuzzing, benchmarking and cache-aware optimizations.

DJVU is a weird format. Did you know that Yann LeCun is one of the creators? That probably explains the heavy math and layering: tiny changes deep inside a codec can produce wildly non-linear visual differences. The dude just loves this stuff.

Full disclaimer: the djvu crate is very much AI slop assisted implementation: I've borrowed test set from djvujs (they are public domain), generated golden snapshots with djvulibre tooling and handed Claude every spec I could gather and let it rip. Ralph style.

About 4 hours of "Lollygagging..." later:

 Pixel-exact match (the GOOD):
  - chicken (color, BG-only)
  - boy_jb2 (grayscale, mask-only)
  - boy_jb2_rot90 (rotation)
  - navm_fgbz_p1 (3-layer)
  - djvu3spec_p5 (3-layer, bundled document)

  With tolerance (the BAD):
  - carte_p1 — 8,488,224 / 32,205,600 bytes differ (26.4%)
  - colorbook_p1 — 2,362,821 / 24,875,820 bytes differ (9.5%)
  - navm_fgbz_p4 — 17,633 / 25,245,000 bytes differ (<0.1%)

Not bad, but not even close for a book reader. Eventually I gave up and ask AI to look at djvujs sources: that got the worst pages down from 26.4% and 9.5% difference to 5.0% and 1.6%.

  With tolerance:
  - carte_p1 — 1,598,859 / 32,205,600 bytes differ (5.0%) — tolerance 1,600,000
  - colorbook_p1 — 386,545 / 24,875,820 bytes differ (1.6%) — tolerance 390,000
  - navm_fgbz_p4 — 9,553 / 25,245,000 bytes differ (<0.1%) — tolerance 10,000

Bookokrat is already AGPL - and I was not planning to white-wash the AGPL license - now officially can not: inherited djvulibre dna.

The results are decent. It’s pure Rust now, and since DJVU is a dying format, spending more than two days on it felt wrong.
But also Bookokrat is for reading books and the current rendering has text-quality issues.

Because DJVU is weird! DjVu stores pages in three layers: a photo layer for images and backgrounds, a stencil that marks where text is, and a color layer that gives text its color. It is smart and super efficient for book scans, the problem is that the stencil is created by the encoder, which has to guess what's text and what's not. And that guess will be forever baked in the djvu file.

If I want my text rendering to be bolder, I can try to widen holes in the stencil and it works for correctly encoded holes. But the mistaken holes grow too, and this creates nasty visual artifacts:

The tradeoff is to have more readable text or image without artifacts. ¯\_(ツ)_/¯

In practice, inverted colors turned out to be the most readable mode for text:

DJVU is a surprisingly under-specified format and djvulibre is AGPL, so all paid DJVU readers in the world render books differently. And the difference is strikingly noticeable.

Fuzzing

I've added fuzzing with cargo-fuzz, using libfuzzer-sys (libFuzzer via Rust bindings).
Six fuzz targets covering the full stack — one per codec plus a full document→render pipeline.

30 minutes of fuzzing found 3 issues: a dictionary bounds panic in JB2, an integer subtract overflow, and an OOM from a fuzz input declaring a 64-million-pixel bitmap. Pretty sure a couple more hours would uncover even more.

Benchmarks

Then I added Criterion-based benchmarks: 8 test cases covering different layer configurations. Easy wins by reusing pre-computed calculations and inlining a few hot paths.
Somewhat interesting one was the IW44 wavelet transform: the original code used Bytemap accessor methods (bm.get(), bm.add(), etc) with #[inline(always)]. Replacing them with direct slice access - one bm.data.as_mut_slice() at the top, then raw indexing - made a visible difference. The column pass was also iterating columns-then-rows (cache-hostile), transposing it to rows-then-columns with reusable state vectors improved locality.

forbid(unsafe_code)

The IFF parser returns a tree of chunk references that borrow directly from the input byte slice — zero-copy parsing. But the public API needs to own the bytes and the parsed tree together in a single Document struct. Moving the owned Box<[u8]> would invalidate the borrows. The initial version used unsafe to fake a 'static lifetime:

  pub struct Document {
      parsed: rdjvu_document::Document<'static>,
      _data: Box<[u8]>,
  }

  impl Document {
      pub fn from_bytes(data: Vec<u8>) -> Result<Self, Error> {
          let data = data.into_boxed_slice();
          let stable_ref: &'static [u8] =
              unsafe { std::slice::from_raw_parts(data.as_ptr(), data.len()) };
          let parsed = rdjvu_document::Document::parse(stable_ref)?;
          Ok(Document { parsed, _data: data })
      }
  }

It worked, but... but the soundness depended on struct field drop order. self_cell is a well known solution that encodes the ordering in the type system.

#![forbid(unsafe_code)]. Good

---

That's pretty much it: weekend project. About 10k lines of code, which should probably be 6k.

Not going to lie: it looks pretty cool in dual-page mode. And this is still in terminal. Mind-blowing.

Other changes in the release

• Revived Windows support.
• Improved terminal detection: better Kitty protocol detection for unknown and Kitty-compatible terminals.
• More robust title extraction for cleaner book metadata in the library and book stats.
• Navigation improvements: better Ctrl+d, Ctrl+u, Ctrl+f, and Ctrl+b behavior, plus a full-page navigation off-by-one fix.
• Popup and edge-case cleanup: shared popup behavior and various corner-case fixes across the app.