9.0.0

Notes:

• Still some features and fixes in the backlog, and the test suite can use some improvement in its coverage, but won't be able to get to them any time soon. So just going to release this version so people can use it.
• Have been using this version myself quite extensively, so somewhat battle tested, but there are known issues. Namely if you have Docfd up for a few hours, it freezes sometimes.

---

• Swapped over to using SQLite for index

  • Memory usage is much slimmer/stays flat

    • For the sample of 1.4GB worth of PDFs used, after indexing, 9.0.0-rc1 uses
      1.9GB of memory, while 9.0.0-rc2 uses 39MB

  • Search is a bit slower

  • Added token length limit of 500 bytes to accommodate word table limit in index DB

    • This means during indexing, if Docfd encounters a very long token,
      e.g. serial number, long hex string, it will be split into chunks of
      up to 500 bytes

• Added Ctrl+C exit key binding to key binding info on screen

• Updated exit keys

  • To exit Docfd: q, Ctrl+Q or Ctrl+C

  • To exit other modes: Esc

• Now defaults to not scanning hidden files and directories

  • This behaviour is now enabled via the --hidden flag

• Changed to allow --add-exts and --single-line-add-exts to be specified multiple times

• Changed return code to be 1 when there are no results for --sample or --search

• Added --no-pdftotext and --no-pandoc flags

  • Docfd also notes the presence of these flags in error message if there
    are PDF files but no pdftotext command is available, and same with files
    relying on pandoc

• Renamed drop path command to just drop

• Added drop unselected key binding, and the associated command drop all except

• Various key binding help info grid adjustments

9.0.0-rc1

Why RC1: Feature set is stable and I use it daily, but have not updated test suite to catch up with the new features yet.

• Changed default cache size from 100 to 10000

  • Index after compression doesn't take up that much space, and storage is
    generally cheap enough these days

• Adjusted cache eviction behaviour to be less strict on when eviction happens
  and thus less expensive

• Renamed --cache-size to --cache-soft-limit

• Removed periodic GC compact call to avoid freezes when working with many
  files

• Removed GC compact call during file indexing core loops to reduce overhead

• Added progress bars to initial document processing stage

• Swapped to using C backend for BLAKE2B hashing, this gives >20x speedup depending on CPU

• Swapped from JSON+GZIP to CBOR+GZIP serialization for indices

• Changed help info rotation key from h to ?

• Renamed discard mode to drop mode

• Added command history editing functionality

• Added --commands-from command line argument

• Added --tokens-per-search-scope-level command line argument

• Concurrency related bug fixes

  • Unlikely to encounter in normal workflows with human input speed

  • 14fcc45

  • bfd63d9

  • 4e0aa67

• Replaced print mode with copy mode

• Replaced single file view with key binding to change screen split ratio
  to remove feature discrepencies

• Added narrow mode for search scope narrowing

• Renamed --index-chunk-token-count to --index-chunk-size

• Renamed --sample-count-per-doc to --samples-per-doc

8.0.3

• Fixed single file view crash

8.0.2

• Reworked asynchronous search/filter UI code to avoid noticeable lag due to
  waiting for cancellations that take too long

  • Previously there was still a lockstep somewhere that would prevent UI
    from progressing if previous search was still being cancelled

  • The current implementation allows newest requests to override older
    requests entirely, and not wait for cancellations at all

• Adjusted document counter in multi-file view to be visible even when no files
  are listed

8.0.1

• Fixed missing file path filter field update when undoing or redoing document
  store updates

• Fixed case insensitive marker handling in glob command line arguments

8.0.0

• Removed --markdown-headings atx from pandoc commandline
  arguments

• Removed Alt+U undo key binding

• Removed Alt+E redo key binding

• Removed Ctrl+Q exit key binding

• Added documentation for undo redo key bindings

• Added clear mode and moved clear search field key binding
  under this mode for multi-file view

• Added file path filtering functionality to multi-file view

7.1.0

• Added initial macOS support

  • Likely to have bugs, but will need macOS users to report back

• Major speedup from letting pdftotext output everything in one pass and split
  on Docfd side instead of asking pdftotext to output one page per invocation

  • For very large PDFs the indexing used to take minutes but now only takes
    seconds

  • Page count may be inaccurate if the PDF page contains form feed character
    itself (not fully sure if pdftotext filters the form feed character from
    content), but should be rare

• Significant reduction of index file size by adding GZIP
  compression to the index JSON

7.0.0

• Added discard mode to multi-file view

• Changed to using thin bars as pane separators, i.e. tmux style

• Added g and G key bindings for going to top and bottom of document list respectively

• Added -l/--files-with-match and --files-without-match for printing just paths
  in non-interactive mode

• Grouped print key bindings under print mode

• Added more print key bindings

• Grouped reload key bindings under reload mode

• Added fixes to ensure Docfd does not exit until all printing is done

• Slimmed down memory usage by switching to OCaml 5.2 which enables use of Gc.compact

  • Still no auto-compaction yet, however, will need to wait for a future
    OCaml release

• Added h key binding to rotate key binding info grid

• Added exact, prefix and suffix search syntax from fzf

• Fixed extraneous document path print in non-interactive mode when documents have no search results

• Added "explicit spaces" token ~ to match spaces

7.0.0-alpha.2

Changes so far for 7.0.0:

• Added discard mode to multi-file view

• Changed to using thin bars as pane separators, i.e. tmux style

• Added g and G key bindings for going to top and bottom of document list respectively

• Added -l/--files-with-match and --files-without-match for printing just paths
  in non-interactive mode

• Grouped print key bindings under print mode

• Added more print key bindings

• Grouped reload key bindings under reload mode

• Added fixes to ensure Docfd does not exit until all printing is done

• Slimmed down memory usage by switching to OCaml 5.2 which enables use of Gc.compact

  • Still no auto-compaction yet, however, will need to wait for a future
    OCaml release

• Added h key binding to rotate key binding info grid

• Added exact, prefix and suffix search syntax from fzf

• Fixed extraneous document path print in non-interactive mode when documents have no search results

7.0.0-alpha.1

Changes so far for 7.0.0:

• Added discard mode to multi-file view

• Changed to using thin bars as pane separators, i.e. tmux style

• Added g and G key bindings for going to top and bottom of document list respectively

• Added -l/--files-with-match and --files-without-match for printing just paths
  in non-interactive mode

• Grouped print key bindings under print mode

• Added more print key bindings

• Grouped reload key bindings under reload mode

• Added fixes to ensure Docfd does not exit until all printing is done

• Slimmed down memory usage by switching to OCaml 5.2 which enables use of Gc.compact

  • Still no auto-compaction yet, however, will need to wait for a future
    OCaml release