Skip to content

Releases: caltechlibrary/datatools

Incremental Improvemnts

29 Sep 21:35
Compare
Choose a tag to compare

Bugs fixes, documentation fixes, jsoncols now uses encoding/csv for output.

jsonjoin improvements

28 Sep 19:22
Compare
Choose a tag to compare

Add support to explicitly create an empty root JSON object to join against or specify a root JSON object read from standard input or a specified file. Improved documentation describing default behavior as branching merge with options for flat merges via -update and -overwrite.

jsonjoin

27 Sep 19:00
Compare
Choose a tag to compare

Added a jsonjoin for joining JSON documents that contains objects.
Normalize some command line options.

Harmonizing csv commands

21 Sep 19:06
Compare
Choose a tag to compare

All the csv commands now support setting a character delimiter. This allows us to use the csv commands with other delimiters (e.g. colon or pipe delimited files). Unless noted the delimiter affects both input and output.

Here's an example of using csvfind one a pipe delimited file called file.txt looking for "Fred" in the second
column.

    csvfind -i file.txt -d '|' -col 2 "Fred"

Combined release of shelltools and datatools

15 Sep 22:56
Compare
Choose a tag to compare

This release marks the merging of shelltools and datatools projects. Moving forward improvements, bug fixes will happen in the datatools project.

New parameters and installation process

23 Jun 21:33
Compare
Choose a tag to compare

The primary feature of this release is individual Zip files for compiled binaries based on platform. Additional two commands, jsoncols and jsonrange, now have a "-permissive" option which suppress errors codes and messages (useful in Bash scripting where an empty string is sufficient).

Column counting from 1 instead of zero

14 Apr 16:38
Compare
Choose a tag to compare

I've updated the command line options for csvcols, csvfind and csvjoin to reference column number starting with column one rather than column zero. This makes the behavior match how cut works (e.g. -f 1 for first field) as well as how most people think of counting columns in spreadsheets.

new cli, bug fixes and cli option normalization

13 Apr 21:45
Compare
Choose a tag to compare

Changes:

  • csvcols has an option change, -f is now -col

Additions:

  • csvfind will help you find things in a CSV file by searching a single colum
    • search options include exact match, contains and Levenshtein edit distance
  • csvjoin will let you join two CSV files on a column
    • includes support for approximate matching via Lenenshtein edit distance on join column

Documentation has been re-organized. Beginning to add a "How To" section for website. Lots of small BUG fixes.

datatools Go package is beginning to combine common functionality used between the CLI. dotpath continues to be debugged.

Added csv2json cli

11 Apr 18:08
Compare
Choose a tag to compare

Added a new command line tool named csv2json. This will convert a CSV file into a series of JSON blobs output as either a JSON array or one JSON blob per line. Use case is turning a CSV file into a set of JSON blobs that could be easily stored in dataset.

Minor tweaks, bug fixes and improvements

10 Apr 16:25
Compare
Choose a tag to compare
  • Adding leading and trailing new line for CSV to Markdown table conversion (csv2mdtable).
  • Improved docs.
  • Linted and formatted Bash scripts.
  • Improved Makefile