Replace TAP filtering with split-output NDJSON pipeline#16
Open
friedenberg wants to merge 1 commit into
Open
Conversation
Switch every test-output surface to amarbel-llc/tap's new split mode so stdout (or, for batsLane, the inline build-log echo) carries only failure NDJSON records, while passing records land in a side-channel file. This dramatically reduces the noise an interactive user or build log has to scroll through. - bats wrapper (nix/packages/batman.nix): pipe through `tap-dancer format-ndjson --split --pass-out` by default. New flags `--no-split` / `--full-output` restore the TAP-14 stream; `--pass-out PATH` overrides the per-run tmpdir for passing records. The hand-rolled hide-passing TAP filter is removed. - batman orchestrator (packages/batman/src/batman.ts): drop the duplicate TS-side hide-passing filter; pass `--no-split` through to the bats wrapper when `--full-output` / `--no-split` is set. - batsLane (nix/packages/bats-lane.nix): new `splitNdjson ? true` option layered on `emitNdjson`. Default produces `$out/run.failures.ndjson` + `$out/run.passes.ndjson`; the inline stderr echo carries failures only with a one-line pass-count summary. `splitNdjson = false` keeps the legacy combined `run.ndjson` and full echo. - Tests + docs updated: bats_wrapper.bats's four hide-passing tests replaced with split / no-split / pass-out / alias coverage; batman and bats-lane manpages updated.
This was referenced May 25, 2026
friedenberg
added a commit
that referenced
this pull request
May 25, 2026
- batman.nix: add pkgs.socat to the bats wrapper runtimeInputs so fence can find socat for its bridge setup even when callers strip /home/* from PATH (e.g. test-batman-fence). Without this, fence was resolving socat only via ~/.nix-profile, which is gone after the PATH strip. - bats-lane.nix: drop the redundant `| tr -d ' '` around `wc -l < file`. GNU wc-on-stdin emits the count with no leading whitespace, so the strip is dead. - batman.ts: drop `fullOutput` from the ParsedArgs return shape. It's only consumed inside parseArgs to build `passthrough`; no caller of parseArgs ever reads it. - bats-testing.7.scd: document that --no-tempdir-cleanup also preserves the wrapper's auto-generated --pass-out tmpdir (caller-supplied --pass-out paths are still never auto-cleaned). Signed-off by Clown 🤡 (https://github.com/amarbel-llc/clown)
This was referenced May 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR replaces the
--hide-passingTAP filtering mechanism with a new split-output NDJSON pipeline. Instead of filtering TAP output on stdout, the wrapper now:run.passes.ndjson), keeping build logs focused on failures--no-split/--full-outputflags to restore the historical TAP-14 stream with all records on stdout--pass-out PATHto override the passing-records file locationChanges
Core wrapper changes (
batman.tsandbatman.nix)--hide-passingflag and themakeHidePassingFilter()TAP filtering function--no-splitand--full-outputaliases to disable split mode--pass-out PATHto specify custom passing-records output locationtap-dancer format-ndjson --splitinto the output pipelinerunGroup()to remove TAP filtering logic and always inherit stdoutBuild system changes (
bats-lane.nix)splitNdjsonparameter (defaulttrue) to control output splittingsplitNdjson = true: Producesrun.failures.ndjsonandrun.passes.ndjsonwith inline stderr echo of failures onlysplitNdjson = false: Produces combinedrun.ndjsonwith full inline echo (original behavior)Test updates
bats_wrapper.batsto use--no-splitin existing tests that expect TAP-14 output--hide-passingtests with new split-mode tests:bats_wrapper_split_emits_failures_only_on_stdout: Verifies failures appear on stdout, passes don'tbats_wrapper_split_writes_passes_to_pass_out: Verifies passing records are written to the filebats_wrapper_full_output_is_alias_for_no_split: Verifies--full-outputworks as an alias--no-splitDocumentation updates
bats-testing.7.scdto document--no-split,--full-output, and--pass-outflagsbats-lane.7.scdto describe the split/combined NDJSON output modes andsplitNdjsonparameterndjson_failure_demo.batscomments to reflect the new split behaviorRationale
The split-output approach provides better signal-to-noise in build logs by default (failures only on stdout) while preserving full test results in structured NDJSON format. The
--no-splitescape hatch maintains backward compatibility for consumers that need the historical TAP-14 stream.Test Plan
Existing test suite covers the new behavior:
bats_wrapper.batsvalidates split mode, full-output mode, and pass-out file handlingbatman.batsvalidates argument parsing for the new flagsbats-lane.nixintegration tests exercise bothsplitNdjson = trueandfalsepathshttps://claude.ai/code/session_017uRx2GuzzkccSduBwG2cX4