crf-search JSON output request

[fff7d1bc]

Hi,

I recently come across ab-av1 and like it a lot, especially love the crf-search feature. I have large amount of video files that I've remuxed and would like to send though av1 encoding to reduce size. Quality is important for me but file size takes precedent. I like that I can use the max encode file target and target a minimal VMAF.

My target are the defaults of VMAF 95 and 80% file size ceiling, in case those cannot be met I am downstepping with 0.2 intervals as low as 93.0. For that reason I wrote a script that does it for me fully unattended and save results so I can later make intentional decision what to go with and for that I parse output of crf-search.

However, the output of crf-search is rather difficult to tame because it is not very machine parsable friendly and under conditions changes the output format.

First, by default the output into termnal has progress bar and output the samples it tried

% ab-av1 crf-search --min-vmaf 95 --preset 4 --input testfile.mkv | cat
- crf 32 VMAF 95.93 (9%) (cache)
- crf 46 VMAF 93.75 (4%) (cache)
  00:00:00 crf 38 3/3 ######################################################################################################################################################################### (eta 0s)
Encode with: ab-av1 encode -i testfile.mkv --crf 38 --preset 4

crf 38 VMAF 95.16 predicted video stream size 72.07 MiB (6%) taking 26 minutes

if I capture stdout and null route stderr I get a format that is really easy to parse and the format is visible in source code with info! macro.

% ab-av1 crf-search --min-vmaf 95 --preset 4 --input testfile.mkv 2>/dev/null | cat
crf 38 VMAF 95.16 predicted video stream size 72.07 MiB (6%) taking 26 minutes

However there might be errors, and if the minimal conditions for percentage ratio or VMAF are not met it will error, so if I redirect stderr into stdout and also have pipe there it changes entire output format into Rust's debug format resulting in non deterministic output

% ab-av1 crf-search --min-vmaf 95 --preset 4 --input testfile.mkv 2>&1 | cat
[2025-07-26T07:57:35Z INFO  ab_av1::command::sample_encode] encoding sample 1/3 crf 32
[2025-07-26T07:57:35Z INFO  ab_av1::command::sample_encode] sample 1/3 crf 32 VMAF 95.62 (8%) (cache)
[2025-07-26T07:57:35Z INFO  ab_av1::command::sample_encode] encoding sample 2/3 crf 32
[2025-07-26T07:57:35Z INFO  ab_av1::command::sample_encode] sample 2/3 crf 32 VMAF 95.34 (11%) (cache)
[2025-07-26T07:57:35Z INFO  ab_av1::command::sample_encode] encoding sample 3/3 crf 32
[2025-07-26T07:57:35Z INFO  ab_av1::command::sample_encode] sample 3/3 crf 32 VMAF 96.83 (7%) (cache)
[2025-07-26T07:57:35Z INFO  ab_av1::command::sample_encode] crf 32 VMAF 95.93 predicted video stream size 102.44 MiB (9%) taking 27 minutes (cache)
[2025-07-26T07:57:35Z INFO  ab_av1::command::crf_search] crf 32 VMAF 95.93 (9%) (cache)
[2025-07-26T07:57:35Z INFO  ab_av1::command::sample_encode] encoding sample 1/3 crf 46
[2025-07-26T07:57:35Z INFO  ab_av1::command::sample_encode] sample 1/3 crf 46 VMAF 93.62 (4%) (cache)
[2025-07-26T07:57:35Z INFO  ab_av1::command::sample_encode] encoding sample 2/3 crf 46
[2025-07-26T07:57:35Z INFO  ab_av1::command::sample_encode] sample 2/3 crf 46 VMAF 91.95 (5%) (cache)
[2025-07-26T07:57:35Z INFO  ab_av1::command::sample_encode] encoding sample 3/3 crf 46
[2025-07-26T07:57:35Z INFO  ab_av1::command::sample_encode] sample 3/3 crf 46 VMAF 95.68 (3%) (cache)
[2025-07-26T07:57:35Z INFO  ab_av1::command::sample_encode] crf 46 VMAF 93.75 predicted video stream size 47.16 MiB (4%) taking 27 minutes (cache)
[2025-07-26T07:57:35Z INFO  ab_av1::command::crf_search] crf 46 VMAF 93.75 (4%) (cache)
[2025-07-26T07:57:35Z INFO  ab_av1::command::sample_encode] encoding sample 1/3 crf 38
[2025-07-26T07:57:35Z INFO  ab_av1::command::sample_encode] sample 1/3 crf 38 VMAF 94.84 (6%) (cache)
[2025-07-26T07:57:35Z INFO  ab_av1::command::sample_encode] encoding sample 2/3 crf 38
[2025-07-26T07:57:35Z INFO  ab_av1::command::sample_encode] sample 2/3 crf 38 VMAF 94.20 (8%) (cache)
[2025-07-26T07:57:35Z INFO  ab_av1::command::sample_encode] encoding sample 3/3 crf 38
[2025-07-26T07:57:35Z INFO  ab_av1::command::sample_encode] sample 3/3 crf 38 VMAF 96.45 (5%) (cache)
[2025-07-26T07:57:35Z INFO  ab_av1::command::sample_encode] crf 38 VMAF 95.16 predicted video stream size 72.07 MiB (6%) taking 26 minutes (cache)
[2025-07-26T07:57:35Z INFO  ab_av1::command::crf_search] crf 38 successful
crf 38 VMAF 95.16 predicted video stream size 72.07 MiB (6%) taking 26 minutes

Which also do not happen if the output is terminal and stderr lands there.

Currently I am able to catch the very last line which happens to keep the expected format and parse it but I think it would be great if there would be JSON format htat would provide deterministic output. I later use those data to run ffmpeg with them, auto-encode does not work well for me due to bunch of other switches that I add there, like -metadata or mapping of streams.

My ideal workflow would be:

• Run crf-search with VMAF 95 and ratio 80.0
• If it fails, downstep VMAF by 0.2 steps until it succeed
• Keep all the data and errors for later investigation
• Fire up ffmpeg with the right parameters

For more context, the reason I do not start with setting minimal VMAF to say 93 is that I do not want 93 if I can get away with higher one and only fallback this low if no other choice works.

[mr44er]

Isn't filtering an option or THE easier option?

#299 (comment)

[fff7d1bc]

I am not looking as much for the workflow optimization but to be able to make sense out of the output in a fully unattended way. There's more usecases where it would make sense, like going over remux of entire show, over all episodes and then making intentional decision which parameters to choose so all episodes are encoded with the same parameters being one of them.

[alexheretic]

Related #241 (comment)

Regarding program friendly output, it is fairly straightforward to add basic --stdout-format json support to crf-search, just outputting a json result at the end. However, we'd probably want json updates during the process. We'll need to flesh out a rough approach to how the json should get spat out.

auto-encode does not work well for me due to bunch of other switches that I add there, like -metadata or mapping of streams.

You can add any ffmpeg args using auto-encode. There are also a few default args that crf-search and auto-encode will use that you'll lose by encoding with ffmpeg directly afterwards, but as long as you're aware then ok.

[Loufe]

As per #334 (comment), any thoughts on the following proposal for --stdout-format json in crf-search?

I.E. serialize existing Update enum variants to JSON, matching the pattern, à la sample-encode --stdout-format json.

Message Types

status — Progress during encode/score work

{"type":"status","crf":42,"run":1,"sample":2,"samples":3,"work":"encode","fps":45.2,"progress":0.35,"full_pass":false}
{"type":"status","crf":42,"run":1,"sample":2,"samples":3,"work":"score","score_kind":"vmaf","fps":120.5,"progress":0.68,"full_pass":false}

Field	Description	Type/Units
type	Message type	"status"
crf	CRF being tested	float
run	CRF search iteration	int (1-based)
sample	Current sample number	int (1-based)
samples	Total sample count	int
work	Current work phase	"encode" | "score"
score_kind	Score type (when work=score)	"vmaf" | "xpsnr" (optional)
fps	Processing speed	float (0 = unknown)
progress	Progress within run	float [0.0-1.0]
full_pass	Encoding entire input	bool

sample_result — Individual sample completed

{"type":"sample_result","crf":42,"sample":2,"score":95.32,"score_kind":"vmaf","encoded_size":1234567,"sample_size":9876543,"encode_time_secs":12.5,"sample_duration_secs":20.0,"from_cache":false}

Field	Description	Type/Units
type	Message type	"sample_result"
crf	CRF used	float
sample	Sample number	int (1-based)
score	Quality score	float
score_kind	Score type	"vmaf" | "xpsnr"
encoded_size	Encoded sample size	bytes
sample_size	Original sample size	bytes
encode_time_secs	Time to encode sample	seconds (float)
sample_duration_secs	Duration of sample	seconds (float)
from_cache	Result was cached	bool

crf_result — Intermediate CRF result (search continues)

{"type":"crf_result","crf":42,"score":93.45,"score_kind":"vmaf","predicted_encode_size":72000000,"predicted_encode_percent":45.5,"predicted_encode_seconds":1560,"from_cache":false}

Field	Description	Type/Units
type	Message type	"crf_result"
crf	CRF tested	float
score	Mean sample score	float
score_kind	Score type	"vmaf" | "xpsnr"
predicted_encode_size	Predicted video stream size	bytes
predicted_encode_percent	Predicted size as % of input	float
predicted_encode_seconds	Predicted encode time	int (seconds)
from_cache	All samples from cache	bool

Emitted after each CRF attempt that doesn't meet the target. If the first CRF tested is optimal, no crf_result is emitted.

done — Search completed successfully

{"type":"done","crf":38,"score":95.16,"score_kind":"vmaf","target_score":95.0,"predicted_encode_size":72070000,"predicted_encode_percent":6.0,"predicted_encode_seconds":1560,"from_cache":false}

Field	Description	Type/Units
type	Message type	"done"
crf	Best CRF found	float
score	Mean sample score	float
score_kind	Score type	"vmaf" | "xpsnr"
target_score	Requested min score	float
predicted_encode_size	Predicted video stream size	bytes
predicted_encode_percent	Predicted size as % of input	float
predicted_encode_seconds	Predicted encode time	int (seconds)
from_cache	All samples from cache	bool

error

{"type":"error","error":"no_good_crf","message":"Failed to find a suitable crf","last_crf":10,"last_score":92.5,"last_score_kind":"vmaf","last_encode_percent":102.3}
{"type":"error","error":"other","message":"Invalid data found when processing input"}

Field	Description	Type/Units
type	Message type	"error"
error	Error code	"no_good_crf" | "other"
message	Human-readable description	string
last_crf	Last CRF attempted (no_good_crf only)	float
last_score	Score of last attempt (no_good_crf only)	float
last_score_kind	Score type (no_good_crf only)	"vmaf" | "xpsnr"
last_encode_percent	Size % of last attempt (no_good_crf only)	float

Open Questions

• --stdout-format json (matching sample-encode) or a dedicated toggle (e.g., --json)?
• Env var support? (e.g., AB_AV1_STDOUT_FORMAT)

Prior Art

• FFmpeg -progress uses key=value format, not JSON
• HandBrake --json mixes human text with JSON, requiring preprocessing (issue)

The typed NDJSON approach (one JSON object per line, each with a type field) follows:

• MiniZinc --json-stream — Clean typed message pattern with type field for solution, status, error, etc.
• NDJSON spec — Newline-delimited JSON standard, works well with jq and line-oriented tools

---

I maintain a python GUI wrapper for this tool, and would love to eliminate the complicated regex parsing ab-av1. Happy to help implement once we agree on the format.

[Loufe]

A few design questions before fleshing out more detail:

1. Scope

My wrapper also needs JSON for encode and auto-encode. Should I extend the proposal to cover those, or keep this focused on crf-search first?

2. Hierarchical vs flattened message types?

Hierarchical (current proposal):

{"type":"status","phase":"sample-encode","work":"encode",...}
{"type":"status","phase":"encode",...}
{"type":"done","command":"crf-search",...}

Requires checking type → phase → work to dispatch.

Flattened:

{"type":"sample_encode_progress",...}
{"type":"encode_progress",...}
{"type":"crf_search_done",...}

Single field dispatch. Each type has fixed structure.

I lean toward flattened — simpler parsing, better for jq, matches MiniZinc's pattern. Thoughts?

3. auto-encode phase transition

When auto-encode finishes crf-search and starts encoding, should there be:

• An explicit transition message?
• Or just infer from first encode_progress after crf_result?

[hashhar]

I wish I had seen this issue and the discussion here BEFORE I implemented my PR. 😢

[Loufe]

Hi @hashhar, Does your PR fit with the discussed schema more or less? Any important differences?

[hashhar]

My implementation looks pretty close to your proposal.

Message types:

• sample - for each encoded sample with actual measured values (score, encoded_size, sample_size, encode_time_secs, from_cache)
• attempt - for each CRF tested with predicted values (predicted_encode_percent, predicted_encode_size, predicted_encode_seconds)
• result - Final successful result with the optimal CRF
• error - Failure with a reason field: score_too_low, encode_too_large, constraints_infeasible, or other

What's NOT implemented:

• status messages for progress updates (like "encoding sample 3/5") - I focused on results only

Example output:

{"type":"sample","crf":28.0,"sample":1,"score_kind":"VMAF","score":95.5,"encoded_size":1456789,"sample_size":2345678,"encode_time_secs":2.6,"from_cache":false}
{"type":"attempt","crf":28.0,"score_kind":"VMAF","score":95.2,"predicted_encode_percent":75.5,"predicted_encode_size":1234567,"predicted_encode_seconds":45.3,"from_cache":false}
{"type":"result","crf":28.0,"score_kind":"VMAF","score":95.21,"predicted_encode_size":1234567,"predicted_encode_percent":75.5,"predicted_encode_seconds":45.3}

Error example:

{"type":"error","reason":"score_too_low","target_score":95,"max_encoded_percent":80,"score_kind":"VMAF","crf":10,"score":91.17,"predicted_encode_percent":138.2}

The implementation uses --stdout-format=json flag and only covers crf-search for now (not encode or auto-encode).

[alexheretic]

fyi #336 (comment) my thoughts on the design.

Probably should have posted that here instead 😓

[alexheretic]

Sorry I've been a bit slow to respond and get up to date on this. Looks like my thoughts are mostly inline with the "flattened" #313 (comment) though i favour kebab-case for the type strings to better match the commands used in the cli.

I also favour continuing to use "vmaf" or "xpsnr" fields instead of "score". Doing so is equivalent, uses fewer fields and allows outputting both in future.

Perhaps the "first phase" of this can be delivered by #336 omitting encode update messages initially.

• sample-encode outputs type: "sample-encode-done" message with a few extra fields (so non-breaking change).
• crf-search outputs:
  • type: "sample-encode-done" messages identical to how sample-encode does
  • type: "crf-search-attempt" messages after all samples of each crf attempt has completed (maybe omitting the final)
  • type: "crf-search-done" message when finished.
  • type: "crf-search-error" message when failed.

I think lets include the minimum fields for the first version to roughly match what we print/log already. So defer improved error messages to later.

Then later improvements we'd:

• Add encoding progress updates
• Add auto-encode, encode, vmaf, xpsnr command support
• Add more fields people want

[alexheretic]

I've kicked off the docs in main by adding the current message spec there: https://github.com/alexheretic/ab-av1/blob/main/stdout-format-json.md

[Loufe]

Flattened works for me. @hashhar are you willing to update your PR to suit?

Let me know if I can help.