field/extensions: bound the vector length read by AsyncReadFrom

[tzh476]

Description

Vector.AsyncReadFrom takes the vector length from the first four bytes of the reader and allocates on it immediately:

sliceLen := binary.BigEndian.Uint32(bufSizeSlice[:])

n := int64(4)
(*vector) = make(Vector, sliceLen)

Nothing stands between the decode and the allocation. The field is a uint32 and each element occupies 4*fr.Bytes, so four bytes of input can ask for tens of gigabytes before any element data has arrived — io.ReadFull on the element bytes only fails afterwards, once the memory is already committed.

This refuses a length that the reader demonstrably cannot back, when the reader can report its remaining size (bytes.Reader, bytes.Buffer, strings.Reader all can). A reader that cannot is left alone, so no caller loses functionality.

Measurement

Apple M3 Pro, a 4 byte input claiming 2^20 elements:

	B/op	allocs/op	ns/op
before	16,777,426	6	506,264
after	210	5	8,153

A valid vector is unaffected: 360 B/op before and after.

I used 2^20 for the benchmark to keep it cheap; the ceiling is what the field allows, math.MaxUint32 * 4 * fr.Bytes.

The change is in the template

field/*/extensions/vector.go is generated and marked DO NOT EDIT, so patching it alone would be reverted by the next regeneration. The fix is in internal/generator/field/template/extensions/vector.go.tmpl; the two generated files here are the output of go generate ./internal/..., which reported 1233 files generated and left exactly these two changed.

Testing

go test ./field/koalabear/... ./field/babybear/... — 13 packages, all pass, before and after regeneration.

I did not add a test because the behaviour is only observable as an allocation size, which is awkward to assert portably. Happy to add one along the lines of "a 4 byte input claiming 2^20 elements returns io.ErrUnexpectedEOF" if you would like it — that part is deterministic.

Disclosure

The site was found by a static checker I wrote that looks for allocations sized by a length field decoded from untrusted bytes with no bounds check in between, and this change was prepared with AI assistance. The before/after numbers are from benchmark runs on this diff, and I will answer review questions myself.

---

Note

Medium Risk
Untrusted binary deserialization is security-sensitive; the guard only applies when the reader implements Len(), so generic io.Reader callers may still allocate on bogus lengths until read fails.

Overview
Vector.AsyncReadFrom no longer allocates from the untrusted uint32 length alone. When the reader exposes remaining size via Len() int (e.g. bytes.Reader), it rejects lengths that cannot fit in the bytes still available—returning io.ErrUnexpectedEOF before make(Vector, sliceLen)—which blocks huge allocations from a tiny malicious header.

The change is applied in internal/generator/field/template/extensions/vector.go.tmpl and e4_test.go.tmpl, with regenerated babybear and koalabear outputs. TestVectorAsyncReadFromLengthBound asserts an error and low allocation for a 4-byte header claiming 2²⁰ elements, and confirms well-formed vectors still round-trip through async unmarshal.

^{Reviewed by Cursor Bugbot for commit 1f97732. Bugbot is set up for automated code reviews on this repo. Configure here.}