Hold final response headers until an async body verdict is ready

[torinnd]

What is the problem your feature solves, or the need it fulfills?

Making upstream_response_body_filter async (#964 / #974) allows a proxy to await response-body inspection, but the upstream status and headers may already be written downstream by the time the verdict is ready. A response that began as 200, for example, can no longer become a 403.

The response pipeline could be extended with an opt-in way to hold the final upstream header. Care would need to be taken to avoid breaking informational responses, protocol framing, downstream modules, cache behavior, or bounded streaming of large bodies.

Describe the solution you'd like

Add a request-scoped policy to ProxyHttp:

fn response_commit_policy(
    &self,
    session: &Session,
    ctx: &Self::CTX,
) -> ResponseCommitPolicy {
    ResponseCommitPolicy::Immediate
}

I have a draft implementation available on deferred-response-header, stacked on #974. I'd be happy to open this as a PR if the direction looks suitable.

In that draft, ResponseCommitPolicy::Hold bypasses cache lookup and admission so the response reaches the upstream inspection hooks. Cache management operations such as PURGE keep their normal behavior.

After response_filter and protocol framing, Pingora holds the transformed final header. Dedicated body and trailer hooks receive capability types instead of &mut Session:

• HeldResponseBody owns the current body task and exposes read-only session state plus the held header, but no writer.
• commit() consumes it and leaves the body task for Pingora.
• commit_for_streaming() clears the body task and returns CommittedResponse.
• CommittedResponse::write_chunk() writes non-terminal chunks; Pingora supplies EOS.

Pingora flushes earlier informational responses before invoking a held hook. Header-only and 101 responses pass through immediately. Missing verdicts, premature writes, invalid informational mutations, and conflicting framing fail closed. The same gate transitions are used for h1, h2, and custom upstream responses.

Describe alternatives you've considered

I'll call out a few lessons-learned / false starts I went down when developing that draft:

Capabilities rather than writer checks

A gate represented only by Session state requires every response-writing method to check that state. Capability types make the transition explicit: a held hook has no writer, and write access appears only when HeldResponseBody is consumed.

Capture after header transformation

Capturing before response_filter loses application changes and protocol framing; capturing in the downstream writer is too late when a header and body share a filtered batch. The proposed gate captures after transformation but before body filtering.

Commit through the ordered response path

Writing directly from the body hook can overtake an informational response still queued in the same batch. The gate flushes earlier informational responses before invoking the held hook, then commits the final header through downstream modules.

Mutate the held header rather than replace it

An arbitrary replacement header bypasses framing and response_filter work already applied to the upstream response. The proposed API exposes the transformed header for mutation and validates basic framing conflicts at commit.

Bypass cache rather than inspect cache hits

Cache hits use the downstream body-filter path rather than upstream_response_body_filter. Async cache-hit inspection would require a separate cache-serving change, so held requests bypass lookup and admission.

Additional context

ResponseCommitPolicy::Immediate remaining the default should result in very little disruption for those who don't need this functionality. This can be implemented with additional state branches but without any allocations.