sidecar: encoder disaggregation treats every request as chat completions

[revit13]

What happened:

The sidecar's encoder (EPD) path assumes every request is a Chat Completions
request. The API type the sidecar already determined at the route is not passed
into the encoder path, and the encoder's multimodal extraction only understands
the chat messages shape. A /v1/responses or /inference/v1/generate request
that carries encoder headers is affected in two ways:

1. The API type is dropped. ecConnectorHandler (proxy.go:341) takes no
   API type, so disaggregatedPrefillHandler cannot pass the one it holds
   (chat_completions.go:198). When the encoder path later hands off to the P/D
   connector it substitutes a literal:

   // connector_ec_common.go:289
   s.handlePDConnector(w, pdRequest, prefillEndPoint, "", APITypeChatCompletions)

   The request body is not converted: the prefiller still receives the client's
   Responses body at /v1/responses. Only the label is wrong, and the label
   decides which fields the sidecar caps. Labelled as chat completions, it caps
   max_tokens, max_completion_tokens and min_tokens, which a Responses
   request does not use, and leaves max_output_tokens at the client's value.
   A generate request keeps its limits under sampling_params and is missed the
   same way. The prefill leg runs the client's full generation instead of
   stopping after one token.

2. The multimodal content is never found. extractMMItems reads
   requestData["messages"] (connector_ec_common.go:59). A Responses request
   carries its content under input; a generate request carries token_ids.
   Neither has messages, so no items are extracted, the encoder is skipped
   with a DEBUG-level log (connector_ec_shared_storage.go:34), and the request
   continues to runPDPipeline regardless (connector_ec_shared_storage.go:70).

The response returned to the client is correct in both cases. The cost is
wasted prefill work and, for (2), encoder disaggregation silently not running
for these APIs.

What you expected to happen:

• The API type determined at the route reaches the P/D connector unchanged, so
  the prefill leg is capped in the field the serving engine actually reads.
• Multimodal content in a Responses request is extracted and encoded, or the
  encoder path rejects the API explicitly instead of skipping at DEBUG level.

How to reproduce it (as minimally and precisely as possible):

No cluster required. Save this as pkg/sidecar/proxy/ec_apitype_proof_test.go
and run go test ./pkg/sidecar/proxy/ -run TestECPreservesAPIType_Responses -v:

package proxy

import (
	"bytes"
	"encoding/json"
	"net/http"
	"net/http/httptest"
	"strings"
	"sync/atomic"
	"testing"

	"sigs.k8s.io/controller-runtime/pkg/log"

	"github.com/llm-d/llm-d-router/pkg/common/routing"
)

// A /v1/responses request that also carries encoder headers should reach the
// P/D connector labelled as the Responses API, and its multimodal input should
// be encoded.
func TestECPreservesAPIType_Responses(t *testing.T) {
	var encoderCalls atomic.Int32
	encoder := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
		encoderCalls.Add(1)
		w.Header().Set("Content-Type", "application/json")
		_, _ = w.Write([]byte(`{"choices":[{"message":{"content":""}}]}`))
	}))
	defer encoder.Close()
	encoderHostPort := strings.TrimPrefix(encoder.URL, "http://")

	srv := NewProxy(Config{Port: "0", ECConnector: ECExampleConnector})
	srv.logger = log.Log
	srv.allowlistValidator = &AllowlistValidator{}
	srv.handleECConnector = srv.handleECSharedStorage

	var gotAPIType APIType
	var gotBody map[string]any
	srv.handlePDConnector = func(_ http.ResponseWriter, r *http.Request, _ string, _ string, apiType APIType) {
		gotAPIType = apiType
		_ = json.NewDecoder(r.Body).Decode(&gotBody)
	}

	// Responses API: multimodal content lives under "input" (not "messages"),
	// and the output cap is max_output_tokens.
	body, _ := json.Marshal(map[string]any{
		"model": "m",
		"input": []any{map[string]any{
			"role": "user",
			"content": []any{map[string]any{
				"type":      "input_image",
				"image_url": map[string]any{"url": "https://example.com/img.jpg"},
			}},
		}},
		"max_output_tokens": 800,
	})

	req := httptest.NewRequest(http.MethodPost, ResponsesPath, bytes.NewReader(body))
	req.Header.Set(routing.PrefillEndpointHeader, "10.0.0.1:8000")
	req.Header.Set(routing.EncoderEndpointsHeader, encoderHostPort)

	srv.disaggregatedPrefillHandler(APITypeResponses)(httptest.NewRecorder(), req)

	t.Logf("apiType seen by the P/D connector: %v", gotAPIType)
	t.Logf("encoder calls: %d", encoderCalls.Load())
	t.Logf("max_output_tokens forwarded to prefill: %v", gotBody["max_output_tokens"])

	if gotAPIType != APITypeResponses {
		t.Errorf("BUG A: P/D connector received apiType %v, want %v", gotAPIType, APITypeResponses)
	}
	if n := encoderCalls.Load(); n != 1 {
		t.Errorf("BUG B: encoder called %d times, want 1 (the image was never encoded)", n)
	}
}

Output on main at d8d0987:

=== RUN   TestECPreservesAPIType_Responses
    ec_apitype_proof_test.go:62: apiType seen by the P/D connector: chat_completions
    ec_apitype_proof_test.go:63: encoder calls: 0
    ec_apitype_proof_test.go:64: max_output_tokens forwarded to prefill: 800
    ec_apitype_proof_test.go:67: BUG A: P/D connector received apiType chat_completions, want responses
    ec_apitype_proof_test.go:70: BUG B: encoder called 0 times, want 1 (the image was never encoded)
--- FAIL: TestECPreservesAPIType_Responses (0.01s)
FAIL
FAIL	github.com/llm-d/llm-d-router/pkg/sidecar/proxy	0.028s
FAIL

Anything else we need to know?:

Both symptoms share one root cause: the encoder path has no notion of the
request API. They are separable fixes (threading the type through
ecConnectorHandler, and teaching extractMMItems the Responses input
shape), so this may be better split into two PRs.

Whether the sidecar encoder path should support /inference/v1/generate at all
is a design question for the maintainers: the coordinator handles generate
multimodal through its own pipeline, so an explicit rejection may be the right
answer there rather than extraction support.

Environment:

• llm-d-router version: main at d8d0987
• Kubernetes version: n/a (reproduced by unit test)