llama3.1-8b: use TokensPrompt in the Offline vLLM SUT and fail fast on worker death

[100-JM]

Problem

SUT.process_queries (Offline) in language/llama3.1-8b/SUT_VLLM.py calls

self.model.generate(prompt_token_ids=input_ids_tensor, sampling_params=...)

prompt_token_ids= was a deprecated keyword that newer vLLM releases no longer accept. On vLLM 0.10.2 (NGC vllm:25.10-py3):

TypeError: LLM.generate() got an unexpected keyword argument 'prompt_token_ids'

SUTServer already passes pre-tokenized input as TokensPrompt objects, which works on both the pinned vllm==0.6.3 and current releases; only the Offline path was left on the old form.

The exception also exposed a second problem: it only killed the worker thread. LoadGen kept waiting for responses that would never arrive and the run sat idle until the job's wall-clock limit — four hours in our case — instead of failing.

Changes

• Offline path builds [TokensPrompt(prompt_token_ids=...) for q in qitem] and passes it positionally, exactly as SUTServer does. Same tokens reach the model; no behavioural change on 0.6.3.
• process_queries in both SUTs now wraps the worker loop: on an unhandled exception it logs the traceback and os._exit(1)s, so a broken SUT fails the run immediately instead of hanging LoadGen. If maintainers would rather keep the harness hands-off here, the first change stands alone.

Verification

• Ran the Offline scenario on vLLM 0.10.2 (B300, TP1, nvidia/Llama-3.1-8B-Instruct-NVFP4) with the equivalent patch — generate([{"prompt_token_ids": ids}, …]), i.e. the same dict TokensPrompt constructs: performance VALID, accuracy run completes and passes all five evaluation metrics (ROUGE1/2/L/Lsum and gen_len above the 99% thresholds). The fail-fast wrapper was exercised only by the original TypeError run, where the thread died and LoadGen waited four hours.
• python -m py_compile and autopep8 -a --max-line-length 79 clean. I could not exercise vllm==0.6.3 itself, but TokensPrompt has existed since 0.4.x and is what SUTServer already uses on that version.

Related: #2341 (Server/SingleStream EngineDeadError) is a different failure in the same file and is not addressed here.

🤖 Generated with Claude Code

[github-actions]

MLCommons CLA bot:
Thank you very much for your submission; we really appreciate it. Before we can accept your contribution,
we ask that you sign the MLCommons CLA (Apache 2). Please submit your GitHub ID to our onboarding form to initiate
authorization. If you are from a MLCommons member organization, we will request that you be added to the CLA.
If you are not from a member organization, we will email you a CLA to sign. For any questions, please contact
support@mlcommons.org.
0 out of 1 committers have signed the MLCommons CLA.
❌ @100-JM
_{You can retrigger this bot by commenting recheck in this Pull Request}

[100-JM]

recheck