Releases: openshift-psap/mlperf-inference-6.0-redhat
Releases · openshift-psap/mlperf-inference-6.0-redhat
Release list
Hard release with supporting scripts and working harness
- We have the harness supporting both the Offline and the Server for gpt-oss-120b
- We would need to update LoadGenServerClient to handle race conditions and timing issues .
- We are observing negative TPOT and TTFT primarily due to the race condition between the worker thread sending first token and the other thread which sends the response before .
- This release containing submission automation scripts both to run submission and create correct submission structure .
Script to launch automated runs
Scripts written to launch and convert into submission edible format
Compliance and accuracy fixes Offline/Server
This is an important release which fixes compliance failures observed
- GPT-OSS-120B has two compliance tests TEST07 and TEST09 .
- TEST07 verifies accuracy during performance run .
- The returned text from vllm was detokenized and sent as response missing out on special tokens .
- For Server streaming case we were also sending the First Token twice
- Using "return_token_ids": True , got the token_ids and sent them as a loadgen response .
- This fixes the accuracy issues seen so far .
Fix offline async concurrency issue
This recent fix helps to fix offline async concurrency hang . Where the client used to hang for a higher thread count . The issue is to cleanly separate the threading aspect and the allow the main thread to update to loadgen
basic functionality working
Working SUT . Need few more changes
Added concurrency specifier
Allow the harness to specify offline-async-concurrency . Uses a big of threads to issue individaul http requests.
Accuracy Check for gpt-oss-120b
This release has a working accuracy check for offline scenario .