Skip to content

Releases: openshift-psap/mlperf-inference-6.0-redhat

Hard release with supporting scripts and working harness

Choose a tag to compare

@naveenmiriyaluredhat naveenmiriyaluredhat released this 11 Feb 02:49
  1. We have the harness supporting both the Offline and the Server for gpt-oss-120b
  2. We would need to update LoadGenServerClient to handle race conditions and timing issues .
  3. We are observing negative TPOT and TTFT primarily due to the race condition between the worker thread sending first token and the other thread which sends the response before .
  4. This release containing submission automation scripts both to run submission and create correct submission structure .

Script to launch automated runs

Choose a tag to compare

@naveenmiriyaluredhat naveenmiriyaluredhat released this 10 Feb 19:59
c9b7f48

Scripts written to launch and convert into submission edible format

Compliance and accuracy fixes Offline/Server

Choose a tag to compare

@naveenmiriyaluredhat naveenmiriyaluredhat released this 06 Feb 03:12

This is an important release which fixes compliance failures observed

  1. GPT-OSS-120B has two compliance tests TEST07 and TEST09 .
  2. TEST07 verifies accuracy during performance run .
  3. The returned text from vllm was detokenized and sent as response missing out on special tokens .
  4. For Server streaming case we were also sending the First Token twice
  5. Using "return_token_ids": True , got the token_ids and sent them as a loadgen response .
  6. This fixes the accuracy issues seen so far .

Fix offline async concurrency issue

Choose a tag to compare

@naveenmiriyaluredhat naveenmiriyaluredhat released this 02 Feb 04:12

This recent fix helps to fix offline async concurrency hang . Where the client used to hang for a higher thread count . The issue is to cleanly separate the threading aspect and the allow the main thread to update to loadgen

basic functionality working

Choose a tag to compare

@naveenmiriyaluredhat naveenmiriyaluredhat released this 02 Feb 04:06
c3b576c

Working SUT . Need few more changes

Added concurrency specifier

Choose a tag to compare

@naveenmiriyaluredhat naveenmiriyaluredhat released this 29 Jan 23:30
62ba69d

Allow the harness to specify offline-async-concurrency . Uses a big of threads to issue individaul http requests.

Accuracy Check for gpt-oss-120b

Choose a tag to compare

@naveenmiriyaluredhat naveenmiriyaluredhat released this 24 Jan 20:34
3b60207

This release has a working accuracy check for offline scenario .