Scheduler refactor: base/main PR for final merge

[markurtz]

Summary

The main PR for the scheduler refactor project that all other PRs will be opened off of for review to limit PR sizes and consolidate into a central location for landing on main at once. Once all PRs are merged for the scope of the work, this will be merged into main.

The scheduler refactor is targeted at enhancing GuideLLM across a number of different targeted features and performance fixes. It rebuilds the core behind GuideLLM from the ground up with both performance and flexibility in mind. The specific features and fixes added with this refactor are the following:

• Scheduler package: new interfaces and underlying objects...
  • TODO
• Backend package: new interfaces and underlying objects...
  • TODO
• Benchmark package: new interfaces and underlying objects...
  • TODO
• Utils package: new utilities added to support the above efforts
  • TODO

Details

PRs for review (in order):

• Utilities - Scheduler refactor [utils]: auto_importer, pydantic_utils, registry, singleton #289 Scheduler refactor [utils]: functions, mixins, statistics, text #290
• Scheduler - TODO add
• Backend - TODO add
• Benchmark - TODO add
• Tests - TODO add
  • Unit tests
  • E2E tests

(Jared, Sam, Alon)

Pending work:

• Bug fixes for progress displays and enhancements
• Bug fixes and reenablement for CSV file output formats (Jared)
• Bug fixes and reenablement for HTML file output formats
• Bug fixes and reenablement for scenarios

Test Plan

• Unit tests -- Mark
• E2E tests -- Alon
  • List out scenarios to run / functionalities (Rob G, Sam)
• Perf testing -- Sam, Mark
• User flow testing -- Rob G, Ben, Mark

Related PRs/Issues

• Resolves Feat/max error rate #171
• Resolves Feat/max error rate - continued #238
• Resolves Unify RPS and Concurrent Scheduler Paths #233
• Resolves Refactor of the Scheduler package to enable or setup for: distributed benchmarks, multi-turn requests, advanced stopping criteria, large concurrency, multi modality, evaluations #251
• TODO

---

• "I certify that all code in this PR is my own, except as noted below."

Use of AI

• Includes AI-assisted code completion
• Includes code generated by an AI application
• Includes AI-generated tests (NOTE: AI written tests should have a docstring that includes ## WRITTEN BY AI ##)