K-QA #45 added

[Manishram-ai]

No description provided.

[warner-benjamin]

Thanks for the PR. Could you address these two comments? Also, make sure to add the license at the top of the files if you reused any of the author's code.

[warner-benjamin]

Using the latest released version of verifiers 0.1.5.post0, the current code doesn't run.

verifiers.rubrics.JudgeRubric - ERROR - Error calling reward function <lambda>: module 'verifiers.utils' has no attribute 'ensure_async'

Also, when using a Judge model, you need to support different api keys and urls for each model. Look at healthbench as an example.

[Manishram-ai]

this pr adds

• k_qa_batched: single call scoring for comprehensiveness + hallucination

• k_qa_batched environment that mirrors k_qa but computes comprehensiveness and hallucination in one judge call.

• Scoring rubric: uses batch_eval_prompt to get both:

  • comprehensiveness.entailed_must_have_claims
  • hallucination.contradictory_generated_claims

• Metrics

  • comprehensiveness = entailed_must_have_claims_count / must_have_count
  • hallucination_count = number of contradictory generated claims

• some small bug fixes like the f" string