RewardBench_Evaluating_Reward_Models_for_Language_Modeling.md

RewardBench: Evaluating Reward Models for Language Modeling

This week's paper is RewardBench: Evaluating Reward Models for Language Modeling.

RewardBench is the first toolkit for benchmarking reward models. In addition to the benchmark, the authors compare scaling, test reasoning capabilities, highlight three buckets of refusal behavior, and share details on the inner workings of RMs.

Further Reading:

• Updated benchmark
• Video on RewardBench by the author
• It's not PPO > DPO, It's policy generated data > stale data twitter thread