DeepSeek R1评估结果细节以及Greedy Accuracy的generation参数

[stackoverflowwwww]

请问leaderboard中的DeepSeek R1的generation参数是如何设置的，temperature，top_p，top_k，repetition_penalty这些。

以及Greedy Accuracy的generation参数和gpass保持一致吗

[jnanliu]

Sorry for the late reply, you can refer to opencompass/config_templates/close.py for evaluating api-based models. For the Deepseek R1 model, if you use the offical api, almost all sampling parameters are not supported.

However, if you deploy R1 by yourself, you can set the paramters through the inference engines like vllm and deploy.

In our experiments, we use the deployed r1 in our group, so we can ensure that sampling parameters are consistent with other models.