CL-Bench 数据的使用

[PeterXiaTian]

有个疑惑,这份数据既然是公开的,在未来的新大模型训练中如果用到了这份数据,打榜得分岂不是非常高?
或者,在模型打榜中,能否屏蔽这个问题?

[cypggs]

来自 notebooklm 的解答

[Ablustrund]

This is a very important question, thank you for raising it!

First, from a licensing perspective, we have made this explicit in data repository (at here):

CL-Bench is released under an evaluation-only license. Concretely, the dataset may only be used for model evaluation, testing, and benchmarking, and must not be used for any form of training, fine-tuning, calibration, distillation, or parameter updating. This restriction is clearly stated in the license.

That said, we also acknowledge a practical limitation of open datasets: it is not technically possible to fully prevent future models from being exposed to this data during training.

As such, we request that users adhere to this principle, namely that evaluation datasets should not be used for model training. 🙇‍♂️

这是一个重要的事情，感谢您的提问！

首先，从使用许可（license）的角度，我们已经在仓库中明确说明 (许可在这里)：
CL-Bench 仅以 evaluation-only 的许可形式发布。具体而言，数据集只能用于模型的评测、测试和基准比较，不得用于任何形式的训练、微调、校准、蒸馏或参数更新。这一点已在 license 中明确写出。

当然，我们也承认确实无法从技术上 100% 阻止 未来的模型在训练过程中使用这份数据。因此，我们希望并默认大家遵守这一准则：不将评测数据用于模型训练。