主观评测中AlpacaEval数据集来源

[wangxu0820]

在评测AlpacaEval数据集时遇到以下报错，在该仓库中没有找到AlpacaEval数据集的资源，于是我在 https://github.com/tatsu-lab/alpaca_eval 下载了AlpacaEval数据集放入到了data目录中。运行时遇到以下报错，看起来是数据格式的问题，有人遇到相同的问题吗？

Traceback (most recent call last):
File "user/opencompass/run.py", line 4, in
main()
File "user/opencompass/opencompass/cli/main.py", line 307, in main
tasks = partitioner(cfg)
File "user/opencompass/opencompass/partitioners/base.py", line 81, in call
tasks = self.partition(**model_and_dataset_args,
File "user/opencompass/opencompass/partitioners/size.py", line 90, in partition
comb['datasets'] = sorted(comb['datasets'],
File "user/opencompass/opencompass/partitioners/size.py", line 91, in
key=lambda x: self.get_cost(x),
File "user/opencompass/opencompass/partitioners/size.py", line 214, in get_cost
dataset = build_dataset_from_cfg(dataset)
File "user/opencompass/opencompass/utils/build.py", line 13, in build_dataset_from_cfg
return LOAD_DATASET.build(dataset_cfg)
File "user/conda/envs/opencompass/lib/python3.10/site-packages/mmengine/registry/registry.py", line 570, in build
return self.build_func(cfg, *args, **kwargs, registry=self)
File "user/conda/envs/opencompass/lib/python3.10/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg
obj = obj_cls(**args) # type: ignore
File "user/opencompass/opencompass/datasets/base.py", line 12, in init
self.dataset = self.load(**kwargs)
File "user/opencompass/opencompass/datasets/subjective/subjective_cmp.py", line 21, in load
question = problem['question']
KeyError: 'question'

[bittersweet1999]

如果你是最新的opencompass，不需要自己取下载数据集的。现在opencompass会自动下载你所需的数据集，你可以把data目录暂时remove掉，看是否会触发自动下载，正常情况下会下载到.cache下的

[wangxu0820]

感谢及时回复，刚刚重新下载没有自动下载，恰好我在 https://github.com/open-compass/opencompass/blob/main/opencompass/utils/datasets_info.py#L259 找到了alpacaeval数据集，已经解决了，谢谢！

[ftgreat]

@wangxu0820 你好，方便共享下 AlpacaEval 评测配置吗，谢谢