How to summarize CDME result?

[b2r66sun]

Only sub item results found in the .txt result file, while the .csv result file remains blank.

How to configure it properly to gather the evaluation results?

Btw, the infer stage only has config with cdme_8k cdme_32k cdme_200k, why it emerges Length 1000? What does the Depth mean?

CDME_Length1000Depth0: {'score': 90.00000000000001}
CDME_Length1000Depth5: {'score': 89.13043478260872}
CDME_Length1000Depth10: {'score': 94.78260869565219}
CDME_Length1000Depth15: {'score': 95.21739130434784}
CDME_Length1000Depth21: {'score': 89.56521739130436}

[Mor-Li]

Hi b2r66sun,

It seems like the issues you are encountering might be related to using an outdated version of the OpenCompass repository, particularly during a Needle In A Haystack evaluation. I recommend updating your local repository to the latest version. You can do this by running git pull in your OpenCompass directory to ensure you have the most recent updates and fixes.

For detailed instructions on how to properly configure and run the Needle In A Haystack evaluation, please refer to the updated documentation in my latest pull request: PR #1102. This should help clarify the process and ensure you can gather the evaluation results effectively.

Please check out the updated documentation and let us know if you have any further questions!

[b2r66sun]

Thanks, I'll try it with an updated one