Skip to content

AttributeError: 'CombinedEvaluations' object has no attribute 'evaluation_modules' #603

@shunk031

Description

@shunk031

Hi, I tried to run the following script by combining two metrics, but the error occurred:

Python 3.9.16 (main, Apr  5 2023, 13:58:15)
Type 'copyright', 'credits' or 'license' for more information
IPython 8.18.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import evaluate

In [2]: evaluations = ("accuracy", "f1", "precision", "recall")

In [3]: clf_metrics = evaluate.combine(evaluations=evaluations)
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[3], line 1
----> 1 clf_metrics = evaluate.combine(evaluations=evaluations)

File ~/.pyenv/versions/3.9.16/envs/my-dev/lib/python3.9/site-packages/evaluate/module.py:1034, in combine(evaluations, force_prefix)
   1008 def combine(evaluations, force_prefix=False):
   1009     """Combines several metrics, comparisons, or measurements into a single `CombinedEvaluations` object that
   1010     can be used like a single evaluation module.
   1011
   (...)
   1031     ```
   1032     """
-> 1034     return CombinedEvaluations(evaluations, force_prefix=force_prefix)

File ~/.pyenv/versions/3.9.16/envs/my-dev/lib/python3.9/site-packages/evaluate/module.py:884, in CombinedEvaluations.__init__(self, evaluation_modules, force_prefix)
    881     self.evaluation_module_names = list(evaluation_modules.keys())
    882 loaded_modules = []
--> 884 for module in self.evaluation_modules:
    885     if isinstance(module, str):
    886         module = load(module)

AttributeError: 'CombinedEvaluations' object has no attribute 'evaluation_modules'

If we use a list format for evaluations, it works fine, but I think we should also support tuple format, which work in the same way as list. This can be easily achieved by modifying the following section.

https://github.com/huggingface/evaluate/blob/main/src/evaluate/module.py#L877

May I create a PR?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions