MeanAvergaePrecision API

[hardikdava]

An overview of an API

from abc import ABC, abstractmethod, dataclass
from typing import Dict, List

@dataclass
class MeanAveragePrecision: 
    map: float
    map50: float
    map75: float

    @classmethod
    def from_detections(
        cls,
        predictions: List[sv.Detections],
        target: List[sv.Detections],
    ) -> MeanAveragePrecision:
        pass

    def from_tensors(
        cls,
        predictions: List[np.ndarray],
        target: List[np.ndarray],
    ) -> MeanAveragePrecision:
        pass

   @classmethod
    def benchmark(
        cls,
        dataset: sv.DetectionDataset,
        callback: Callable[[np.ndarray], sv.Detections],
    ) -> MeanAveragePrecision:
        pass

[hardikdava]

Hi @SkalskiP , do you think we should integrate ConfusionMatrix and MeanAveragePrecision into single API as ObjectDetectionEvaluation because both is gonna need bbox_iou and computing them twice is not a good idea. Another idea is that when user uses ObejctDetectionEvaluator and that will take care of ConfusionMatrix and mAP together. Mostly, both of them together is most likely the use case in practice. What do you think of this?

[SkalskiP]

Hi @hardikdava 👋🏻 let's add from_tensors here as well. 🙏🏻 To have consistent API with ConfusionMatrix.

As for the recalculation of IoU, you don't need to worry about it for now.

I plan to create the from_iou method and inject iou values from outside. But as I said, it is a feature improvement. Let's focus on the correct mAP value calculation.

[hardikdava]

@SkalskiP I have already added it to my code. The API will be similar to ConfusionMatrix. I will let you know once I made some progress.

[SkalskiP]

Awesome!