Support for Torchvision Datapoints?

[edwardclem]

🚀 Feature

Hi everyone! Curious on everyone's thoughts here - are there any plans to support the new torchvision datapoints API as input for vision-centric metrics (mostly mAP)? The alternative is just to convert datapoints back to tensors before sending them through the metrics, which certainly isn't bad.

If this is in the roadmap, happy to take a stab at it.

[SkafteNicki]

Hi @edwardclem, thanks for raising this issue.
I was actually not aware of the new datapoints API, so thanks for making me aware of that.

My first troughs are that we are not going to support this directly, that said (without having tested this in anyway I would expect that most metrics actually should work out of box because the datapoints are just torch.tensor and will be auto unpacked into normal tensors whenever they encounter some operation.

For mAP specifically it maybe makes sense to support BoundingBox and Mask datapoints, but I would then wait until this feature is stable (and has seen some adoption), so it will first be something we want to consider in the future.

cc: @justusschock and @Borda for opinions.

[Borda]

For mAP specifically it maybe makes sense to support BoundingBox and Mask datapoints, but I would then wait until this feature is stable (and has seen some adoption), so it will first be something we want to consider in the future.

Sounds good to me...
@edwardclem, Would you be interested in implementing it?

[edwardclem]

Sure, happy to! I'll hold off on implementing until the torchvision feature is a bit more mature.