New objects for consideration

[EK701]

Some new detection objects for consideration:

• Frog
• Lizard
• Snake
• Airplane
• Helicopter
• Drone

Thanks for all the work on Frigate!

[Archy54]

I am not an expert on AI or Machine vision so take with a grain of salt. The developers will know more. From my quick research the issues are having the man hours put in to annotate lots of angles and photos, boundary box the object and add to training datasets + have it still small enough to fit model sizes the frigate style hardware uses. I believe the priority is most used objects (named head so lookup head vs tail machine vision), cliff notes head are objects seen a lot and have lots of images available vs tail where they are seen less from what I understand and harder to get accuracy on.

I know there are limitations on hardware and model sets but can custom trained models based off LVIS but with less of the categories/objects (eg snake, rat, chicken?) be added to a second detector in a cut down version of say snake, rat, chicken, op's request or is the longtail issue going to reduce accuracy to detect? I'm new to learning machine vision so apologies if I get anything wrong. My guess is they aren't added because it's a LOT harder to detect those objects because of them being tail if that's the right term.

So can frigate currently do frigate base model + frigate+, and then another one customized with extracted categories from lvis or another training set running on a different detector? Or is it really just not worth it yet until there's far more manhours put into training datasets for those objects? My guess is the accuracy is going to be very low vs head objects/categories (to the op not an actual head, commonly seen objects with lots of work done to annotate them, etc). It's a deep dive. Hope this comment is ok, feel free to delete it if isn't. Cost to train models is huge although I am surprised rat/mouse wasn't included as they cause millions of dollars of damage to infrastructure. (in COCO, not frigate itself).

[NickM-27]

There's a few things here:

• the main models are trained on COCO, so unless COCO changes or there is another model that contains the objects you want, labels won't be added there by Frigate
• Frigate+ does have more objects added often, but many of these objects IMO do not make sense.

If you wanted to detect a snake, you are effectively talking about a line, that would be extremely prone to false positives and it would require extremely sensitive motion detection.

Here is a snake on my camera

[EK701]

Thanks! Yes, I was referring to Frigate+ and I wasn't clear in my post. I could see how a snake would be prone to false positives.

[NickM-27]

it looks like it won't be useful for small snakes, but for places with large snakes it may be added in the future