Is it ever advisable to delete all training images and start over with Frigate+ custom models?

[shred86]

It's been over a year since I've updated the trained model I have with over 1,300 images. There's a couple new labels I'd like to add but obviously having to revalidate all these images isn't something I'm exactly looking forward to. 😬 That raised the question in my mind, is it perhaps better to just "start over" with a new base model? My thoughts are:

• Haven't some of the images I submitted been used for the newer base model? I realize it's probably impossible to tell just how many, if any, of them were used. However, it's obviously been updated with more home environments in general so I would expect performance to be better compared to the old base model.
• I did have some backyard landscaping changes, so I'm not sure if continuing to train a new model off the old backyard is the best thing to do. There were some bushes in particular that drove a lot of false positives that are since gone.

Anyways, curious what folks thoughts are. I figured I'd just start with a new base model, maybe capture and verify a smaller set of images that are lower confidence and train an updated model on that.

[inkeep-ai-assistant[bot]]

Based on the available sources, deleting old training images and starting over is generally not recommended for Frigate+ models.(1)

When cameras are moved or environments change (like your landscaping updates), you don't need to delete your existing images.(1) The documentation indicates that users have moved cameras multiple times without deleting old images and haven't experienced issues.(1)

Regarding your specific concerns:

Base Model Improvements: Your intuition is correct that newer base models incorporate improvements from community submissions.(2) Frigate+ models are built by fine-tuning a base model with your annotated images, and the base model is trained from a sampling of images across all Frigate+ user submissions.(2) When you request a new model, it will automatically use the current base model version, which benefits from these community contributions.(1)

Landscaping Changes: Rather than deleting your old images, the better approach is to submit new true positives from the updated environment.(3) The documentation recommends submitting both true positives and false positives, aiming for approximately 80% true positives and 20% false positives across all images.(3) If you were experiencing false positives in specific areas (like those bushes), submitting true positives for any object type near that area in similar lighting conditions helps teach the model what that area looks like when no objects are present.(3)

Your Proposed Approach: Your plan to request a new model with the current base and add a smaller set of lower-confidence images is sound. You can simply request a new model on the Models page, which will use your existing verified images plus any new ones you add.(3) Each model request uses one of your 12 annual trainings.(3)

The diversity in your existing 1,300+ images should continue to help the model generalize well, even with the environmental changes.(2)

📚 Sources:

1. Frigate+ Base Model 2024.1 Update
2. FAQ
3. Requesting your first model

---

Have another question? Just tag @inkeep.