Model Reliability v.s. Model Robustness #11
Jingkang50
started this conversation in
General
Replies: 1 comment
-
|
Calibration is also one way to evaluate whether the model is reliable or not. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Some readers asked me about the difference of these two terms.
Here is my thought.
Model Reliability:
Reliability modeling is the process of predicting or understanding the reliability of a component or system prior to its implementation. [ref]
Model Robustness:
A model is robust when the accuracy does not change significantly from the baseline accuracy under various conditions. [ref]
So in my opinion, OOD detection is pursuing model reliability, as the models are required to estimate the probability whether the given sample is ID. OOD detection is all about confidence. A good example of model robustness is OOD generalization, which requires stable performance even with domain shifts. OOD generalization is all about accuracy.
In sum:
OOD detection -> Model Reliability
OOD generalization -> Model Robustness
Beta Was this translation helpful? Give feedback.
All reactions