We need to have tests that assert that on release, the sample data produces the same output through time for the same model. Right now we don't specify whether the model outputs themselves have changed. There is ground truth with each of the same data, so we can assert the eval score.
We need to have tests that assert that on release, the sample data produces the same output through time for the same model. Right now we don't specify whether the model outputs themselves have changed. There is ground truth with each of the same data, so we can assert the eval score.