This is somewhat related to #462.

The problem:

If I call model.predict(), I get a label. If I call model.predict_proba() and try to get the final string via model.labels or model.id2label, I get a different label.

Why this happens:

I think it's because the sklearn model head (where the probabilities are extracted from) might reorganize the labels, leading to an inconsistency between what the model thinks the order is and how the probs are returned.

Workaround

Doing model.model_head.classes_[int(np.argmax(probs))]. However, I think that's not the intended way since classes_ is a private attribute?