Mistral crashes or outputs <unk> tokens with more than 4k tokens in the input

[FontaineRiant]

There's a temporary fix to limit input to mitral's window size (4k tokens) but since the pytorch update, there's errors beyond that. Mistral isn't supposed to have a history cap, so it would be nice to find a proper fix.

[FontaineRiant]

Crashes are fixed with a update to pytorch to the new ROCm version. It must be a mismatch with the driver's ROCm version.
Once updated, the <unk> tokens appear instead but those can be avoided by reducing max_tokens to the sliding window's size.

[FontaineRiant]

Reducing the sliding window's doesn't completely fix the <unk> token issue.

[FontaineRiant]

be3504e
Not using 'eager' attention implementation seems to fix <unk> tokens