Skip to content

Mistral crashes or outputs <unk> tokens with more than 4k tokens in the input #6

@FontaineRiant

Description

@FontaineRiant

There's a temporary fix to limit input to mitral's window size (4k tokens) but since the pytorch update, there's errors beyond that. Mistral isn't supposed to have a history cap, so it would be nice to find a proper fix.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions