70B base model benchmarks? #87
BarfingLemurs
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I care about 70B 2.3-2.55bit comparisons, for use with a single 24gb. What are the default MMLU and perplexity scores when you use a 4bpw base llama model (or those model sizes most similar to GPTQ variants)?
P.S.
Is the codebase (the quantization conversion), in development?, the nature of converting these models require a lot of bandwidth and compute, so updating and sharing large models is more difficult.
Beta Was this translation helpful? Give feedback.
All reactions