Hey guys! Today, we’re releasing the highest accuracy Qwen3.8-27B GGUFs.
Our new Unsloth AI Dynamic V3.1 GGUFs lead by 5–10% on average across Terminal Bench, DeepSWE & more benchmarks. We’re working on even better version of Dynamic quants for Qwen4
Run on 12GB+ RAM setups.
To compare GSQ-RCO and Exllama-v3, we had to exclude the token embedding layer from the x axis, as Exllama-v3 uses BF16 for it. If you consider the full download size and total RAM usage for Exllama-v3, then adding the BF16 embedding table and including all embedding sizes changes the plots. Our main plots’s x axis removes the embedding layer sizes from all quant providers. However on pure RAM usage and disk space usage vs accuracy, the below plot in the comments the true reflection of actual usage.
From their Reddit post.
Just in time for Qwen 4.
man if they could have just asked the AI if anyone with colorblindness could read these fucking charts.
No colorblindness here, charts are still fucked to read.
Well glad it wasent just me then!
The weird part is I find ai shove color blindness options into fucking everything even if you don’t ask. Lol
I mean if its for public consumption I am sure from understanding marketing you want the most people able to engage with the output. But this is just a mess to my eyes. Maybe a user got in the way on this and they wanted to keep it with in certain colours and they didnt understand color theory or marketing.
Anyone know which variant would work best with 32GB RAM / 16GB VRAM?
Hugging Face has some kind of hardware compatibility feature, but I’m trying to avoid making yet another account to use it.For a 16Gb card I use
UD-IQ4_XSwith94208of context and Q4 for KV cache. But I don’t have desktop on this GPU, so you’ll need to either reduce the context size or use a 3 bit quant.It’s is a dense model, so RAM is irrelevant. It will be too slow if the model leaks into RAM. You’ll need to put all layers on the GPU.
I’m gonna try this, but I have a feeling my machine will be too slow to run it.
What are your specs?
deleted by creator
But what is your GPU? It’s a dense model, running it from RAM will be extremely slow.
If you want to utilize your RAM, you need a MoE model.
deleted by creator
