- 27 Posts
- 53 Comments
11·3 days agoQwen3-Coder?? What year is it?!
Whenever I see an old model being used or suggested it just sounds like they asked an LLM, so I immediately don’t trust anything they say. LLMs love suggesting old models.
7.72 TB, I don’t even have enough SSD storage for this in the entire house lol
if this is real then just the fact that it’s 14T is interesting info about the frontier models
makes me feel like China’s models are at least as efficient as the US frontier models, they’re only behind in hardware
EDIT: this seems to be fake https://huggingface.co/burner22/numbat/discussions/13
this is not a regression just a performance inefficiency
I have 8GB VRAM and 32GB RAM, mostly using llama.cpp to run Tiel 35b at Q4, it’s ok at writing code in an agent as long as it can compile and run tests to iteratively fix its mistakes.
Though there’s been new optimized engines, like Strata, that could maybe allow you to run Qwen Flash Next at decent speeds.
There’s also Qwen 4 coming out soon that may give some good new options for us VRAM poors. Just stick to MoE models.
Usually people don’t use it themselves, you just go on huggingface and search for “Qwen 3.6 35b heretic” or something like that for a premade heretic model.
What are the specs on the computer?
This can happen because of tokenization. Tell it to execute code to do the letter counting and both models will succeed every time
Lol yeah someone tell them about local LLMs, open weights, and abliteration
ANTHROPIC IS CRACKING DOWN ON VIDEO GAME REWRITES 🚨
Claude Code picked up a new rule around Oct 27, sent from Anthropic’s servers under a flag called “tengu_heron_brook”. There’s no setting to turn it off.
The full rule: “Do build original games, games in a genre, and games from the user’s own designs and assets. Don’t build a recreation of a specific existing game, even if it is renamed or called original.”
Is this the end of game rewrites, modding and everything in between?
But it isn’t illegal, so why?
so typical of the American models to be the best at maximizing prices lol
I thought Kimi K3 was funny https://foodtruckbench.com/blog/kimi-k3
It almost seems like it was trying to be as helpful as possible by feeding as many people as it could, too nice for its own good
I agree, I was just pointing out that the issue you quoted earlier is easily avoided
without the possibility of veering off into a tangential rant about the nature of free will.
you can do that with a regular LLM by using structured responses
for home users they would be more palatable if they didn’t require any extra VRAM, if it was just a different “mode” for your normal LLM
LM Studio is good to start with, I do wish it was open source. Internally it just uses llama.cpp except with a GUI for configuration so it’s a bit easier to use. Unsloth Studio is similar to LM Studio except open source, but I haven’t tried it yet.
I actually have LM Studio on a Linux live boot USB along with Ling 3.0 Tiny, and it’s preconfigured to run with the Vulkan backend (no CUDA in a live boot, and might plug it into an Intel or AMD graphics). I haven’t needed to use it yet but I figure it could be nice in a pinch away from home.
try the Strata engine instead of llama.cpp
https://github.com/Niko1221/Strata/blob/main/docs/MODELS.md#pick-by-memory
You should be able to run Qwen 3.8 Flash Next decently
With Strata, things have gotten more flexible. Similar optimizations are coming to llama.cpp eventually.
https://github.com/Niko1221/Strata/blob/main/docs/MODELS.md#pick-by-memory
It says Qwen 3.8 Flash Next can run with just 8GB VRAM and 96GB RAM for a good quant of it. This is a really good model, and the new version 4 should be coming in the next month or so, probably the same size but smarter.
DeepSeek v4.1 Flash is probably even smarter, says it needs 12+GB VRAM and 64GB of RAM. For decent speed I think you’ll want at least 32GB VRAM and 256GB RAM for this one. 96GB VRAM should be very good.
You’re gonna have big diminishing returns chasing for even bigger models, partly because Strata doesn’t support anything bigger and llama.cpp is currently not nearly as optimized. There might be an inference engine I don’t know about for the biggest models.
what are the specs? Try Ling 3.0 Tiny, that’s probably the fastest decent model, it should be both smarter and faster than Qwen 3.5 4b
Sure if compaction was free then I’d do it all the time. One advancement I’d like to see is MoE models that use fewer active parameters/experts when compacting. We already have DeepSeek 4.1 that uses a different number of active parameters depending on what its doing (8b for prefill and 16b for decode) so it doesn’t seem unreasonable to use fewer for compaction. It’d be like using a REAP model, but without having to load the model separately.