Tja wer hätte gedacht, dass das grob fahrlässige Verhalten von Hersteller und Supermarktketten irgendwann mal um auf die Füße fällt. Es gibt einfach kein fähiges Management mehr in der Wirtschaft.

Jep ein moderner, nicht smarter, reparierbarer und haltbarer PKW mit Elektroantrieb fehlt derzeit am Markt.

One use case is auto mode for your local coding agent (automatic permission approval).

There is a lot of room for optimizing the inference for specific models. For example, when I switch llamacpp to Strata, I get to 45ts (granted on a slightly poorer quant).

You don’t need to load the full model into VRAM. In my case it is offloaded into RAM. Some also get away with offloading to an NVME SSD.

Yup great combo. It baffles me why anyone one would favour to depend oneself on an online service instead

Yup worth a try. I got from 13ts to 21ts for Qwen 3.8 Flash Next on a RX 9070 XT 16GB. As always, the additional memory demand requires some compromise. For me reducing context from full 262k to 200k and reducing kv cache down to q8_0 was the best option. moe-cache-mib at 6544. Cache hit rate at ~70%.