Qwen3-Coder-Next-UD-Q4-K_XL.gguf on a Ryzen 9/4080ti. Run a 48GB SOTA Tensor-Balanced on a $2K set of House Parts.

The Qwen3-Coder fork released by Unsloth just benchmarked in performance proximity with 235B sized models and is looking to have SOTA level performance for April 2026! Here is the low-down.  At the time of this writing the model was so new that HuggingFace.com did not have the auto-full benchmarks out for it, but we were able to find them. Again incredible accolades to Unsloth AI, who just keep making incredible models!

The best part is you can run frontier models such as this Qwen3-Coder that are efficient, fast and reliable on under $2000 in house computer parts! Here is the setup:

/usr/bin/llama-server --jinja \
-m /home/c/models/Qwen3-Coder-Next-UD-Q4_K_XL.gguf \
--host 192.168.1.3 \
--n-gpu-layers 999 \
--flash-attn on \
--override-tensor "\.ffn_.*_exps\.weight=CPU" \
--cache-type-k turbo3 \
--cache-type-v turbo3 \
-c 262144 \
--temp 0.7

The key was right here, as the normal '--fit' actually fails on llama-cpp. what '--fit' does is the llama-server will then try to balance the model between the VRAM/CPU.. It cannot do it. However this actually worked!

--override-tensor "\.ffn_.*_exps\.weight=CPU" \

This is telling the llama-cpp that heavy tensor models - load to the GPU, light layers keep on the CPU.  This overcame the RAM/VRAM bottleneck plauging setups everywhere and people were spending $1000's to buy unified VRAM systems such as Mac Pros.

Turbo3 is an Advanced Quant Fork of Llama-cpp!

• Did you notice the non-normal --cache-type-k turbo3 / --cache-type-v turbo3?  Those are the bleeding edge as of March 2026 TurboQuant forks.
• Full credit to 'The Tom' who graciously created one of the TurboQuant forks!

GitHub - TheTom/turboquant_plus

Contribute to TheTom/turboquant_plus development by creating an account on GitHub.

GitHubTheTom

Would you Like a Detailed Guide for a Custom Compiled Llama-ccp TurboQuant Fork?

Here is a full self-compiling guide where a 112B used the same TensorBalance method to fit it, ran on a 16GB GPU, and ran the rest in RAM. Incredibly it was good - and usable with a respectable 12.8 Tokens/s!

Qwen3.5-122B-A10B-Q4_K_M.gguf - Run it at 13 Tokens/s with 262,000 Contexts on a Ryzen 9 3900 and a 4080ti. w/128GB RAM.

Qwen3.5-122B-A10B-Q4_K_M.gguf - Run it at 13 Tokens/s with 262,000 Contexts on a Ryzen 9 3900 and a 4080ti. w/128GB RAM.

Hot Configthinkmeltprotonmail.com

You will need the specialized llama-cpp fork TurboQuant w/  that override-tensor configuration.  In our case wow. 25.48 t/s  as we ask it to one-shot an Asteroids game that is 3x more complex.

Is it powerful? We asked it to one-shot an entire Asteroids game, then asked it to one-shot a game that was 3x more complex.  It got a majority of it right.