r/LocalLLaMA

9070xt inference for q3 qwen 27B

In llamacpp I'm getting 12tok/s, does this number look right to you and what can I do to increase th…

Chasm: A text adventure / interactive fiction game

submitted by /u/_supert_ [link] [comments]…

BeeLlama.cpp: advanced DFlash & TurboQuant with support of reasoning and vision. Qwen 3.6 27B Q5 with 200k context on 3090, 2-3x faster than baseline (peak 135 tps!)

TL;DR New llama.cpp fork! I wanted a Windows-friendly inference to run Qwen 3.6 27B Q5 on a single R…

The many sides of Mimo v2.5 Pro

I was excited to try Mimo given all the buzz on here, so before downloading the quantized local vers…

Should we use a non-thinking model for code after using a thinking one for plan? (Agentic coding)

I usually use Qwen3.6 27B (slow as heck on my RX 6800 but it works) for plan and Qwen3.6 35B A3B for…

More Qwen3.6-27B MTP success but on dual Mi50s

TLDR: The hype is real! 1.5x speedup. Up to 2x speedup with tensor parallelism! After reading the PR…

Spec decoding for minimax m2.7?

MTP was not released for m2.7, so would anyone have experience with setting up speculative decoding …

80 tok/sec and 128K context on 12GB VRAM with Qwen3.6 35B A3B and llama.cpp MTP

Just wanted to share my config in hopes of helping other 12GB GPU owners achieve what I see as very …

DeepSeek Rejects Alibaba: Prioritizing Corporate Independence Over Big Tech Ecosystems

In April, DeepSeek launched a rare, massive financing plan that attracted interest from two of China…

Pi and Qwen3.6 27B make setting up Archlinux really easy.

Just thought I'd share this use case. I was setting up a miniPC as a home theatre with Archlinux (It…

Model(s) for Creative Writing & Conversational Intuition

We can all agree that the new Qwen models are truly amazing, and we are blessed to have them. In cod…

Testing MiMo-V2.5-IQ3_S with 1'048'576 context

llama-server.exe --model "H:\gptmodel\AesSedai\MiMo-V2.5-GGUF\MiMo-V2.5-IQ3_S-00001-of-00004.gguf" -…