r/LocalLLaMA

A streamlined Hugging Face model search utility coded by Qwen 3.6-27B

Hi all. As some may have been aware, Hugging Face's model search had issues recently. (It seems to b…

[WIP] Gemma 4 MTP

Gemma 4 MTP from u/am17an It’s a work in progress so you have to compile it yourself, and you should…

How accurate can “whichllm” be?

Hello people I think the question is clear but I wanted to add some context: I work on internal tool…

RTX 5080 16GB: Qwen3.6 35B MoE at 128k context — 56 tok/s, and why MTP doesn't help

MTP (Multi-Token Prediction) just merged into mainline llama.cpp at b9190. I promised u/WarthogConfi…

Anyone else fighting Blackwell GSP timeout in production passthrough? How are you handling recovery without a host reboot?

Environment: GPU: NVIDIA RTX Pro 5000 (Blackwell Architecture, PCI ID: 10de:2bb3) Host OS: Linux (KV…

The MTP function in LMStudio causes a decrease in output quality.

The prompt is very simple, you can see it at the end. Both tests used the exact same settings, the o…

Gemma 4 MTP with LlamaCPP

I am running Gemma 4 31B for a project using LlamaCPP. There is no integrated main model + MTP draft…

What small speech to text (STT) model is best at recognizing whispered speech?

Speaking to a phone is not appropriate in all social situations. What STT model, runnable on a midra…

Impulse Purchase.

submitted by /u/Outrageous_Permit154 [link] [comments]…

Qwen3.7 Max scored by Artificial Analysis, 27B/35B waiting room

https://preview.redd.it/42ak5qmus82h1.png?width=1133&format=png&auto=webp&s=744ea3dfc06c83d0c4d8aa12…

Guardrails take an 8B model from 53% to 99% on agentic tasks [ACM CAIS '26 preprint]

submitted by /u/billy_booboo [link] [comments]…

LM Studio finally added support for MTP Speculative Decoding

https://preview.redd.it/1uuzjm0ll72h1.png?width=923&format=png&auto=webp&s=1af7d7594be1e08ff7ad6797e…