r/LocalLLaMA

ByteShape Qwen3.6-35B-A3B: 30% faster than Unsloth IQ on 6GB VRAM laptop

A few days ago I posted about my experiments with MTP on a 6GB VRAM laptop. That didn't work so well…

Experts first llama.cpp

This is for all with 12GB VRAM. Hi, I created a fork of llama.cpp with an experimental implementatio…

I ran a quantization shootout on Qwen3-Coder and the results are... interesting

Out of random curiousity I ran a shootout on Qwen3-Coder-Next. I've been using the MXFP4_MOE from un…

Qwen-27B-IQ4_KS for ik_llama.cpp, especially for NVIDIA with 16GB VRAM

Hi everyone, I'm presenting a new quantization of the Qwen-27B model, created specifically with 16GB…

Seeking resources to read about llama.cpp server and how offloading works

SETUP INFO: Amd R9700 AI PRO. Using llama-cpp server, ROCM docker version. Using the --ngl option to…

OpenBMB presents the model BitCPM-CANN 1.58 bit

Se están probando los modelos nuevos en el Huawei Ascend 910B Link : https://x.com/i/status/20578163…

[llama.cpp] Asymmetric KV q8/q4 cache: current caveats and discussion in GGML repo

Probably most of you are aware that using anything other than -ctk q8_0 -ctv q8_0 / -ctk q4_0 -ctv q…

[NEW] Supra-50M Released!

https://preview.redd.it/kx39ammxno2h1.jpg?width=1080&format=pjpg&auto=webp&s=d1a2d5b27920a5b61a50547…

DeepSeek is pushing forward with $10.29 billion financing round, with Liang Wenfeng committing to continue developing open-source AI models rather than pursuing short-term commercialization goals

https://www.bloomberg.com/news/articles/2026-05-22/deepseek-founder-declares-agi-goal-as-10-billion-…

New Release of ROCm based MLX LLM Engine - lemon-mlx-engine

Hey everyone lemon-mlx-engine just got done integrating TheRock / ROCm 7.13 into the lemon-mlx-engin…

Gmail tie-ins

hey folks. I’m looking to setup a way to give a local LLM access to google cloud SDK for Gmail funct…

When your LLM treats data center GPUs like an optional DLC

submitted by /u/noprompt [link] [comments]…