r/LocalLLaMA

Levi: Run AlphaEvolve on your local QWEN 30B

Hi r/LocalLLaMA, Wanted to share something I'm excited about. I've been fascinated by AlphaEvolve an…

Xiaomi just claimed 1,000+ tps on a 1T model using a standard 8-GPU server

Just saw Xiaomi MiMo announce MiMo-V2.5-Pro UltraSpeed, claiming they broke the 1,000 tokens/sec out…

Nex N2 has a funny "few words do trick" reasoning

I've been playing with Nex N2 Pro (Qwen 3.5 397B finetune) locally today. I noticed straight away th…

Luce Spark: a 35B MoE on a 16 GB GPU, without the offload tax

Hey fellow Llamas, your time is precious, so I'll keep it short. TL;DR: 33-35B MoE on a 16 GB GPU. …

Gemma 4 QAT + MTP: max 33% speed increase in token generation, any ideas?

Hello, My setup is 2x RTX 3060 Ti 8GB, without the assistant model (MTP) I get around 75t/s, adding …

Looking for a local "NotebookLM for lawyers" setup – what am I doing wrong?

Hello everyone I am totally new to LocalLLMs and only used chatGPT/Claude/NotebookLM before. So bear…

OpenEnv is now owned by HF, Torch, Prime Intellect, Unsloth, Modal, Mercor, and more! Use it for training agents.

OpenEnv is a tool for creating an agentic execution environment like terminals, browsers, or anythin…

[3090] Gemma4 QAT + MTP quick TPS numbers [TLDR 1.2-1.8x better]

These last few weeks have been godsend for 24GB (and below) gpu poor peeps. Killer models released …

llama-launcher Release

Hello everyone, I've been working on a point and click GUI to make tinkering with llama-server flags…

mtmd : add video input support by ngxson · Pull Request #24269 · ggml-org/llama.cpp

Show your videos to Gemma or Qwen today submitted by /u/jacek2023 [link] [comments]…

Gemma 4 Chat Template now has preserve thinking

submitted by /u/seamonn [link] [comments]…

Used local Ollama (gemma4:e4b + nomic-embed-text) to bulk-generate AI summaries for 4300 arXiv papers and push them to a remote Cloudflare DB — pipeline walkthrough

I built ArxivExplorer, a semantic arXiv search engine with AI-generated summaries. The live version …