r/LocalLLaMA

Let's build claude code from scratch!

So, I made this video about how to create claude code from scratch. Here's the video: https://youtu.…

1M datasets on HF !

This community is gold ! Congrats for pushing AI forward together with open datasets ! submitted …

Anyone running Mimo-v2.5 quants with multimodal and MTP?

Has anyone been able to run Q4 or Q5 of XiaomiMiMo/MiMo-V2.5, with functioning multimodal capability…

After 8 months of running everything local, ive accepted the productivity tools also have to be local

Quick context: M3 max 64gb, currently running llama 3.3 70b q4 as my daily driver via ollama, qwen3 …

Local LLM autocomplete + agentic coding on a single 16GB GPU + 64GB RAM

Today I set up a full coding toolbox on a single RTX 5080 (with RAM offloading) that's actually viab…

MagicQuant (v2.0) - Hybrid Mixed GGUF Models + Unsloth Dynamic Learned Quant Configurations + Benchmark table with collapsed winners and more

I spent the past 5+ months building a pipeline that creates hybrid GGUF quant mixes. I also built it…

MTP+GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 - llama.cpp

I was wondering what will be the difference in results with flag: GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 …

Gemma 4 MTP vs DFlash on 1x H100: dense vs MoE results

Benchmarked Gemma 4 MTP and z-lab's DFlash on a single H100 80GB using vLLM and NVIDIA's SPEED-Bench…

Which model is SOTA rn? 7~8B for coding

For fine tuning on ada RTX 5000 to work with python framework submitted by /u/omar07ibrahim1 …

examples : add llama-eval by ggerganov · Pull Request #21152 · ggml-org/llama.cpp

now you can evaluate your models at home, sounds like a perfect tool to compare quants and finetunes…

How do I use MTP?

Hi, I'm trying to use MTP with llama.cpp, I built from source the mtp-pr, download an MTP model from…

Best practice for accurate translation at minimal cost?

I've been meaning to translate forum post type content for one of my partner's sites. Objective to o…