r/LocalLLaMA

Tuning Qwen 3.8 27B and OMP as a coding agent on 2× 3090s

Oh My Pi + vLLM on two 3090s. Average wait per turn went from 28s to 7s, mostly from changing omp se…

M5 Ultra and M6 Chip Benchmark Results Reveal Graphics Performance

First benchmarks submitted by /u/DustNearby2848 [link] [comments]…

Is HF starting to move against abliterated models?

Baseten launched a new safety infrastructure standard alongside its Base Labs research arm on Wednes…

768gb vram for less than the price of one RTX 6000

I have always posted about budget builds on here, and often asked how we are going to run the next b…

[Bi-Weekly Megathread] Project Showcase

Do you have something you'd like to share with the r/LocalLLaMA community. This is the place for it!…

gfx906-llama-cpp: New PP/TG gains for MI50/MI60/Radeon VII/AMD GCN

Time for another update! We have been busy and managed to improve the gains substantially (mostly fr…

The gap has closed, open source will win

I've been trying the latest models from the frontier labs and honestly, after extensive testing I ca…

NInfer vs llama.cpp vs vLLM: quality + speed comparison for Qwen3.8-27B NVFP4 on RTX 5090

I've been running Qwen3.8-27B as a local inference server for a production content intelligence pipe…

People with "non-enthusaist hardware": how do you use it?

I don't want to be super gatekeepy about this, but let's loosely define "non-enthusaist" as "a singl…

Qwen3.8-Flash-Next (UD-Q4_K_XL) on a single RTX 3090 24GB + 128GB DDR4, is this config optimal?

Qwen3.8-Flash-Next (UD-Q4_K_XL) on a single RTX 3090 24GB + 128GB DDR4 — is this config optimal? Har…

Title: MLX-DLSS: NVIDIA's DLSS 5 neural renderer and frame generator, running on a Mac (and in PyTorch anywhere)

NVIDIA ships two neural networks inside DLSS: the DLSS 5 renderer that makes game frames look more p…

AA Update! Here's how the small models score.

Ling 3.0 Tiny still seems to be leading the pack despite only having 1.3B active submitted by …