AI tools, cybersecurity and development news aggregated from top sources — saved permanently with unique URLs.
The power of structured workflows and small local models
A month ago, I experimented with a very basic home-rolled agent loop with a handful of tools and fou…
llama: avoid copying logits during prompt decode in MTP by am17an · Pull Request #23198 · ggml-org/llama.cpp
time to update your llama.cpp -> improved prompt processing speed submitted by /u/jacek2023 …
MiroThinker-1.7, an open-weight deep research agent (Qwen3 MoE base) — mini is 30B/3B active, curious what tok/s people get on consumer hardware
As usual, disclosure first: I'm on the team that built this. Our MiroThinker-1.7-deepresearch and 1.…
Made a simple template manager and GUI for llama.cpp so I don't have to keep memorizing CLI flags.
Introducing Hexllama Hey, I’ve always found llama-server to be more than enough for testing out loca…
Developers who use local AI - Q4_0 vs Q8_0 KV quant?
I'd love to hear from developers who use big context windows if they notice a difference? Obviously …
MTP for Qwen3.6-35B-A3B on 6GB VRAM laptop: not worth it
I have an Asus gaming laptop from 2021 that I bought used for 500€ last year. I wanted to see if the…
How do I get the superfast DFlash / MTP tokens per second that I'm seeing on here? Dual 3090s
I'm trying to get these high tokens per second that I'm seeing on here using the new speculative dec…
Recent Developments in LLM Architectures: KV Sharing, mHC, and Compressed Attention
submitted by /u/seraschka [link] [comments]…
Qwen3.6-27B MTP depth benchmark — RTX 3090Ti
Hardware: RTX 3090Ti, 64GB RAM Model: unsloth/Qwen3.6-27B-UD-Q4_K_XL. (MTP Version) Prompt: "make a …
MTP mmproj and uncensored gguf
I'm quite confused how to migrate my models library to be mtp enabled. Can i use the non-mtp mmproj…
85 GPU-hours comparing 5 abliteration methods on Qwen3.6-27B: benchmarks, safety, weight forensics - Abliterlitics
I've been building Abliterlitics, an open-source abliteration forensics toolkit. The idea is straigh…
Open Source vs frontier models on a single-file HTML canvas driving animation - results
Hey yall, I was inspired by this post : https://www.reddit.com/r/LocalLLaMA/comments/1tf3p6c/local_q…