AI tools, cybersecurity and development news aggregated from top sources — saved permanently with unique URLs.
Training LFM-2.5-350M on Reddit post summarization with GRPO on my 3x Mac Minis — final evals and t-test evals are here
So, with this project I want to see if a length constrained (like 64 tokens only) quality summarizat…
llama.cpp DeepSeek v4 Flash experimental inference
Hi, here you can find experimental llama.cpp support for DeepSeek v4, and here there is the GGUF you…
[Qwen3.6 35b a3b] Used the top config for my setup 8gb vram and 32gb ram, and found that somehow the Q4_K_XL model from Unsloth runs just slightly faster and used less tokens for output compared to Q4_K_M despite more memory usage
Config CtxSize: 131,072 GpuLayers: 99 CpuMoeLayers: 38 Threads: 16 BatchSize/UBatchSize: 4096/4096 …
Benchmark: Windows 11 vs Lubuntu 26.04 on Llama.cpp (RTX 5080 + i9-14900KF). I didn't expect the gap to be this big.
As a life-long Windows user (don't hate me, I was exposed to it at a young age) I was wondering how …
Qwen3.6-27B-INT4 clocking 100 tps with 256k context length on 1x RTX 5090 via vllm 0.19
Thanks to the community the Qwen3.6-27B speed keeps getting better. The following improves upon my r…
Using PaddleOCR-VL-1.5 with llama-server for book OCR
I've been running PaddleOCR-VL-1.5 via llama.cpp's server for OCR on book pages. It handles complex …
Will llama.cpp multislot improve speed?
I've heard mostly bad opinions about multiple slots with llama.cpp (--parallel > 1). I guess compari…
New model for detecting and masking PII from OpenAI
submitted by /u/doesitoffendyou [link] [comments]…
kreuzcrawl, an open source Rust crawling engine with 11 language bindings
kreuzcrawl is a high-performance web crawling engine. It was designed to reliably extract structured…
The exact KV cache usage of DeepSeek V4
Figure 1 of DSV4 paper seems to imply that DSV3.2 uses ~50GB at 1m context and DSV4 uses ~5GB: https…
Experts-Volunteers needed for Vulkan on ik_llama.cpp
ik_llama.cpp is great for both CPU & CUDA. Need legends to make Vulkan better as well. https://githu…
Qwen3.5/3.6 Coder?
With practically all of LocalLlama glazing Qwen 3.5/3.6 for it's coding skills. Along with the fact …