AI tools, cybersecurity and development news aggregated from top sources — saved permanently with unique URLs.
PFlash: 10x prefill speedup over llama.cpp at 128K on a RTX 3090
Hey fellow Llamas, thank you for all the nice words and great feedback on the last post I made. We h…
GitHub - intel/auto-round: A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.
submitted by /u/muyuu [link] [comments]…
OpenAI's Privacy Filter vs GLiNER on 600 PII samples
Both models are open weight, both run on a local CPU workstation, both detect PII in text. Quick run…
MiMo-V2.5-Pro - the actual best open-weights model
Following an impressive shake-up by Kimi K2.6, I've now got some results for Xiaomi's MiMo-V2.5-Pro.…
gemma-4-31B-it-DFlash has been released
https://huggingface.co/z-lab/gemma-4-31B-it-DFlash I guess we'll have to wait until this PR is merge…
Qwen3.6-27B - Closed-loop SVG Images
Yesterday, I saw an impressive presentation of Qwen 3.6 27B's SVG capabilities on the sub. To maxim…
Whats the latest status on 7900xtx multi-GPU setups?
I am currently running dual RTX 5060 ti 16gb (both of which are easy to sell or re-use in other PCs …
Got DFlash speculative decoding working on Qwen3.5-35B-A3B with an RTX 2080 SUPER 8GB
## Got DFlash speculative decoding working on Qwen3.5-35B-A3B with an RTX 2080 SUPER 8GB I managed t…
By when do you think will TurboQuant get a proper release and be adopted by everyone
The gains when using asymmetric setup on K and V are quite huge submitted by /u/Crystalagent47…
Blog: AI evals are becoming the new compute bottleneck
Hi! I wanted to share my new blog on the costs of running AI Evals. We dig into how benchmarking fro…
I hate this group but not literally
True story, I got interested in AI after seeing it at work and wanted to run models locally. I start…
What in tarnation is going on with the cost of compute
Does anyone know? I can’t even find a server gpu submitted by…