AI tools, cybersecurity and development news aggregated from top sources — saved permanently with unique URLs.
VLLM gives 5x speed of llama but quants not available (unsloth/gguf). What to do?
Hi - I want to run unsloth dynamic quant on vllm. Why? vllm is giving faster prefill speed - Llam…
I implemented Laguna (XS.2) as a model in Llama.cpp
submitted by /u/linuxid10t [link] [comments]…
unsloth dynamic quants for vllm?
Hi - I want to run unsloth dynamic quant on vllm. Why? vllm is giving 5X faster prefill speed - …
losing my mind fine-tuning jina-v5 for a legal corpus
For the last month i've been trying to fine-tune jina-v5 (which has performed best on my corpus out …
Reachy Mini goes fully local!
Hi! Andi from Hugging Face here! My team has been working over the last few months on creating a sup…
"Western Open-Weight SOTA is between Gemma4-31B and Nemotron3-Super-120B"
These are fine models, but it's one hell of a gut punch to realize this. There's a 4-way debate of C…
Zai replaced the network architecture running GLM-5.1 inference and the gains are pretty wild
Been following the infrastructure side of AI more lately and stumbled on this from Zai. They upgrade…
Annoying QwenCode v.0.16.0 - How to disable this thing? do I need to roll back to 0.15.x, disable auto-updates and call it day? why Qwen... WHY!!??
submitted by /u/JLeonsarmiento [link] [comments]…
I'm seeing low draft acceptance when using Qwen3.x MTP, what am I doing wrong?
I'm using llama.cpp, and I've tried Bartowski's and my own quants. When using Qwen3.5-122B or Qwen3.…
HF models page now has a "Base only" toggle to filter out finetunes/quants/etc
a feature that was requested a lot: https://huggingface.co/models?base_model_relation=base submit…
Distributed ML Checkpoint Storage System
Wrote up an article, diving deep into 4x Raspberry pi 4B 4GB RAM Cluster based Distributed Checkpoin…
PaddlePaddle/PaddleOCR-VL-1.6
submitted by /u/SarcasticBaka [link] [comments]…