AI tools, cybersecurity and development news aggregated from top sources — saved permanently with unique URLs.
Nvidia H100(94GB VRAM) - should I run llama.cpp or vllm for 30 users inference?
I was given the great opportunity to borrow a H100 with 94GB VRAM at work until it is needed by a cu…
Q4_K_M is fine for chat and a trap for agents. Here is math mathing.
saw the Q4_K_M vs Q6 thread earlier and the comments are talking past each other. "few errors per ho…
I ran 8 open-weight models as agents in a persistent MMO for 10 days. Here's the 93k event dataset and some things that I learned
Howdy everyone! Quick disclosure: I work on this - it's a project my studio created called the Null …
litellm vs any-llm (otari)
I am considering switching from litellm (sdk) to Mozilla’s any-llm. They also have a proxy to go wit…
LMStudio with MTP support - which model?
Looks like LMStudio released support for Multi-Token-Prediction (MTP) and the release notes say to u…
Llama.cpp Console released
https://github.com/alekk89/llama.cpp-Console/ for windows users submitted by /u/Bulky-Priorit…
Fused MoE dispatch kernel in pure Triton: 89-131% of Megablocks, runs on AMD with zero code changes
I've been working on MoE inference and wrote a fused dispatch kernel entirely in Triton, no CUDA. At…
Has anyone gotten their editor to work with Deepseek v4 FIM?
I tried to follow the docs here https://api-docs.deepseek.com/guides/fim_completion to get it up and…
Finally pioneering beyond the local 256k context window frontier!
The autocompact at 341.5k tokens is manually set and I'll be slowly pushing it back now I'm confiden…
Is Granite-4.1-30b Overshadowed by Qwen3.6 & Gemma4 models?
I don't see any threads on this model. Is it because it's dense and/or without-reasoning? Anyone tri…
RTX5080 vs RTX 3090 ?
Hey guys, i’m looking for some educated advice / opinions on runing local LLM. I own an RTX 5080 and…
AI is not for everyone
This may be a controversial take, but AI is not for everyone. I've made a post here before about the…