AI tools, cybersecurity and development news aggregated from top sources — saved permanently with unique URLs.
llama-server router: a model pinned to one GPU still grabs a CUDA context on every card, so it OOMs when my others are full. Am I missing a flag or is this just how it is?
Running into something annoying with llama-server in router mode (`--models-preset`) and I can't tel…
I built a PyTorch MoE/MoD training framework with custom CUDA kernels [Apache 2.0]
PyTorch framework for training transformer LLMs with MoE and MoD architecture support, custom CUDA k…
Qwen 3.6 27B on DeepSWE
Overview: It scored 2% (1.79% rounded up) It is 18/20th place scoring above Haiku 4.5 and Minimax M…
2-bit QAT model releases
So far model releases that take advantage of Quantization a Aware Training (QAT) have been focused o…
MTP and QTA - what is the relation?
I'm an old guy and I hate when things change so fast surrounded by noise and breaking news! MTP, I k…
NVFP4 on llama.cpp?
Hey everyone, Even through I check the subreddit daily, some things are a bit hard to grasp for me d…
QAT variant of Gemma4 26B A4B is not working well for me
I am using llama.cpp version b9549 with this arguments as recommended: llama-server --temp 1.0 --top…
Control a 3D avatar with language instead of buttons
I built a 3D character you can control with language: https://programasweights.com/avatar Traditiona…
GMKtec Crams OCuLink, Wi-Fi 7 and Dual PCIe 4.0 Into the EVO-X3, With a 192GB Ryzen AI MAX+ 495 Monster Following Later This Year
First strix 495 hardware i have seen announced/leaker. Looks like decent hardware upgraded io. No p…
Guys, it just happened
My x99 just died. F submitted by /u/robertpro01 [link] [comments]…
Budget llm for chatting and analysing pdf documents
My dad and I want an llm that can scan pdfs and get the useful data out of it at a reasonable speed …
Qwen3.6 35B-A3B on a Laptop: My Zero to One Moment
Hi everyone, I'm new here - because I only have a laptop and I only just realized local models are a…