AI tools, cybersecurity and development news aggregated from top sources — saved permanently with unique URLs.
Distributed inference in DwarfStar
submitted by /u/Interesting_Key3421 [link] [comments]…
Qwen3.6-35B-A3B-APEX / 128K ctx on RTX 3060 12GB — 37 t/s gen with 72k ctx filled, PPL 3.25, offloading 17GB model
I'm posting this because it may be helpful to squeeze the 12GB VRAM in the 3060. All credit goes to …
Local run for multi users: which software set?
Context: I am testing and running local LLM on Linux for some months, first with llama.cpp and now w…
I need more storage...
submitted by /u/goldbookleaf [link] [comments]…
MiniMax M3 Is Coming Up
https://preview.redd.it/h0cwpag7vu3h1.png?width=1178&format=png&auto=webp&s=454b0289c2cf78c10bab1d80…
Krasis update: Qwen3.6-35B-A3B (Q4) at reading speed, 1x 8GB 3070 Mobile laptop (32GB RAM)
Context Krasis is an LLM runtime for running models that don't fit into VRAM. Krasis streams the mod…
My new home office radiator 🥵
4 x RTX Pro Max-Q We will not speak about the 64GB system RAM... submitted by /u/lantern_lol …
Qwen/Qwen-Image-Bench · Hugging Face
Model Description Q-Judger is a vision-language model fine-tuned specifically for automated evaluati…
What's your favorite local MCP server?
I've seen so many rag this, memory that projects. What projects are people actually using day to day…
Question: Llama cpp, whats good right now for: MTP, KV cache quant, Long context.
Used the vllm version of https://github.com/noonghunna/club-3090 It worked fine for myabe 20 40k con…
Nvidia LocateAnything - Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding. (10x faster than Qwen3-VL)
https://huggingface.co/nvidia/LocateAnything-3B https://github.com/NVlabs/Eagle demo https://hugging…
Data Gathering
Hello everyone I'm looking to gather some information about local model users for a college project.…