How do I get the superfast DFlash / MTP tokens per second that I'm seeing on here? Dual 3090s
I'm trying to get these high tokens per second that I'm seeing on here using the new speculative decoding techniques. Hardware: 2x3090, AMD 9900X, 32GB RAM, Gigabyte B850 AI TOP. Running Ubuntu 24.04, CUDA 13.0, NVIDIA-S
📄
This source provides headlines only. Use the button below to read the complete article on the original site.
📰 Read the original article on r/LocalLLaMA
Originally published by r/LocalLLaMA. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.