How do you increase prompt processing speed ?
I am rocking Qwen like we all know, at 24GB 7900XTX 230k context, but it starts at 850t/s and then lowers to 350t/s when its at 160k context prefill speed, which is frustrating me for my long agentic runs. What is there
π
This source provides headlines only. Use the button below to read the complete article on the original site.
π° Read the original article on r/LocalLLaMA
Originally published by r/LocalLLaMA. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.