vLLM Just Merged TurboQuant Fix for Qwen 3.5+
Previously it was throwing a 'Not Implemented' error due to Mamba layers. Going to test it now! https://github.com/vllm-project/vllm/pull/39931 Edit: Works with Qwen 3.6, tested with 27B Can be used with argument; --kv-c
📄
This source provides headlines only. Use the button below to read the complete article on the original site.
📰 Read the original article on r/LocalLLaMA
Originally published by r/LocalLLaMA. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.