webgpu : add Q2_0 support by teddygood · Pull Request #30914 · ggml-org/llama.cpp
Tested on an RTX 3060 with native Dawn WebGPU. llama-bench -m MODEL -p 512 -n 128 -r 5 -ngl 99 -dev WebGPU -t 2 -fa off Model Test master (t/s) PR (t/s) Ternary Bonsai 1.7B Q2_0 pp512 35.03 ± 0.46 1915.91 ± 3.36
📄
This source provides headlines only. Use the button below to read the complete article on the original site.
📰 Read the original article on r/LocalLLaMA
Originally published by r/LocalLLaMA. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.