Anyone using Flash Attention 2 (ai-bond) on their V100's? How is the performance?
I just Installed Flash Attention 2 from here: https://github.com/ai-bond/flash-attention-v100" I did some basic benchmarks and I am getting from 4x-7x memory utilization. However, benchmarks don't always translate to rea
📄
This source provides headlines only. Use the button below to read the complete article on the original site.
📰 Read the original article on r/LocalLLaMA
Originally published by r/LocalLLaMA. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.