Understanding and Enhancing Kimi Delta Attention [R]
TLDR: We demonstrate and explain the difference in expressivity of Gated Deltanet (GDN) and Kimi Delta Attention (KDA). We show how the full diagonal gate in KDA can act as a reflection allowing 2D rotations to be carrie
![Understanding and Enhancing Kimi Delta Attention [R]](https://preview.redd.it/v7oqopy3v1rh1.gif?frame=1&width=140&height=140&auto=webp&s=e5a3fafa125d82696dd092844108cc6917ec33c6)
π
This source provides headlines only. Use the button below to read the complete article on the original site.
π° Read the original article on r/MachineLearning
Originally published by r/MachineLearning. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.