Geometry Meets Attention: Understanding SVDA from the Ground Up Based on our newly (2025) published IEEE Access paper: Geometry Meets Attention: Interpretable Transformers via SVD Inspiration Self-attention is the engine behind modern Transformers, but standard dot-product attention is a black box. Why does a model attend to certain tokens? Can we understand what dimensions matter […]
