SVDA

sepdek September 24, 2026

My July 2025 introduction to SVDA focused on understanding attention. Here I want to explore a practical question: once we can inspect a model’s internal directions, how do we decide which ones to keep—and how do we check that an implementation change does what we intended? Research context Our paper, Prescriptive SVD-Inspired Attention via Spectral […]

sepdek July 17, 2025

Geometry Meets Attention: Understanding SVDA from the Ground Up Based on our newly (2025) published IEEE Access paper: Geometry Meets Attention: Interpretable Transformers via SVD Inspiration Self-attention is the engine behind modern Transformers, but standard dot-product attention is a black box. Why does a model attend to certain tokens? Can we understand what dimensions matter […]

sepdek April 8, 2024

A depth map can look like a convincing relief of a scene. That visual impression is useful, but it does not tell us whether its values are in metres, whether two objects have the correct relative size, or whether the geometry is accurate enough for documentation. Before using a predicted depth map, I would ask […]