Modern language models learn rich internal representations, but these representations are difficult to compare, interpret, or ground in a stable theory of meaning. Different models may solve similar linguistic tasks while organizing their hidden spaces in apparently different ways.
This project asks whether language representations have canonical forms: task- or meaning-derived structures that learned representations converge toward. The goal is to understand which parts of a representation are arbitrary, and which parts are forced by the linguistic or semantic structure being represented.
If such canonical structures can be identified, they could provide a basis for comparing models, explaining features, and building more principled interpretability tools for language systems.