Harvard DSR
Social Sciences
The Hidden Geometry Inside Transformers: Where Context and Position Actually Live
Think of a city layout where streets represent positions and neighborhoods represent meanings—this work reveals that transformers store these two kinds of information in separate, discoverable regions of their "mental map."
This means we're getting better at understanding how language models actually organize knowledge internally, not just what they output.
Bug reported: No