What Makes a Useful Representation? Geometry, Factors, Relations, and Abstraction in Machine Learning

A neural network can turn an image, a sentence, or a complex observation into a vector. But what makes that vector meaningful? Why do some representations transfer across tasks, reveal semantic directions, or capture relationships that are difficult to see in the original data?

This article looks beyond embeddings as mere collections of numbers to explore the structure they can encode—from geometry and manifolds to reusable features, relationships, and uncertainty. Along the way, it examines how representations are learned, compared, aligned, and interpreted, and develops a central idea: a useful representation is one that makes the right information and computations easier to access.