Embedding Space Explorer (3D)
About this app
Words as vectors in a space of meaning: similarity as cosine in the full space, analogy arrows, neighbours and three 3D shadows (map, PCA, axes) with how much neighbourhood each one keeps.
What an embedding is
A language model stores every word or token as a list of numbers, a vector. These vectors are called embeddings. Words with similar meanings point in similar directions, and whole topics form clouds. Similarity is measured with the cosine of the angle between two vectors: cos θ = 1 means the same direction, 0 means no relation. Directions carry meaning too, which is why analogies such as king − man + woman ≈ queen work. Real embeddings are learned from text and have hundreds to several thousand unnamed axes; analogies only hold approximately there.
What the app shows
The app uses a hand-built model with 98 words and 17 named axes such as gender, status, size or capital, so the patterns work out exactly. The words belong to five groups: royals and family, countries, capitals, animals and professions. Search and arithmetic always run in the full 17-dimensional space. What you see is only a 3D shadow: a hand-arranged map, a PCA or three single axes. A neighbour fidelity score shows how many of the five true neighbours are also the nearest ones in the picture.
Things to try
Type a word such as king, tiger or Paris and look at its nearest neighbours with angle and cosine. In the vector panel, compute analogies like Germany − Berlin + Paris or doctor − man + woman; arrows show the path, and the app tells you whether a word is hit exactly or is just the nearest one. Under dimensions, switch between map, PCA and three axes and compare the neighbour fidelity: no 3D shadow reaches 100 %.
Frequently asked questions
- What are word embeddings?
- Word embeddings are vectors that a language model uses to represent words or tokens as lists of numbers. They are learned so that words appearing in similar contexts get similar vectors. This makes similarity of meaning measurable as an angle or distance.
- What does cosine similarity measure?
- Cosine similarity is the cosine of the angle between two vectors: their dot product divided by the product of their lengths. It ranges from −1 to 1, and 1 means the same direction. It compares directions only, not lengths, which is why it is the standard measure for embeddings.
- Why does king − man + woman give queen?
- In a good embedding space, the difference between man and woman corresponds roughly to a fixed direction. Subtracting it from king and adding the other form lands near queen. In learned models this only holds approximately; you look for the word with the highest cosine to the resulting point.
- What can I try in this app?
- You search for words and their neighbours, compute analogies with arrows and hide topic groups. Under dimensions you see a word’s number vector and compare three 3D projections by how well they preserve the true neighbourhood.
Subject: Neural networks
More from Neural networks
- Agent loop
- Attention Visualiser
- Backpropagation — blame in reverse
- Basin of Attraction
- Your job under AI
- The temperature dial
- Diffusion model — images from noise (3D)
- Gradient Descent (3D)
A VisuApp by heyprof: interactive, free in your browser, no sign-up. What is a VisuApp? · All apps