Skip to content
All apps

Transformer (GPT-2 live)

About this app

GPT-2 runs in the browser — experience token generation, probabilities and architecture live.

How a transformer writes text

A language model like GPT-2 produces text one token at a time. The text is split into tokens, each token becomes a vector (embedding) and passes through a stack of identical transformer blocks made of attention and a feed-forward network. At the end comes a probability distribution over the whole vocabulary, about 50,000 tokens for GPT-2. The next token is drawn from it, appended, and the pass starts again. This procedure is called autoregressive.

What the app shows

The app loads the smallest GPT-2, with 124 million parameters, as an ONNX model and runs it entirely in the browser; the first start downloads and caches about 500 MB. At every step a bar chart shows the top-k candidates with their probabilities, the chosen token and the remainder spread over all other tokens. In the background the pipeline lights up, from tokenizer and embedding through the 12 blocks to the softmax.

Things to try

Enter an English prompt, since GPT-2 was trained on English text. Set the temperature between 0.1 and 2.0: low values make the choice cautious and repetitive, high values bolder and more erratic. Top-k between 1 and 50 limits the choice to the k most likely tokens; with k = 1 the model always takes the front-runner. You choose an output length between 5 and 80 tokens.

Frequently asked questions

What is GPT-2?
GPT-2 is a language model released by OpenAI in 2019, based on the transformer architecture. Given a text, it predicts the next token. The smallest version has 124 million parameters and 12 transformer blocks.
What does temperature do in a language model?
Temperature divides the raw scores (logits) before the softmax. A low temperature makes the distribution sharper, so the model almost always picks the most likely token. A high temperature flattens it, and rarer tokens get chosen more often.
What is top-k sampling?
Top-k sampling keeps only the k most likely tokens and discards the rest. The next token is drawn at random from these k candidates, weighted by probability. This keeps very unlikely tokens from derailing the text.
Does the model really run in my browser?
Yes. The app downloads GPT-2 with 124 million parameters once (about 500 MB) and then computes locally in your browser. Later visits use the cache and start immediately.

Subject: Neural networks

More from Neural networks

A VisuApp by heyprof: interactive, free in your browser, no sign-up. What is a VisuApp? · All apps