Undergraduate researcher · Caldwell University
Hover any word. The arcs are scaled dot-product attention — real softmax over real dot products, computed live on toy embeddings of these words. It is the same arithmetic on page 8 of the paper below, and the same mechanism every language model runs billions of times a second.
About
I'm a Computer Science undergraduate at Caldwell University, studying how language models decide what to say next. My research with Dr. Sushil Poudel takes the attention mechanism apart symbol by symbol and puts it back together — and my companion project, Attention Atlas, lets anyone do the same by hand, in a browser.
I'm drawn to the places where AI is still uneven: what models are confident about versus what they actually get right, and how poorly today's tokenizers handle languages like Nepali. I like building things that make hard ideas feel reachable, and I'm impatient to work on problems I don't yet know how to solve.
- Studying B.S. Computer Science, Caldwell University — expected May 2028
- Researching Transformer interpretability and next-token uncertainty in LLMs
- Building Open, interactive tools that make the mathematics of AI hand-checkable
- Next Tokenization and uncertainty in Devanagari-script languages
Research & artifacts
Two things, built to go together.
Paper · manuscript, 2026
Every Symbol Explained: A Dependency-Ordered Path to the Attention Mechanism
Software · archived, citable
Attention Atlas — an interactive course on how transformers work
Measured in the paper, on GPT-2:
- 0.36 → 14.19 bitsnext-token entropy across a single temperature sweep
- 63×variation in nucleus (top-p) pool size from one context to another
- “2+2=” → “ 3”the model's single most likely completion
From the paper
How sure is the model?
Before every word, a language model holds a probability over every token it knows. Temperature reshapes that distribution. Entropy measures how spread out it is, in bits. Nucleus sampling keeps only the smallest set of tokens that covers 90% of the mass — the bright bars. Drag the temperature and watch all three move at once.
The logits here are toy values chosen to be legible. The measured GPT-2 numbers — including why formulaic openers turn out to be the sharpest category — are in the paper.
“”NEXT TOKEN?
softmax(z / T) over eight candidate tokens · hover a bar for its probability
- Entropy
- —bits
- Nucleus, p = 0.9
- —tokens
- Top token
- —
What I'm working on next
Not every language costs the same.
GPT-2 and most models like it read bytes, not characters. A Latin letter is one byte. A Devanagari character is three. Before a model has learned anything at all, Nepali already costs roughly three times as much to represent as English — which eats context, inflates cost, and shows up as worse performance for the languages that can least afford it.
Type anything below. The numbers are exact UTF-8 byte counts, computed in your browser.
Elsewhere
Find the rest of it.
Contact
Say something.
Research questions, collaborations, or anything about the Atlas — this reaches me directly. If you'd rather not use a form, email works just as well.