Haystack

an LLM reads the haystack · a skeptic and an enthusiast disagree
return to loghow it works
what the skeptic counts
scanned0
matched0
junk0
wrong guesses0
what the enthusiast counts
fragments joined0
links0
leaps that held0
stories found0/ 5
the LLM skeptic
the LLM enthusiast
skeptic enthusiast
drag to orbit · scroll to zoom · click a fragment to ask about it · ← → shift the lens · space pause · r reseed · ? how it worksdrag to orbit · pinch to zoom · tap a fragment to ask about it · slide the lens
same LLM. same search.
the skeptic's numbers are right. so are the enthusiast's.
an LLM is reading 2,600 fragments it has never seen.
five stories are hidden in them. a skeptic and an enthusiast are both watching it work.
are LLMs actually able to discover new things, or are they just searching through garbage very fast?
paused

how this works

The haystack is 2,600 fragments. Each one is two or three words of accounting vocabulary, 65 words in all, with the generic ones appearing far more often than the rare ones, the way real records do.

Five stories are planted in it. Each is a chain of twelve fragments laid along a hidden curve through the field. Neighbours in a chain share exactly one word. Fragments two steps apart share nothing.

The LLM is a simulation, not a model. A search is a wave that expands from one fragment and flags every fragment that shares a word with it. A flagged fragment counts as a link if it belongs to the same planted story and as junk if it doesn't. That check is made against the planted answer, which is exactly what a real model does not have. Treat the piece as a picture of the argument, not as evidence in it.

A leap is an attempt to connect two fragments with no word in common. Leaps succeed 70% of the time, by design. A wrong leap is drawn in full and then withdrawn, and the skeptic's ledger keeps it.

Neither ledger is cooked. Both count the same events. The skeptic's junk share is high because common words occur in hundreds of fragments. The enthusiast always ends with five stories because following a chain of shared words always gets there eventually. Which number matters is the question the page leaves to you.

Seed reproduces this haystack and this run, as long as you don't click a fragment. One file, no libraries. The 3D is points projected onto a flat canvas by hand.

close