Newbie
Picture a room full of tiny voters. Each one looks at part of the photo and shouts louder when it sees something it likes: pointy ears, whiskers… The room says "cat" if enough of them shout. Learning is teaching them who to listen to.
Of all the fools,
I'm the king.
My AI and science notes, explained three times: for newbies, for the halfway there and for the technical crowd. The first lesson is already here, and the blog has more to read.
One idea, three cards
Every lesson comes in three levels. Pick your card once and the site remembers it.
Example: how does a neural network recognize a cat?
Intermediate
Each neuron multiplies its inputs by weights, adds them up with a bias and runs the result through an activation function. Layers chain together: early ones spot edges, later ones shapes, the last one "cat". Training means nudging the weights to cut the error.
Technical
| In ♦ we said | In ♠ we write |
|---|---|
| a layer's weights and bias | W⁽ˡ⁾, b⁽ˡ⁾ |
| what comes out of one layer and into the next | h⁽ˡ⁾ |
| multiply by the weights, add up, add the bias | affine: W⁽ˡ⁾h⁽ˡ⁻¹⁾ + b⁽ˡ⁾ |
| the activation function | σ, the non-linearity |
| the error | L, the loss |
| how much the error changes if you nudge a weight | ∂L/∂W, via the chain rule (backprop) |
| nudge the weights to cut the error | W ← W − η∇L (η: step size) |
| takes time in proportion to the number of weights | O(parameters) |
h⁽ˡ⁾ = σ(W⁽ˡ⁾h⁽ˡ⁻¹⁾ + b⁽ˡ⁾)
The network is a composition of affine maps and non-linearities. Backprop applies the chain rule to get ∂L/∂W in O(parameters), and the optimizer updates W ← W − η∇L. Without σ the whole network collapses into one linear map.
Latest on the blog
Mixture of Experts (MoE): the trick behind giant LLMs
DeepSeek-V4-Pro has 1.6 trillion parameters and uses 49 billion per word. How an MoE works, what it costs, and when a dense model is the better call.
- Artificial intelligence
- LLM
- Deep learning
- Machine learning
MoE myths: the experts aren't experts
They don't know medicine and they aren't eight glued-together models. Six Mixture of Experts myths, debunked with what Mixtral, OpenMoE and others measured.
- Artificial intelligence
- LLM
- Deep learning
Could Fullmetal Alchemist's alchemy work in real life?
Earth walls, steel spears, gold and people: what each FMA transmutation would cost with real physics, what exists today and why the real problem is time.
- Fullmetal Alchemist
- Anime
- Physics
- Chemistry
- Thermodynamics
- Nanoscience
- Energy
- Nuclear physics
Why does King of Dumbs exist?
What this site is, why every lesson is explained three times, who the jester keeping you company is, and what's coming: AI, data, quantum and nanoscience.
- About the site
- Artificial intelligence
- Quantum
- Nanoscience
Looking for something?
Filter by type of post and by topic.
Lessons
The first one is out. Next up: AI, data, quantum and nanoscience, with widgets you can poke.
How does a computer turn a photo into numbers?
A photo is millions of numbers, and almost everything AI does with it is multiply and add. Vectors, the dot product and matrices at three levels, with two widgets.
- Mathematics
- Artificial intelligence
Why doesn't diamond turn into graphite?
Breaking bonds costs energy and making them pays it back, but almost no reaction starts without climbing a hill. Why diamond lasts and water works as a battery.
- Chemistry
- Energy
- Thermodynamics