A side project built around one question. Instead of training a model on a large dataset, can you build a system whose understanding emerges from live experience, using a small set of simple, interpretable update rules? No neural networks, no gradient descent, no train/test split. The system just runs, and running is how it learns.
The test platform is deliberately small: a simulated planar robot arm interacting with a ball on a plane. It runs in minutes and still shows where the idea holds and where it breaks.
The core idea
A system understands its world to the extent that it can predict it. Learning is the process of reducing the gap between what the system predicted and what actually happened. That gap is the prediction error. The question is whether a handful of rules, all variations on "adjust your internal state in proportion to prediction error", can reproduce the patterns seen in biological learning.
The first real decision was prediction error versus correlation. The two look similar until you run a blocking experiment:
- Train cue A so it reliably predicts a shock.
- Then present A and a new cue B together, still followed by the shock.
- Test B on its own. The response is almost nothing. B never got learned, even though it appeared with the shock every time.
A correlation rule has to learn B, because the two always occur together. A prediction-error rule does what biology does: by the time B appears, A already predicts the shock, so there is no surprise, so nothing drives learning about B. That result rules out correlation as the trigger and sets prediction error as the basis for the rest of the project.
How it runs
Each claim is its own round: a small experiment with a hypothesis written down before running it, the prior work it borrows from, and a one-word verdict. Rounds are never reordered or deleted. A round that turns out wrong is closed as a failure and left in place.
Two rules keep it honest. Every term in every formula has to mean something physical; if a term cannot be explained, the design has drifted and I stop. And negative results count as results, so there is no tuning the parameters until something appears to work.
The log so far
| # | Question | Verdict |
|---|---|---|
| 01 | Does the Rescorla-Wagner implementation match the closed-form acquisition curve? | pass |
| 02 | Does prediction-error learning show Kamin blocking where a correlation rule cannot? | pass |
| 03 | Do two separately trained cues lose strength when reinforced together (overexpectation)? | pass |
| 04 | Does per-channel reliability tracking beat vanilla Kalman when one sensor intermittently lies? | pass |
| 05 | Does cosine-alignment-gated updating form a sharper concept than fixed-rate averaging? | fail |
| 06 | Does align-based routing with spawn-and-prune recover true patterns without spurious concepts? | fail |
| 07 | Does an innate value signal turn a stalling surprise-only agent goal-directed? | partial |
| 08 | Can a controller learn its joint-to-TCP Jacobian online by prediction error and reach arbitrary goals? | fail |
| 09 | Does a hardened online-Jacobian controller (RLS, exploration warm-start, gated dither) reach the robustness to be the outer layer? | open |
The first four rounds set the foundation. The learning rule behaves the way the psychology says it should, including the competition effects that separate it from plain correlation, plus a reliability layer that learns to distrust a sensor that intermittently lies.
Rounds 5 and 6 are where it got honest. The idea that concepts would sharpen by gating updates on alignment looked clean but did not beat the simple baseline, and routing on top of it created concepts that were not really there. Both were written up as failures rather than dropped.
Rounds 8 and 9 move toward robotics. A controller that knows nothing about its own kinematics learns the joint-to-end-effector Jacobian online, from the error between the motion it expected and the motion it got. This is standard adaptive control, validated in this small world. Round 8 only reached about 66 percent, because a straight reach does not excite the estimator enough, which is the known persistent-excitation problem. Round 9, still open, hardens the estimator to see if it can clear the bar.
Why I keep it
It is the most first-principles thing I work on outside my day job: control theory, online estimation, and a strict honesty rule on a problem with no guaranteed answer. The robotics rounds connect directly to adaptive control, and the learning rounds are a regular reminder that a clean baseline beats a clever idea more often than you would like. The next step is to close round 9, then decide whether the prediction-error approach extends past control into perception.