Model training log

A project log on training language models under 2B parameters by pushing reasoning into an external executor. Claude Code ran the experiments continuously on James’s RTX 2070 while James set direction, challenged assumptions and pulled the work back to the question.

A small model writes a fact; an executor carries the chain The model reads a turn and writes to its own feed-forward memory. An external executor repeats one-hop lookups when deeper composition is needed. Small model read · write · retrieve 30M / 0.6B Plastic FFN weights become memory Written conclusion available next turn Executor repeat one-hop questions for depth state persists in weights composition stays explicit
The working bet

Learn the state.
Keep the loop outside.

The useful result is not an unlimited reasoning algorithm inside a tiny network. It is a model that can read and write state, paired with an executor that repeats the operations the model does not reliably extend.

Capability, transfer and task framing are measured separately.
A clean number is not trusted until its instrument is checked.
Loading entries…