Advanced · Sutton 1992, incremental delta-bar-delta

IDBD Explorer

IDBD is the LMS rule with a separate learning rate for every input, and it adapts each rate while it learns. Inputs whose recent corrections keep pointing the same way get larger steps, and inputs whose corrections cancel out get smaller ones.

GitHub minikin/idbd crates.io idbd

The story's words, in the paper's terms

The story's home has 12 sensors. This page runs the paper's task instead: 20 inputs, of which 5 matter.

sensor
input x0 to x19
reading
sample: one set of inputs and the target
trust
weight w
speed
step size alpha, kept as beta = log alpha
tuning the speeds
meta step size theta
miss
error: target minus prediction, before learning
accuracy
rolling mean squared error (MSE), lower is better
fixed speed (LMS)
LMS with one step size for every input

One update, term by term

What a single IDBD update does

sample 0

A three-input learner on the task y = 2 x0 - x1 + 0 x2. Advance one sample and read the update in the order the algorithm performs it. Every number below is read from the Rust core before and after its own update.

The sample

Inputs, true coefficients and target of the guided sample
Input Value True coefficient

Follow one input

State after this sample

Weight, step size and trace of each input after the guided sample
Input Weight Alpha Trace

The update for x0, in order

    Twenty inputs, one shared stream

    The experiment

    sample 0

    Paused

    Changing any setting above starts a new run from sample 0.

    Run definition: task, seed, settings and versions

    Learned weight per input

    Select an input to inspect it. Arrow keys move between inputs.

    sample 0

    Fills in after the first sample. Choose Step or Run.

    Prediction error over samples

    sample 0

    Fills in after the first sample. Choose Step or Run.

    The chart keeps the most recent 2,000 samples. A run starts all three learners at zero weights, so its first samples include an initial transient. This single run on synthetic data does not show that any one rule is better in general.

    IDBD step sizes, all 20 inputs

    Alpha is the size of the learning adjustment an input receives, not its importance or a confidence.

    sample 0

    Fills in after the first sample. Choose Step or Run.

    World events

    Changes to the target-generating relationship, applied before a sample.

    sample 0

    Fills in after the first sample. Choose Step or Run.

      Input x0

      The true coefficient is known simulator information, not something any learner is told.

      sample 0

      Fills in after the first sample. Choose Step or Run.

      Values

      The same state the charts draw, as numbers rounded for reading. Export run saves every value at full precision.

      sample 0

      Fills in after the first sample. Choose Step or Run.

      Per-input state after the last completed sample
      Input Value True coefficient IDBD weight LMS weight Alpha Trace Autostep weight Autostep alpha