mod contextual

module contextual

The cooling-schedule trait: epoch -> temperature. Choosing a move from where the chain is standing, not from a global rate.

The allocator in crate::allocate keeps one success rate per move and samples from it. That is the right model when a move has a rate; it is the wrong model when a move has a precondition. Wales and Doye’s angular move is the clear case: it is not applied at some frequency, it is applied when the worst-bound point crosses a pair-energy criterion. A context-free allocator learns the average of the times it was and was not appropriate, and that average describes no situation the chain is ever in.

The model here is hierarchical. Each move’s value is linear in a context vector, with a component shared across moves and a per-move deviation:

value(context c, move a) = c . (w_shared + w_a)

Sharing matters because the arms do not get equal data. A move the allocator has learned to avoid is sampled rarely, so its own coefficients stay uncertain forever; the shared component is fitted on every observation and carries what is true of the landscape rather than of one move. In factor terms this is a rank-one-plus-residual decomposition of the context-by-move value matrix, which is what makes it estimable from far less data than the full matrix.

Selection is Thompson sampling: draw coefficients from each move’s posterior, score the current context, take the best. A floor keeps every move sampled at some rate whatever the posterior says, for the same reason the screen in crate::screen keeps an exploration floor: a rule that only ever picks what it already believes never learns it was wrong.

Structs and Unions

struct ContextualAllocator

A contextual allocator over a fixed set of moves.

floor: f64

Rate at which a move is chosen uniformly, whatever the posterior says.

exploration: f64

Scale on the posterior standard deviation when sampling.

One is Thompson sampling proper. Larger explores more, smaller behaves more greedily; exposed because the reward here is bounded in [0, 1] and a posterior fitted on a bounded reward is over-confident at its edges.

picks: Vec<usize>

Times each move was chosen.

wins: Vec<usize>

Times each move was rewarded.

forced: usize

Choices made uniformly by the floor.

Implementations

impl ContextualAllocator

Functions

fn is_empty(&self) -> bool

Whether there are no moves, which the constructor forbids.

fn len(&self) -> usize

Moves this allocator chooses between.

fn new(n_moves: usize, dim: usize, floor: f64) -> Self

Allocator over n_moves with a dim-dimensional context.

The context must include an intercept if the caller wants one; nothing here adds it.

fn select<R: Rng + ?Sized>(&mut self, context: ArrayView1<f64>, rng: &mut R) -> usize

Picks a move for context by Thompson sampling.

fn update(&mut self, move_index: usize, context: ArrayView1<f64>, reward: f64)

Records what happened, with reward in [0, 1].

fn value(&self, move_index: usize, context: ArrayView1<f64>) -> f64

Posterior mean value of move_index in context.