LeanMachineLearning

2.4. Online.Bandit.BayesRegret🔗

Bayesian regret

This file defines actionMean, bestAction, gap, and regret as random variables in a measurable space Ω. These definitions are useful when IsBayesAlgEnvSeq Q κ alg E A Y P.

Recall that IsBayesAlgEnvSeq Q κ alg E A Y P states that there is a measure P : Measure Ω such that the parameter E : Ω → 𝓔 has law Q and that the sequences of actions A : ℕ → Ω → 𝓐 and feedbacks Y : ℕ → Ω → 𝓨 are generated by the algorithm alg : Algorithm 𝓐 𝓨 interacting with an underlying environment that depends on E and κ (stationaryEnv (κ.sectR (E ω)))

Main definitions

  • actionMean κ E a: the mean feedback associated with action a : 𝓐 based on the parameter E, which defines the underlying stationary environment together with the kernel κ.

  • bestAction κ E: (one of) the action(s) with the highest associated mean feedback based on E.

  • gap κ E A n: the difference between the highest mean feedback associated with an action and the mean feedback associated with the action at time n based on E and the sequence of actions A.

  • regret κ E A n: the regret at time n based on E and the sequence of actions A. If IsBayesAlgEnvSeq Q κ alg E A Y P, then P[regret κ E A n] is the so-called Bayesian regret of algorithm alg under the prior Q.

Module LeanMachineLearning.Online.Bandit.BayesRegret contains 17 exposed declarations.