LeanMachineLearning

Bandits.ETC.sumRewards_bestArm_le_of_arm_mul_eq๐Ÿ”—

Lemma

If at time K * m the algorithm chooses arm a, then the total reward obtained by pulling arm a is at least the total reward obtained by pulling the best arm.

๐Ÿ”—theorem
Bandits.ETC.sumRewards_bestArm_le_of_arm_mul_eq.{u_1} {K : โ„•}
  {hK : 0 < K} {m : โ„•} {ฮฝ : ProbabilityTheory.Kernel (Fin K) โ„}
  [ProbabilityTheory.IsMarkovKernel ฮฝ] {ฮฉ : Type u_1}
  {mฮฉ : MeasurableSpace ฮฉ} {P : MeasureTheory.Measure ฮฉ}
  [MeasureTheory.IsProbabilityMeasure P] {A : โ„• โ†’ ฮฉ โ†’ Fin K}
  {R : โ„• โ†’ ฮฉ โ†’ โ„} [Nonempty (Fin K)]
  (h :
    Learning.IsAlgEnvSeq A R (etcAlgorithm hK m)
      (Learning.stationaryEnv ฮฝ) P)
  (a : Fin K) (hm : m โ‰  0) :
  โˆ€แต (h : ฮฉ) โˆ‚P,
    A (K * m) h = a โ†’
      Learning.sumRewards A R (bestArm ฮฝ) (K * m) h โ‰ค
        Learning.sumRewards A R a (K * m) h
Bandits.ETC.sumRewards_bestArm_le_of_arm_mul_eq.{u_1}
  {K : โ„•} {hK : 0 < K} {m : โ„•}
  {ฮฝ : ProbabilityTheory.Kernel (Fin K) โ„}
  [ProbabilityTheory.IsMarkovKernel ฮฝ]
  {ฮฉ : Type u_1} {mฮฉ : MeasurableSpace ฮฉ}
  {P : MeasureTheory.Measure ฮฉ}
  [MeasureTheory.IsProbabilityMeasure P]
  {A : โ„• โ†’ ฮฉ โ†’ Fin K} {R : โ„• โ†’ ฮฉ โ†’ โ„}
  [Nonempty (Fin K)]
  (h :
    Learning.IsAlgEnvSeq A R
      (etcAlgorithm hK m)
      (Learning.stationaryEnv ฮฝ) P)
  (a : Fin K) (hm : m โ‰  0) :
  โˆ€แต (h : ฮฉ) โˆ‚P,
    A (K * m) h = a โ†’
      Learning.sumRewards A R (bestArm ฮฝ)
          (K * m) h โ‰ค
        Learning.sumRewards A R a (K * m)
          h

Code

lemma sumRewards_bestArm_le_of_arm_mul_eq [Nonempty (Fin K)]
    (h : IsAlgEnvSeq A R (etcAlgorithm hK m) (stationaryEnv ฮฝ) P) (a : Fin K) (hm : m โ‰  0) :
    โˆ€แต h โˆ‚P, A (K * m) h = a โ†’ sumRewards A R (bestArm ฮฝ) (K * m) h โ‰ค
      sumRewards A R a (K * m) h
Proof
by
  filter_upwards [arm_mul h hm, pullCount_mul h a, pullCount_mul h (bestArm ฮฝ)]
    with h h_arm ha h_best h_eq
  have h_max := isMaxOn_argmax
    (empMean' (K * m - 1) (history A R (K * m - 1) h)) (bestArm ฮฝ)
  rw [โ† h_arm, h_eq] at h_max
  rw [sumRewards_eq_pullCount_mul_empMean, sumRewards_eq_pullCount_mul_empMean, ha, h_best]
  ยท gcongr
    have : 0 < K * m := Nat.mul_pos hK hm.bot_lt
    rwa [empMean_eq_empMean' this.ne', empMean_eq_empMean' this.ne']
  ยท simp [ha, hm]
  ยท simp [h_best, hm]

Actions: Source ยท Open Issue

Meaning last changed in v4.34.0-rc2-14-gf86702d (2026-08-25), the 5th recorded change.

Self-contained, with its dependencies inlined and proofs replaced by sorry: download the raw file ยท open it in the Lean web editor.

Dependency graph

Audit surface: 18 project declarations, 132 external constants

โœ“ Proved: no sorry anywhere in its closure

This is the tool's own reading of one build's recorded axioms, and it is not robust against an author who wants it to pass. Checking meant to be relied on should go through Comparator, which replays the proof through the kernel from an export against an explicit list of permitted axioms.