`Bandits.ArrayModel.hasCondDistrib_reward_pullCount_action`🔗

Lemma

The conditional distribution of the reward at time n + 1, given the action at time n + 1 and the number of times that action has been pulled before time n + 1, is equal to the kernel ν.

theorem

Bandits.ArrayModel.hasCondDistrib_reward_pullCount_action.{u_1, u_2}
  {𝓐 : Type u_1} {R : Type u_2} {m𝓐 : MeasurableSpace 𝓐}
  {mR : MeasurableSpace R} [Nonempty 𝓐] [StandardBorelSpace 𝓐]
  [DecidableEq 𝓐] [Countable 𝓐] [StandardBorelSpace R] [Nonempty R]
  (alg : Learning.Algorithm 𝓐 R) (ν : ProbabilityTheory.Kernel 𝓐 R)
  [ProbabilityTheory.IsMarkovKernel ν] (n : ℕ) :
  ProbabilityTheory.HasCondDistrib (reward alg (n + 1))
    (fun ω =>
      (action alg (n + 1) ω,
        Learning.pullCount (action alg) (action alg (n + 1) ω) (n + 1)
          ω))
    (ProbabilityTheory.Kernel.prodMkRight ℕ ν) (arrayMeasure ν)
Bandits.ArrayModel.hasCondDistrib_reward_pullCount_action.{u_1,
    u_2}
  {𝓐 : Type u_1} {R : Type u_2}
  {m𝓐 : MeasurableSpace 𝓐}
  {mR : MeasurableSpace R} [Nonempty 𝓐]
  [StandardBorelSpace 𝓐] [DecidableEq 𝓐]
  [Countable 𝓐] [StandardBorelSpace R]
  [Nonempty R]
  (alg : Learning.Algorithm 𝓐 R)
  (ν : ProbabilityTheory.Kernel 𝓐 R)
  [ProbabilityTheory.IsMarkovKernel ν]
  (n : ℕ) :
  ProbabilityTheory.HasCondDistrib
    (reward alg (n + 1))
    (fun ω =>
      (action alg (n + 1) ω,
        Learning.pullCount (action alg)
          (action alg (n + 1) ω) (n + 1)
          ω))
    (ProbabilityTheory.Kernel.prodMkRight
      ℕ ν)
    (arrayMeasure ν)

Code

lemma hasCondDistrib_reward_pullCount_action
    (alg : Algorithm 𝓐 R) (ν : Kernel 𝓐 R) [IsMarkovKernel ν] (n : ℕ) :
    HasCondDistrib (reward alg (n + 1))
      (fun ω ↦ (action alg (n + 1) ω, pullCount (action alg) (action alg (n + 1) ω) (n + 1) ω))
      (ν.prodMkRight _) (arrayMeasure ν)

Proof

by
  have h_meas : Measurable fun ω ↦ pullCount (action alg) (action alg (n + 1) ω) (n + 1) ω := by
    change Measurable ((fun p : (probSpace 𝓐 R) × 𝓐 ↦ pullCount (action alg) p.2 (n + 1) p.1) ∘
      (fun ω : probSpace 𝓐 R ↦ (ω, action alg (n + 1) ω)))
    exact (measurable_uncurry_pullCount (by fun_prop) _).comp (by fun_prop)
  refine hasCondDistrib_of_condDistrib_eq (by fun_prop) (by fun_prop) ?_
  refine (condDistrib_ae_eq_cond
    (Measurable.prodMk (by fun_prop) (by fun_prop)) (by fun_prop)).trans ?_
  rw [Filter.EventuallyEq, ae_iff_of_countable]
  intro ⟨a, m⟩ ham
  simp only [Kernel.prodMkRight_apply]
  calc
    Measure.map (reward alg (n + 1))
      (arrayMeasure ν)[|(fun ω ↦ (action alg (n + 1) ω,
        pullCount (action alg) (action alg (n + 1) ω) (n + 1) ω)) ⁻¹' {(a, m)}]
  _ = Measure.map (fun ω ↦ ω.2 m a)
      (arrayMeasure ν)[|(fun ω ↦ (action alg (n + 1) ω,
        pullCount (action alg) (action alg (n + 1) ω) (n + 1) ω)) ⁻¹' {(a, m)}] := by
    rw [reward_eq]
    refine Measure.map_congr
      (ae_cond_of_forall_mem ((measurableSet_singleton _).preimage (by fun_prop)) (fun x hx ↦ ?_))
    simp only [Set.mem_preimage, Set.mem_singleton_iff, Prod.mk.injEq] at hx
    simp only [hx.1] at hx ⊢
    simp [hx.2]
  _ = Measure.map (fun ω ↦ ω.2 m a)
      (arrayMeasure ν)[|({ω | action alg (n + 1) ω = a ∧
        pullCount (action alg) a (n + 1) ω = m}).indicator 1 ⁻¹' {1}] := by
    congr with ω
    simp only [Set.mem_preimage, Set.mem_singleton_iff, Prod.mk.injEq, Set.indicator_apply,
      Set.mem_ofPred_eq, Pi.one_apply, ite_eq_left_iff, not_and, zero_ne_one, imp_false,
      Classical.not_imp, Decidable.not_not, and_congr_right_iff]
    intro ha
    simp [ha]
  _ = ν a := by
    rw [cond_of_indepFun, map_snd_apply_arrayMeasure m a]
    · exact (indepFun_snd_apply_pullCount_action alg ν a m n).symm
    · refine Measurable.indicator (by fun_prop) ?_
      exact MeasurableSet.inter ((measurableSet_singleton _).preimage (by fun_prop))
        ((measurableSet_singleton _).preimage (by fun_prop))
    · fun_prop
    · simp
    · rw [Measure.map_apply (by fun_prop) (by simp)] at ham
      convert ham
      ext ω
      simp only [Set.mem_preimage, Set.indicator_apply, Set.mem_ofPred_eq, Pi.one_apply,
        Set.mem_singleton_iff, ite_eq_left_iff, not_and, zero_ne_one, imp_false, Classical.not_imp,
        Decidable.not_not, Prod.mk.injEq, and_congr_right_iff]
      intro ha
      simp [ha]

Actions: Source · Open Issue

Meaning last changed in v4.34.0-rc2-1-g439785b (2026-08-23), the 5th recorded change.

Self-contained, with its dependencies inlined and proofs replaced by sorry: download the raw file · open it in the Lean web editor.

Dependency graph

Audit surface: 14 project declarations, 94 external constants

✓ Proved: no sorry anywhere in its closure

This is the tool's own reading of one build's recorded axioms, and it is not robust against an author who wants it to pass. Checking meant to be relied on should go through Comparator, which replays the proof through the kernel from an export against an explicit list of permitted axioms.

Bandits.ArrayModel.hasCondDistrib_reward_pullCount_action🔗

`Bandits.ArrayModel.hasCondDistrib_reward_pullCount_action`🔗