LeanMachineLearning exposition

Learning.IsAlgEnvSeq.isPredictable_empMean🔗

This page has the declaration's own card below, then its dependency graph, then a card for each dependency (type dependencies first, then the rest of the transitive closure). For a theorem, the graph and the dependency cards only follow its statement's dependencies (its proof is replaced by sorry, so what it proves doesn't depend on how); for everything else, both the type and the body/value are followed, since their content is part of what later declarations build on.

Minimal Lean file

isPredictable_empMean🔗

LemmaLearning.IsAlgEnvSeq.isPredictable_empMean
Details

No docstring.

🔗theorem
Learning.IsAlgEnvSeq.isPredictable_empMean.{u_1, u_3} {𝓐 : Type u_1}
  {Ω : Type u_3} {m𝓐 : MeasurableSpace 𝓐} {mΩ : MeasurableSpace Ω}
  [DecidableEq 𝓐] {P : MeasureTheory.Measure Ω}
  [MeasureTheory.IsProbabilityMeasure P] {A : ℕ → Ω → 𝓐}
  [StandardBorelSpace 𝓐] {R' : ℕ → Ω → ℝ} {alg : Algorithm 𝓐 ℝ}
  {env : Environment 𝓐 ℝ} (h : IsAlgEnvSeq A R' alg env P) (a : 𝓐) :
  MeasureTheory.IsStronglyPredictable (filtration ⋯ ⋯) (empMean A R' a)
Learning.IsAlgEnvSeq.isPredictable_empMean.{u_1,
    u_3}
  {𝓐 : Type u_1} {Ω : Type u_3}
  {m𝓐 : MeasurableSpace 𝓐}
  {mΩ : MeasurableSpace Ω} [DecidableEq 𝓐]
  {P : MeasureTheory.Measure Ω}
  [MeasureTheory.IsProbabilityMeasure P]
  {A : ℕ → Ω → 𝓐} [StandardBorelSpace 𝓐]
  {R' : ℕ → Ω → ℝ} {alg : Algorithm 𝓐 ℝ}
  {env : Environment 𝓐 ℝ}
  (h : IsAlgEnvSeq A R' alg env P)
  (a : 𝓐) :
  MeasureTheory.IsStronglyPredictable
    (filtration ⋯ ⋯) (empMean A R' a)

Code

lemma IsAlgEnvSeq.isPredictable_empMean [StandardBorelSpace 𝓐] {R' : ℕ → Ω → ℝ}
    {alg : Algorithm 𝓐 ℝ} {env : Environment 𝓐 ℝ}
    (h : IsAlgEnvSeq A R' alg env P) (a : 𝓐) :
    IsStronglyPredictable (IsAlgEnvSeq.filtration h.measurable_action h.measurable_feedback)
      (empMean A R' a)
Type uses (5)
Body uses (5)
Used by (1)

Actions: Source · Open Issue

Proof
by
  unfold empMean
  refine StronglyMeasurable.div₀' ?_ ?_
  · exact h.isPredictable_sumRewards a
  · have h_meas := (isPredictable_pullCount h.measurable_action h.measurable_feedback a).measurable
    fun_prop

Dependency graph

Type dependencies (5)

Algorithm🔗

StructureLearning.Algorithm
Details

A stochastic, sequential algorithm.

🔗structure
Learning.Algorithm.{u_4, u_5} (𝓐 : Type u_4) (𝓨 : Type u_5)
  [MeasurableSpace 𝓐] [MeasurableSpace 𝓨] : Type (max u_4 u_5)
Learning.Algorithm.{u_4, u_5}
  (𝓐 : Type u_4) (𝓨 : Type u_5)
  [MeasurableSpace 𝓐]
  [MeasurableSpace 𝓨] : Type (max u_4 u_5)

Code

structure Algorithm (𝓐 𝓨 : Type*) [MeasurableSpace 𝓐] [MeasurableSpace 𝓨] where
  /-- Policy or sampling rule: distribution of the next action. -/
  policy : (n : ℕ) → Kernel (Iic n → 𝓐 × 𝓨) 𝓐
  /-- The policy is a Markov kernel. -/
  [h_policy : ∀ n, IsMarkovKernel (policy n)]
  /-- Distribution of the first action. -/
  p0 : Measure 𝓐
  /-- The first action distribution is a probability measure. -/
  [hp0 : IsProbabilityMeasure p0]
Used by (216)

Actions: Source · Open Issue

Environment🔗

StructureLearning.Environment
Details

A stochastic environment.

🔗structure
Learning.Environment.{u_4, u_5} (𝓐 : Type u_4) (𝓨 : Type u_5)
  [MeasurableSpace 𝓐] [MeasurableSpace 𝓨] : Type (max u_4 u_5)
Learning.Environment.{u_4, u_5}
  (𝓐 : Type u_4) (𝓨 : Type u_5)
  [MeasurableSpace 𝓐]
  [MeasurableSpace 𝓨] : Type (max u_4 u_5)

Code

structure Environment (𝓐 𝓨 : Type*) [MeasurableSpace 𝓐] [MeasurableSpace 𝓨] where
  /-- Distribution of the next observation as function of the past history. -/
  feedback : (n : ℕ) → Kernel ((Iic n → 𝓐 × 𝓨) × 𝓐) 𝓨
  /-- The feedback kernels are Markov kernels. -/
  [h_feedback : ∀ n, IsMarkovKernel (feedback n)]
  /-- Distribution of the first observation given the first action. -/
  ν0 : Kernel 𝓐 𝓨
  /-- The initial observation kernel is a Markov kernel. -/
  [hp0 : IsMarkovKernel ν0]
Used by (128)

Actions: Source · Open Issue

IsAlgEnvSeq🔗

StructureLearning.IsAlgEnvSeq
Details

An algorithm-environment sequence: a sequence of actions and feedbacks generated by an algorithm interacting with an environment.

🔗structure
Learning.IsAlgEnvSeq.{u_1, u_2, u_3} {𝓐 : Type u_1} {𝓨 : Type u_2}
  {Ω : Type u_3} {m𝓐 : MeasurableSpace 𝓐} {m𝓨 : MeasurableSpace 𝓨}
  {mΩ : MeasurableSpace Ω} (A : ℕ → Ω → 𝓐) (Y : ℕ → Ω → 𝓨)
  (alg : Algorithm 𝓐 𝓨) (env : Environment 𝓐 𝓨)
  (P : MeasureTheory.Measure Ω) [MeasureTheory.IsFiniteMeasure P] : Prop
Learning.IsAlgEnvSeq.{u_1, u_2, u_3}
  {𝓐 : Type u_1} {𝓨 : Type u_2}
  {Ω : Type u_3} {m𝓐 : MeasurableSpace 𝓐}
  {m𝓨 : MeasurableSpace 𝓨}
  {mΩ : MeasurableSpace Ω} (A : ℕ → Ω → 𝓐)
  (Y : ℕ → Ω → 𝓨) (alg : Algorithm 𝓐 𝓨)
  (env : Environment 𝓐 𝓨)
  (P : MeasureTheory.Measure Ω)
  [MeasureTheory.IsFiniteMeasure P] : Prop

Code

structure IsAlgEnvSeq
    (A : ℕ → Ω → 𝓐) (Y : ℕ → Ω → 𝓨) (alg : Algorithm 𝓐 𝓨) (env : Environment 𝓐 𝓨)
    (P : Measure Ω) [IsFiniteMeasure P] : Prop where
  /-- The action sequence is measurable. -/
  measurable_action n : Measurable (A n) := by fun_prop
  /-- The feedback sequence is measurable. -/
  measurable_feedback n : Measurable (Y n) := by fun_prop
  /-- The first action has the correct law. -/
  hasLaw_action_zero : HasLaw (fun ω ↦ (A 0 ω)) alg.p0 P
  /-- The first feedback has the correct conditional distribution. -/
  hasCondDistrib_feedback_zero : HasCondDistrib (Y 0) (A 0) env.ν0 P
  /-- The next action has the correct conditional distribution given the history. -/
  hasCondDistrib_action n :
    HasCondDistrib (A (n + 1)) (history A Y n) (alg.policy n) P
  /-- The next feedback has the correct conditional distribution given the history and
  next action. -/
  hasCondDistrib_feedback n :
    HasCondDistrib (Y (n + 1)) (fun ω ↦ (history A Y n ω, A (n + 1) ω))
      (env.feedback n) P
Type uses (3)
Used by (111)

Actions: Source · Open Issue

filtration🔗

DefinitionLearning.IsAlgEnvSeq.filtration
Details

Filtration generated by the history up to time n.

🔗def
Learning.IsAlgEnvSeq.filtration.{u_1, u_2, u_3} {𝓐 : Type u_1}
  {𝓨 : Type u_2} {Ω : Type u_3} {m𝓐 : MeasurableSpace 𝓐}
  {m𝓨 : MeasurableSpace 𝓨} {mΩ : MeasurableSpace Ω} {A : ℕ → Ω → 𝓐}
  {Y : ℕ → Ω → 𝓨} (hA : ∀ (n : ℕ), Measurable (A n))
  (hY : ∀ (n : ℕ), Measurable (Y n)) : MeasureTheory.Filtration ℕ mΩ
Learning.IsAlgEnvSeq.filtration.{u_1, u_2,
    u_3}
  {𝓐 : Type u_1} {𝓨 : Type u_2}
  {Ω : Type u_3} {m𝓐 : MeasurableSpace 𝓐}
  {m𝓨 : MeasurableSpace 𝓨}
  {mΩ : MeasurableSpace Ω} {A : ℕ → Ω → 𝓐}
  {Y : ℕ → Ω → 𝓨}
  (hA : ∀ (n : ℕ), Measurable (A n))
  (hY : ∀ (n : ℕ), Measurable (Y n)) :
  MeasureTheory.Filtration ℕ mΩ

Code

def IsAlgEnvSeq.filtration (hA : ∀ n, Measurable (A n)) (hY : ∀ n, Measurable (Y n)) :
    Filtration ℕ mΩ where
  seq i := MeasurableSpace.comap (history A Y i) inferInstance
  mono' i j hij := by
    simp only
    rw [← measurable_iff_comap_le]
    have : history A Y i = (fun h k ↦ h ⟨k.1, by grind⟩) ∘ history A Y j := rfl
    rw [this]
    exact measurable_comp_comap _ (by fun_prop)
  le' i := by
    rw [← measurable_iff_comap_le]
    exact Learning.measurable_history hA hY i
Body uses (3)
Used by (18)

Actions: Source · Open Issue

empMean🔗

DefinitionLearning.empMean
Details

Empirical mean reward obtained when pulling action a up to time t (exclusive).

🔗def
Learning.empMean.{u_1, u_3} {𝓐 : Type u_1} {Ω : Type u_3}
  [DecidableEq 𝓐] (A : ℕ → Ω → 𝓐) (R' : ℕ → Ω → ℝ) (a : 𝓐) (t : ℕ)
  (ω : Ω) : ℝ
Learning.empMean.{u_1, u_3} {𝓐 : Type u_1}
  {Ω : Type u_3} [DecidableEq 𝓐]
  (A : ℕ → Ω → 𝓐) (R' : ℕ → Ω → ℝ) (a : 𝓐)
  (t : ℕ) (ω : Ω) : ℝ

Code

noncomputable
def empMean (A : ℕ → Ω → 𝓐) (R' : ℕ → Ω → ℝ) (a : 𝓐) (t : ℕ) (ω : Ω) : ℝ :=
  sumRewards A R' a t ω / pullCount A a t ω
Body uses (2)
Used by (34)

Actions: Source · Open Issue

All dependencies, transitively (5)

history🔗

DefinitionLearning.history
Details

History of the algorithm-environment sequence up to time n.

🔗def
Learning.history.{u_1, u_2, u_3} {𝓐 : Type u_1} {𝓨 : Type u_2}
  {Ω : Type u_3} (A : ℕ → Ω → 𝓐) (Y : ℕ → Ω → 𝓨) (n : ℕ) (ω : Ω) :
  ↥(Finset.Iic n) → 𝓐 × 𝓨
Learning.history.{u_1, u_2, u_3}
  {𝓐 : Type u_1} {𝓨 : Type u_2}
  {Ω : Type u_3} (A : ℕ → Ω → 𝓐)
  (Y : ℕ → Ω → 𝓨) (n : ℕ) (ω : Ω) :
  ↥(Finset.Iic n) → 𝓐 × 𝓨

Code

def history (A : ℕ → Ω → 𝓐) (Y : ℕ → Ω → 𝓨) (n : ℕ) (ω : Ω) : Iic n → 𝓐 × 𝓨 :=
  fun i ↦ (A i ω, Y i ω)
Used by (72)

Actions: Source · Open Issue

measurable_comp_comap🔗

LemmaMeasureTheory.measurable_comp_comap
Details

No docstring.

🔗theorem
MeasureTheory.measurable_comp_comap.{u_1, u_2, u_3} {α : Type u_1}
  {β : Type u_2} {γ : Type u_3} {mβ : MeasurableSpace β}
  {mγ : MeasurableSpace γ} (f : α → β) {g : β → γ} (hg : Measurable g) :
  Measurable (g ∘ f)
MeasureTheory.measurable_comp_comap.{u_1,
    u_2, u_3}
  {α : Type u_1} {β : Type u_2}
  {γ : Type u_3} {mβ : MeasurableSpace β}
  {mγ : MeasurableSpace γ} (f : α → β)
  {g : β → γ} (hg : Measurable g) :
  Measurable (g ∘ f)

Code

lemma measurable_comp_comap (f : α → β) {g : β → γ} (hg : Measurable g) :
    Measurable[mβ.comap f] (g ∘ f)
Used by (10)

Actions: Source · Open Issue

Proof
by
  rw [measurable_iff_comap_le, ← MeasurableSpace.comap_comp]
  exact MeasurableSpace.comap_mono hg.comap_le

measurable_history🔗

LemmaLearning.measurable_history
Details

No docstring.

🔗theorem
Learning.measurable_history.{u_1, u_2, u_3} {𝓐 : Type u_1}
  {𝓨 : Type u_2} {Ω : Type u_3} {m𝓐 : MeasurableSpace 𝓐}
  {m𝓨 : MeasurableSpace 𝓨} {mΩ : MeasurableSpace Ω} {A : ℕ → Ω → 𝓐}
  {Y : ℕ → Ω → 𝓨} (hA : ∀ (n : ℕ), Measurable (A n))
  (hY : ∀ (n : ℕ), Measurable (Y n)) (n : ℕ) :
  Measurable (history A Y n)
Learning.measurable_history.{u_1, u_2,
    u_3}
  {𝓐 : Type u_1} {𝓨 : Type u_2}
  {Ω : Type u_3} {m𝓐 : MeasurableSpace 𝓐}
  {m𝓨 : MeasurableSpace 𝓨}
  {mΩ : MeasurableSpace Ω} {A : ℕ → Ω → 𝓐}
  {Y : ℕ → Ω → 𝓨}
  (hA : ∀ (n : ℕ), Measurable (A n))
  (hY : ∀ (n : ℕ), Measurable (Y n))
  (n : ℕ) : Measurable (history A Y n)

Code

lemma measurable_history (hA : ∀ n, Measurable (A n))
    (hY : ∀ n, Measurable (Y n)) (n : ℕ) :
    Measurable (history A Y n)
Type uses (1)
Used by (10)

Actions: Source · Open Issue

Proof
by
  unfold history
  fun_prop

sumRewards🔗

DefinitionLearning.sumRewards
Details

Sum of rewards obtained when pulling action a up to time t (exclusive).

🔗def
Learning.sumRewards.{u_1, u_3} {𝓐 : Type u_1} {Ω : Type u_3}
  [DecidableEq 𝓐] (A : ℕ → Ω → 𝓐) (R' : ℕ → Ω → ℝ) (a : 𝓐) (t : ℕ)
  (ω : Ω) : ℝ
Learning.sumRewards.{u_1, u_3}
  {𝓐 : Type u_1} {Ω : Type u_3}
  [DecidableEq 𝓐] (A : ℕ → Ω → 𝓐)
  (R' : ℕ → Ω → ℝ) (a : 𝓐) (t : ℕ)
  (ω : Ω) : ℝ

Code

def sumRewards (A : ℕ → Ω → 𝓐) (R' : ℕ → Ω → ℝ) (a : 𝓐) (t : ℕ) (ω : Ω) : ℝ :=
  ∑ s ∈ range t, if A s ω = a then R' s ω else 0
Used by (44)

Actions: Source · Open Issue

pullCount🔗

DefinitionLearning.pullCount
Details

Number of times action a was chosen up to time t (excluding t).

🔗def
Learning.pullCount.{u_1, u_3} {𝓐 : Type u_1} {Ω : Type u_3}
  [DecidableEq 𝓐] (A : ℕ → Ω → 𝓐) (a : 𝓐) (t : ℕ) (ω : Ω) : ℕ
Learning.pullCount.{u_1, u_3}
  {𝓐 : Type u_1} {Ω : Type u_3}
  [DecidableEq 𝓐] (A : ℕ → Ω → 𝓐) (a : 𝓐)
  (t : ℕ) (ω : Ω) : ℕ

Code

noncomputable
def pullCount (A : ℕ → Ω → 𝓐) (a : 𝓐) (t : ℕ) (ω : Ω) : ℕ :=
  #(filter (fun s ↦ A s ω = a) (range t))
Used by (146)

Actions: Source · Open Issue