LeanMachineLearning exposition

Learning.rewardByCount_zero🔗

This page has the declaration's own card below, then its dependency graph, then a card for each dependency (type dependencies first, then the rest of the transitive closure). For a theorem, the graph and the dependency cards only follow its statement's dependencies (its proof is replaced by sorry, so what it proves doesn't depend on how); for everything else, both the type and the body/value are followed, since their content is part of what later declarations build on.

Minimal Lean file

rewardByCount_zero🔗

LemmaLearning.rewardByCount_zero
Details

The value at 0 does not matter (it would be the "zeroth" reward). It should be considered a junk value.

🔗theorem
Learning.rewardByCount_zero.{u_1, u_2, u_3} {𝓐 : Type u_1}
  {R : Type u_2} {Ω : Type u_3} [DecidableEq 𝓐] {A : ℕ → Ω → 𝓐}
  {R' : ℕ → Ω → R} (a : 𝓐) (ω : Ω × (ℕ → 𝓐 → R)) :
  rewardByCount A R' a 0 ω =
    if A 0 (Prod.fst ω) = a then Prod.snd ω 0 a else R' 0 (Prod.fst ω)
Learning.rewardByCount_zero.{u_1, u_2,
    u_3}
  {𝓐 : Type u_1} {R : Type u_2}
  {Ω : Type u_3} [DecidableEq 𝓐]
  {A : ℕ → Ω → 𝓐} {R' : ℕ → Ω → R} (a : 𝓐)
  (ω : Ω × (ℕ → 𝓐 → R)) :
  rewardByCount A R' a 0 ω =
    if A 0 (Prod.fst ω) = a then
      Prod.snd ω 0 a
    else R' 0 (Prod.fst ω)

Code

lemma rewardByCount_zero (a : 𝓐) (ω : Ω × (ℕ → 𝓐 → R)) :
    rewardByCount A R' a 0 ω = if A 0 ω.1 = a then ω.2 0 a else R' 0 ω.1
Type uses (1)
Body uses (4)

Actions: Source · Open Issue

Proof
by
  rw [rewardByCount_eq_ite]
  by_cases ha : A 0 ω.1 = a
  · simp [ha, stepsUntil_zero_of_eq]
  · simp [stepsUntil_zero_of_ne, ha]

Dependency graph

Type dependencies (1)

rewardByCount🔗

DefinitionLearning.rewardByCount
Details

Reward obtained when pulling action a for the m-th time. If it is never pulled m times, the reward is given by the second component of ω, which in applications will be indepedent with same law.

🔗def
Learning.rewardByCount.{u_1, u_2, u_3} {𝓐 : Type u_1} {R : Type u_2}
  {Ω : Type u_3} [DecidableEq 𝓐] (A : ℕ → Ω → 𝓐) (R' : ℕ → Ω → R)
  (a : 𝓐) (m : ℕ) (ω : Ω × (ℕ → 𝓐 → R)) : R
Learning.rewardByCount.{u_1, u_2, u_3}
  {𝓐 : Type u_1} {R : Type u_2}
  {Ω : Type u_3} [DecidableEq 𝓐]
  (A : ℕ → Ω → 𝓐) (R' : ℕ → Ω → R) (a : 𝓐)
  (m : ℕ) (ω : Ω × (ℕ → 𝓐 → R)) : R

Code

noncomputable
def rewardByCount (A : ℕ → Ω → 𝓐) (R' : ℕ → Ω → R) (a : 𝓐) (m : ℕ) (ω : Ω × (ℕ → 𝓐 → R)) : R :=
  match (stepsUntil A a m ω.1) with
  | ⊤ => ω.2 m a
  | (n : ℕ) => R' n ω.1
Body uses (1)
Used by (15)

Actions: Source · Open Issue

All dependencies, transitively (2)

pullCount🔗

DefinitionLearning.pullCount
Details

Number of times action a was chosen up to time t (excluding t).

🔗def
Learning.pullCount.{u_1, u_3} {𝓐 : Type u_1} {Ω : Type u_3}
  [DecidableEq 𝓐] (A : ℕ → Ω → 𝓐) (a : 𝓐) (t : ℕ) (ω : Ω) : ℕ
Learning.pullCount.{u_1, u_3}
  {𝓐 : Type u_1} {Ω : Type u_3}
  [DecidableEq 𝓐] (A : ℕ → Ω → 𝓐) (a : 𝓐)
  (t : ℕ) (ω : Ω) : ℕ

Code

noncomputable
def pullCount (A : ℕ → Ω → 𝓐) (a : 𝓐) (t : ℕ) (ω : Ω) : ℕ :=
  #(filter (fun s ↦ A s ω = a) (range t))
Used by (146)

Actions: Source · Open Issue

stepsUntil🔗

DefinitionLearning.stepsUntil
Details

Number of steps until action a was pulled exactly m times.

🔗def
Learning.stepsUntil.{u_1, u_3} {𝓐 : Type u_1} {Ω : Type u_3}
  [DecidableEq 𝓐] (A : ℕ → Ω → 𝓐) (a : 𝓐) (m : ℕ) (ω : Ω) : ℕ∞
Learning.stepsUntil.{u_1, u_3}
  {𝓐 : Type u_1} {Ω : Type u_3}
  [DecidableEq 𝓐] (A : ℕ → Ω → 𝓐) (a : 𝓐)
  (m : ℕ) (ω : Ω) : ℕ∞

Code

noncomputable
def stepsUntil (A : ℕ → Ω → 𝓐) (a : 𝓐) (m : ℕ) (ω : Ω) : ℕ∞ :=
  sInf ((↑) '' {s | pullCount A a (s + 1) ω = m})
Body uses (1)
Used by (46)

Actions: Source · Open Issue