LeanMachineLearning exposition

Learning.rewardByCount_eq_ite🔗

This page has the declaration's own card below, then its dependency graph, then a card for each dependency (type dependencies first, then the rest of the transitive closure). For a theorem, the graph and the dependency cards only follow its statement's dependencies (its proof is replaced by sorry, so what it proves doesn't depend on how); for everything else, both the type and the body/value are followed, since their content is part of what later declarations build on.

Minimal Lean file

rewardByCount_eq_ite🔗

LemmaLearning.rewardByCount_eq_ite
Details

No docstring.

🔗theorem
Learning.rewardByCount_eq_ite.{u_1, u_2, u_3} {𝓐 : Type u_1}
  {R : Type u_2} {Ω : Type u_3} [DecidableEq 𝓐] {A : ℕ → Ω → 𝓐}
  {R' : ℕ → Ω → R} (a : 𝓐) (m : ℕ) (ω : Ω × (ℕ → 𝓐 → R)) :
  rewardByCount A R' a m ω =
    if stepsUntil A a m (Prod.fst ω) = ⊤ then Prod.snd ω m a
    else R' (ENat.toNat (stepsUntil A a m (Prod.fst ω))) (Prod.fst ω)
Learning.rewardByCount_eq_ite.{u_1, u_2,
    u_3}
  {𝓐 : Type u_1} {R : Type u_2}
  {Ω : Type u_3} [DecidableEq 𝓐]
  {A : ℕ → Ω → 𝓐} {R' : ℕ → Ω → R} (a : 𝓐)
  (m : ℕ) (ω : Ω × (ℕ → 𝓐 → R)) :
  rewardByCount A R' a m ω =
    if
        stepsUntil A a m (Prod.fst ω) =
          ⊤ then
      Prod.snd ω m a
    else
      R'
        (ENat.toNat
          (stepsUntil A a m (Prod.fst ω)))
        (Prod.fst ω)

Code

lemma rewardByCount_eq_ite (a : 𝓐) (m : ℕ) (ω : Ω × (ℕ → 𝓐 → R)) :
    rewardByCount A R' a m ω =
      if (stepsUntil A a m ω.1) = ⊤ then ω.2 m a else R' (stepsUntil A a m ω.1).toNat ω.1
Type uses (2)
Used by (6)

Actions: Source · Open Issue

Proof
by
  unfold rewardByCount
  cases stepsUntil A a m ω.1
  · simp; rfl
  · simp

Dependency graph

Type dependencies (2)

rewardByCount🔗

DefinitionLearning.rewardByCount
Details

Reward obtained when pulling action a for the m-th time. If it is never pulled m times, the reward is given by the second component of ω, which in applications will be indepedent with same law.

🔗def
Learning.rewardByCount.{u_1, u_2, u_3} {𝓐 : Type u_1} {R : Type u_2}
  {Ω : Type u_3} [DecidableEq 𝓐] (A : ℕ → Ω → 𝓐) (R' : ℕ → Ω → R)
  (a : 𝓐) (m : ℕ) (ω : Ω × (ℕ → 𝓐 → R)) : R
Learning.rewardByCount.{u_1, u_2, u_3}
  {𝓐 : Type u_1} {R : Type u_2}
  {Ω : Type u_3} [DecidableEq 𝓐]
  (A : ℕ → Ω → 𝓐) (R' : ℕ → Ω → R) (a : 𝓐)
  (m : ℕ) (ω : Ω × (ℕ → 𝓐 → R)) : R

Code

noncomputable
def rewardByCount (A : ℕ → Ω → 𝓐) (R' : ℕ → Ω → R) (a : 𝓐) (m : ℕ) (ω : Ω × (ℕ → 𝓐 → R)) : R :=
  match (stepsUntil A a m ω.1) with
  | ⊤ => ω.2 m a
  | (n : ℕ) => R' n ω.1
Body uses (1)
Used by (15)

Actions: Source · Open Issue

stepsUntil🔗

DefinitionLearning.stepsUntil
Details

Number of steps until action a was pulled exactly m times.

🔗def
Learning.stepsUntil.{u_1, u_3} {𝓐 : Type u_1} {Ω : Type u_3}
  [DecidableEq 𝓐] (A : ℕ → Ω → 𝓐) (a : 𝓐) (m : ℕ) (ω : Ω) : ℕ∞
Learning.stepsUntil.{u_1, u_3}
  {𝓐 : Type u_1} {Ω : Type u_3}
  [DecidableEq 𝓐] (A : ℕ → Ω → 𝓐) (a : 𝓐)
  (m : ℕ) (ω : Ω) : ℕ∞

Code

noncomputable
def stepsUntil (A : ℕ → Ω → 𝓐) (a : 𝓐) (m : ℕ) (ω : Ω) : ℕ∞ :=
  sInf ((↑) '' {s | pullCount A a (s + 1) ω = m})
Body uses (1)
Used by (46)

Actions: Source · Open Issue

All dependencies, transitively (1)

pullCount🔗

DefinitionLearning.pullCount
Details

Number of times action a was chosen up to time t (excluding t).

🔗def
Learning.pullCount.{u_1, u_3} {𝓐 : Type u_1} {Ω : Type u_3}
  [DecidableEq 𝓐] (A : ℕ → Ω → 𝓐) (a : 𝓐) (t : ℕ) (ω : Ω) : ℕ
Learning.pullCount.{u_1, u_3}
  {𝓐 : Type u_1} {Ω : Type u_3}
  [DecidableEq 𝓐] (A : ℕ → Ω → 𝓐) (a : 𝓐)
  (t : ℕ) (ω : Ω) : ℕ

Code

noncomputable
def pullCount (A : ℕ → Ω → 𝓐) (a : 𝓐) (t : ℕ) (ω : Ω) : ℕ :=
  #(filter (fun s ↦ A s ω = a) (range t))
Used by (146)

Actions: Source · Open Issue