LeanMachineLearning exposition

Learning.rewardByCount_pullCount_add_one_eq_reward🔗

This page has the declaration's own card below, then its dependency graph, then a card for each dependency (type dependencies first, then the rest of the transitive closure). For a theorem, the graph and the dependency cards only follow its statement's dependencies (its proof is replaced by sorry, so what it proves doesn't depend on how); for everything else, both the type and the body/value are followed, since their content is part of what later declarations build on.

Minimal Lean file

rewardByCount_pullCount_add_one_eq_reward🔗

LemmaLearning.rewardByCount_pullCount_add_one_eq_reward
Details

No docstring.

🔗theorem
Learning.rewardByCount_pullCount_add_one_eq_reward.{u_1, u_2, u_3}
  {𝓐 : Type u_1} {R : Type u_2} {Ω : Type u_3} [DecidableEq 𝓐]
  {A : ℕ → Ω → 𝓐} {R' : ℕ → Ω → R} (t : ℕ) (ω : Ω × (ℕ → 𝓐 → R)) :
  rewardByCount A R' (A t (Prod.fst ω))
      (pullCount A (A t (Prod.fst ω)) t (Prod.fst ω) + 1) ω =
    R' t (Prod.fst ω)
Learning.rewardByCount_pullCount_add_one_eq_reward.{u_1,
    u_2, u_3}
  {𝓐 : Type u_1} {R : Type u_2}
  {Ω : Type u_3} [DecidableEq 𝓐]
  {A : ℕ → Ω → 𝓐} {R' : ℕ → Ω → R} (t : ℕ)
  (ω : Ω × (ℕ → 𝓐 → R)) :
  rewardByCount A R' (A t (Prod.fst ω))
      (pullCount A (A t (Prod.fst ω)) t
          (Prod.fst ω) +
        1)
      ω =
    R' t (Prod.fst ω)

Code

lemma rewardByCount_pullCount_add_one_eq_reward (t : ℕ) (ω : Ω × (ℕ → 𝓐 → R)) :
    rewardByCount A R' (A t ω.1) (pullCount A (A t ω.1) t ω.1 + 1) ω = R' t ω.1
Type uses (2)
Body uses (3)
Used by (1)

Actions: Source · Open Issue

Proof
by
  rw [rewardByCount, ← pullCount_action_eq_pullCount_add_one, stepsUntil_pullCount_eq]

Dependency graph

Type dependencies (2)

rewardByCount🔗

DefinitionLearning.rewardByCount
Details

Reward obtained when pulling action a for the m-th time. If it is never pulled m times, the reward is given by the second component of ω, which in applications will be indepedent with same law.

🔗def
Learning.rewardByCount.{u_1, u_2, u_3} {𝓐 : Type u_1} {R : Type u_2}
  {Ω : Type u_3} [DecidableEq 𝓐] (A : ℕ → Ω → 𝓐) (R' : ℕ → Ω → R)
  (a : 𝓐) (m : ℕ) (ω : Ω × (ℕ → 𝓐 → R)) : R
Learning.rewardByCount.{u_1, u_2, u_3}
  {𝓐 : Type u_1} {R : Type u_2}
  {Ω : Type u_3} [DecidableEq 𝓐]
  (A : ℕ → Ω → 𝓐) (R' : ℕ → Ω → R) (a : 𝓐)
  (m : ℕ) (ω : Ω × (ℕ → 𝓐 → R)) : R

Code

noncomputable
def rewardByCount (A : ℕ → Ω → 𝓐) (R' : ℕ → Ω → R) (a : 𝓐) (m : ℕ) (ω : Ω × (ℕ → 𝓐 → R)) : R :=
  match (stepsUntil A a m ω.1) with
  | ⊤ => ω.2 m a
  | (n : ℕ) => R' n ω.1
Body uses (1)
Used by (15)

Actions: Source · Open Issue

pullCount🔗

DefinitionLearning.pullCount
Details

Number of times action a was chosen up to time t (excluding t).

🔗def
Learning.pullCount.{u_1, u_3} {𝓐 : Type u_1} {Ω : Type u_3}
  [DecidableEq 𝓐] (A : ℕ → Ω → 𝓐) (a : 𝓐) (t : ℕ) (ω : Ω) : ℕ
Learning.pullCount.{u_1, u_3}
  {𝓐 : Type u_1} {Ω : Type u_3}
  [DecidableEq 𝓐] (A : ℕ → Ω → 𝓐) (a : 𝓐)
  (t : ℕ) (ω : Ω) : ℕ

Code

noncomputable
def pullCount (A : ℕ → Ω → 𝓐) (a : 𝓐) (t : ℕ) (ω : Ω) : ℕ :=
  #(filter (fun s ↦ A s ω = a) (range t))
Used by (146)

Actions: Source · Open Issue

All dependencies, transitively (1)

stepsUntil🔗

DefinitionLearning.stepsUntil
Details

Number of steps until action a was pulled exactly m times.

🔗def
Learning.stepsUntil.{u_1, u_3} {𝓐 : Type u_1} {Ω : Type u_3}
  [DecidableEq 𝓐] (A : ℕ → Ω → 𝓐) (a : 𝓐) (m : ℕ) (ω : Ω) : ℕ∞
Learning.stepsUntil.{u_1, u_3}
  {𝓐 : Type u_1} {Ω : Type u_3}
  [DecidableEq 𝓐] (A : ℕ → Ω → 𝓐) (a : 𝓐)
  (m : ℕ) (ω : Ω) : ℕ∞

Code

noncomputable
def stepsUntil (A : ℕ → Ω → 𝓐) (a : 𝓐) (m : ℕ) (ω : Ω) : ℕ∞ :=
  sInf ((↑) '' {s | pullCount A a (s + 1) ω = m})
Body uses (1)
Used by (46)

Actions: Source · Open Issue