LeanMachineLearning exposition

Learning.rewardByCount_eq_add🔗

This page has the declaration's own card below, then its dependency graph, then a card for each dependency (type dependencies first, then the rest of the transitive closure). For a theorem, the graph and the dependency cards only follow its statement's dependencies (its proof is replaced by sorry, so what it proves doesn't depend on how); for everything else, both the type and the body/value are followed, since their content is part of what later declarations build on.

Minimal Lean file

rewardByCount_eq_add🔗

LemmaLearning.rewardByCount_eq_add
Details

No docstring.

🔗theorem
Learning.rewardByCount_eq_add.{u_1, u_2, u_3} {𝓐 : Type u_1}
  {R : Type u_2} {Ω : Type u_3} [DecidableEq 𝓐] {A : ℕ → Ω → 𝓐}
  {R' : ℕ → Ω → R} [AddMonoid R] (a : 𝓐) (m : ℕ) :
  rewardByCount A R' a m =
    (Set.indicator {ω | stepsUntil A a m (Prod.fst ω) ≠ ⊤} fun ω =>
        R' (ENat.toNat (stepsUntil A a m (Prod.fst ω))) (Prod.fst ω)) +
      Set.indicator {ω | stepsUntil A a m (Prod.fst ω) = ⊤} fun ω =>
        Prod.snd ω m a
Learning.rewardByCount_eq_add.{u_1, u_2,
    u_3}
  {𝓐 : Type u_1} {R : Type u_2}
  {Ω : Type u_3} [DecidableEq 𝓐]
  {A : ℕ → Ω → 𝓐} {R' : ℕ → Ω → R}
  [AddMonoid R] (a : 𝓐) (m : ℕ) :
  rewardByCount A R' a m =
    (Set.indicator
        {ω |
          stepsUntil A a m (Prod.fst ω) ≠
            ⊤}
        fun ω =>
        R'
          (ENat.toNat
            (stepsUntil A a m
              (Prod.fst ω)))
          (Prod.fst ω)) +
      Set.indicator
        {ω |
          stepsUntil A a m (Prod.fst ω) =
            ⊤}
        fun ω => Prod.snd ω m a

Code

lemma rewardByCount_eq_add [AddMonoid R] (a : 𝓐) (m : ℕ) :
    rewardByCount A R' a m =
      {ω : Ω × (ℕ → 𝓐 → R) | stepsUntil A a m ω.1 ≠ ⊤}.indicator
          (fun ω ↦ R' (stepsUntil A a m ω.1).toNat ω.1)
        + {ω | stepsUntil A a m ω.1 = ⊤}.indicator (fun ω ↦ ω.2 m a)
Type uses (2)
Body uses (1)

Actions: Source · Open Issue

Proof
by
  ext ω
  simp only [rewardByCount_eq_ite, ne_eq, Pi.add_apply, Set.indicator_apply, Set.mem_setOf_eq,
    ite_not]
  grind

Dependency graph

Type dependencies (2)

rewardByCount🔗

DefinitionLearning.rewardByCount
Details

Reward obtained when pulling action a for the m-th time. If it is never pulled m times, the reward is given by the second component of ω, which in applications will be indepedent with same law.

🔗def
Learning.rewardByCount.{u_1, u_2, u_3} {𝓐 : Type u_1} {R : Type u_2}
  {Ω : Type u_3} [DecidableEq 𝓐] (A : ℕ → Ω → 𝓐) (R' : ℕ → Ω → R)
  (a : 𝓐) (m : ℕ) (ω : Ω × (ℕ → 𝓐 → R)) : R
Learning.rewardByCount.{u_1, u_2, u_3}
  {𝓐 : Type u_1} {R : Type u_2}
  {Ω : Type u_3} [DecidableEq 𝓐]
  (A : ℕ → Ω → 𝓐) (R' : ℕ → Ω → R) (a : 𝓐)
  (m : ℕ) (ω : Ω × (ℕ → 𝓐 → R)) : R

Code

noncomputable
def rewardByCount (A : ℕ → Ω → 𝓐) (R' : ℕ → Ω → R) (a : 𝓐) (m : ℕ) (ω : Ω × (ℕ → 𝓐 → R)) : R :=
  match (stepsUntil A a m ω.1) with
  | ⊤ => ω.2 m a
  | (n : ℕ) => R' n ω.1
Body uses (1)
Used by (15)

Actions: Source · Open Issue

stepsUntil🔗

DefinitionLearning.stepsUntil
Details

Number of steps until action a was pulled exactly m times.

🔗def
Learning.stepsUntil.{u_1, u_3} {𝓐 : Type u_1} {Ω : Type u_3}
  [DecidableEq 𝓐] (A : ℕ → Ω → 𝓐) (a : 𝓐) (m : ℕ) (ω : Ω) : ℕ∞
Learning.stepsUntil.{u_1, u_3}
  {𝓐 : Type u_1} {Ω : Type u_3}
  [DecidableEq 𝓐] (A : ℕ → Ω → 𝓐) (a : 𝓐)
  (m : ℕ) (ω : Ω) : ℕ∞

Code

noncomputable
def stepsUntil (A : ℕ → Ω → 𝓐) (a : 𝓐) (m : ℕ) (ω : Ω) : ℕ∞ :=
  sInf ((↑) '' {s | pullCount A a (s + 1) ω = m})
Body uses (1)
Used by (46)

Actions: Source · Open Issue

All dependencies, transitively (1)

pullCount🔗

DefinitionLearning.pullCount
Details

Number of times action a was chosen up to time t (excluding t).

🔗def
Learning.pullCount.{u_1, u_3} {𝓐 : Type u_1} {Ω : Type u_3}
  [DecidableEq 𝓐] (A : ℕ → Ω → 𝓐) (a : 𝓐) (t : ℕ) (ω : Ω) : ℕ
Learning.pullCount.{u_1, u_3}
  {𝓐 : Type u_1} {Ω : Type u_3}
  [DecidableEq 𝓐] (A : ℕ → Ω → 𝓐) (a : 𝓐)
  (t : ℕ) (ω : Ω) : ℕ

Code

noncomputable
def pullCount (A : ℕ → Ω → 𝓐) (a : 𝓐) (t : ℕ) (ω : Ω) : ℕ :=
  #(filter (fun s ↦ A s ω = a) (range t))
Used by (146)

Actions: Source · Open Issue