LeanMachineLearning exposition

Learning.rewardByCount_of_stepsUntil_eq_top๐Ÿ”—

This page has the declaration's own card below, then its dependency graph, then a card for each dependency (type dependencies first, then the rest of the transitive closure). For a theorem, the graph and the dependency cards only follow its statement's dependencies (its proof is replaced by sorry, so what it proves doesn't depend on how); for everything else, both the type and the body/value are followed, since their content is part of what later declarations build on.

Minimal Lean file

rewardByCount_of_stepsUntil_eq_top๐Ÿ”—

LemmaLearning.rewardByCount_of_stepsUntil_eq_top
Details

No docstring.

๐Ÿ”—theorem
Learning.rewardByCount_of_stepsUntil_eq_top.{u_1, u_2, u_3}
  {๐“ : Type u_1} {R : Type u_2} {ฮฉ : Type u_3} [DecidableEq ๐“]
  {A : โ„• โ†’ ฮฉ โ†’ ๐“} {R' : โ„• โ†’ ฮฉ โ†’ R} {a : ๐“} {m : โ„•} {ฯ‰ : ฮฉ ร— (โ„• โ†’ ๐“ โ†’ R)}
  (h : stepsUntil A a m (Prod.fst ฯ‰) = โŠค) :
  rewardByCount A R' a m ฯ‰ = Prod.snd ฯ‰ m a
Learning.rewardByCount_of_stepsUntil_eq_top.{u_1,
    u_2, u_3}
  {๐“ : Type u_1} {R : Type u_2}
  {ฮฉ : Type u_3} [DecidableEq ๐“]
  {A : โ„• โ†’ ฮฉ โ†’ ๐“} {R' : โ„• โ†’ ฮฉ โ†’ R} {a : ๐“}
  {m : โ„•} {ฯ‰ : ฮฉ ร— (โ„• โ†’ ๐“ โ†’ R)}
  (h :
    stepsUntil A a m (Prod.fst ฯ‰) = โŠค) :
  rewardByCount A R' a m ฯ‰ =
    Prod.snd ฯ‰ m a

Code

lemma rewardByCount_of_stepsUntil_eq_top (h : stepsUntil A a m ฯ‰.1 = โŠค) :
    rewardByCount A R' a m ฯ‰ = ฯ‰.2 m a
Type uses (2)
Body uses (1)
Used by (1)

Actions: Source ยท Open Issue

Proof
by simp [rewardByCount_eq_ite, h]

Dependency graph

Type dependencies (2)

stepsUntil๐Ÿ”—

DefinitionLearning.stepsUntil
Details

Number of steps until action a was pulled exactly m times.

๐Ÿ”—def
Learning.stepsUntil.{u_1, u_3} {๐“ : Type u_1} {ฮฉ : Type u_3}
  [DecidableEq ๐“] (A : โ„• โ†’ ฮฉ โ†’ ๐“) (a : ๐“) (m : โ„•) (ฯ‰ : ฮฉ) : โ„•โˆž
Learning.stepsUntil.{u_1, u_3}
  {๐“ : Type u_1} {ฮฉ : Type u_3}
  [DecidableEq ๐“] (A : โ„• โ†’ ฮฉ โ†’ ๐“) (a : ๐“)
  (m : โ„•) (ฯ‰ : ฮฉ) : โ„•โˆž

Code

noncomputable
def stepsUntil (A : โ„• โ†’ ฮฉ โ†’ ๐“) (a : ๐“) (m : โ„•) (ฯ‰ : ฮฉ) : โ„•โˆž :=
  sInf ((โ†‘) '' {s | pullCount A a (s + 1) ฯ‰ = m})
Body uses (1)
Used by (46)

Actions: Source ยท Open Issue

rewardByCount๐Ÿ”—

DefinitionLearning.rewardByCount
Details

Reward obtained when pulling action a for the m-th time. If it is never pulled m times, the reward is given by the second component of ฯ‰, which in applications will be indepedent with same law.

๐Ÿ”—def
Learning.rewardByCount.{u_1, u_2, u_3} {๐“ : Type u_1} {R : Type u_2}
  {ฮฉ : Type u_3} [DecidableEq ๐“] (A : โ„• โ†’ ฮฉ โ†’ ๐“) (R' : โ„• โ†’ ฮฉ โ†’ R)
  (a : ๐“) (m : โ„•) (ฯ‰ : ฮฉ ร— (โ„• โ†’ ๐“ โ†’ R)) : R
Learning.rewardByCount.{u_1, u_2, u_3}
  {๐“ : Type u_1} {R : Type u_2}
  {ฮฉ : Type u_3} [DecidableEq ๐“]
  (A : โ„• โ†’ ฮฉ โ†’ ๐“) (R' : โ„• โ†’ ฮฉ โ†’ R) (a : ๐“)
  (m : โ„•) (ฯ‰ : ฮฉ ร— (โ„• โ†’ ๐“ โ†’ R)) : R

Code

noncomputable
def rewardByCount (A : โ„• โ†’ ฮฉ โ†’ ๐“) (R' : โ„• โ†’ ฮฉ โ†’ R) (a : ๐“) (m : โ„•) (ฯ‰ : ฮฉ ร— (โ„• โ†’ ๐“ โ†’ R)) : R :=
  match (stepsUntil A a m ฯ‰.1) with
  | โŠค => ฯ‰.2 m a
  | (n : โ„•) => R' n ฯ‰.1
Body uses (1)
Used by (15)

Actions: Source ยท Open Issue

All dependencies, transitively (1)

pullCount๐Ÿ”—

DefinitionLearning.pullCount
Details

Number of times action a was chosen up to time t (excluding t).

๐Ÿ”—def
Learning.pullCount.{u_1, u_3} {๐“ : Type u_1} {ฮฉ : Type u_3}
  [DecidableEq ๐“] (A : โ„• โ†’ ฮฉ โ†’ ๐“) (a : ๐“) (t : โ„•) (ฯ‰ : ฮฉ) : โ„•
Learning.pullCount.{u_1, u_3}
  {๐“ : Type u_1} {ฮฉ : Type u_3}
  [DecidableEq ๐“] (A : โ„• โ†’ ฮฉ โ†’ ๐“) (a : ๐“)
  (t : โ„•) (ฯ‰ : ฮฉ) : โ„•

Code

noncomputable
def pullCount (A : โ„• โ†’ ฮฉ โ†’ ๐“) (a : ๐“) (t : โ„•) (ฯ‰ : ฮฉ) : โ„• :=
  #(filter (fun s โ†ฆ A s ฯ‰ = a) (range t))
Used by (146)

Actions: Source ยท Open Issue