LeanMachineLearning exposition

Learning.sum_rewardByCount_eq_sumRewards๐Ÿ”—

This page has the declaration's own card below, then its dependency graph, then a card for each dependency (type dependencies first, then the rest of the transitive closure). For a theorem, the graph and the dependency cards only follow its statement's dependencies (its proof is replaced by sorry, so what it proves doesn't depend on how); for everything else, both the type and the body/value are followed, since their content is part of what later declarations build on.

Minimal Lean file

sum_rewardByCount_eq_sumRewards๐Ÿ”—

LemmaLearning.sum_rewardByCount_eq_sumRewards
Details

No docstring.

๐Ÿ”—theorem
Learning.sum_rewardByCount_eq_sumRewards.{u_1, u_3} {๐“ : Type u_1}
  {ฮฉ : Type u_3} [DecidableEq ๐“] {A : โ„• โ†’ ฮฉ โ†’ ๐“} {R' : โ„• โ†’ ฮฉ โ†’ โ„}
  (a : ๐“) (t : โ„•) (ฯ‰ : ฮฉ ร— (โ„• โ†’ ๐“ โ†’ โ„)) :
  โˆ‘ m โˆˆ Finset.Icc 1 (pullCount A a t (Prod.fst ฯ‰)),
      rewardByCount A R' a m ฯ‰ =
    sumRewards A R' a t (Prod.fst ฯ‰)
Learning.sum_rewardByCount_eq_sumRewards.{u_1,
    u_3}
  {๐“ : Type u_1} {ฮฉ : Type u_3}
  [DecidableEq ๐“] {A : โ„• โ†’ ฮฉ โ†’ ๐“}
  {R' : โ„• โ†’ ฮฉ โ†’ โ„} (a : ๐“) (t : โ„•)
  (ฯ‰ : ฮฉ ร— (โ„• โ†’ ๐“ โ†’ โ„)) :
  โˆ‘
      m โˆˆ
        Finset.Icc 1
          (pullCount A a t (Prod.fst ฯ‰)),
      rewardByCount A R' a m ฯ‰ =
    sumRewards A R' a t (Prod.fst ฯ‰)

Code

lemma sum_rewardByCount_eq_sumRewards {R' : โ„• โ†’ ฮฉ โ†’ โ„} (a : ๐“) (t : โ„•) (ฯ‰ : ฮฉ ร— (โ„• โ†’ ๐“ โ†’ โ„)) :
    โˆ‘ m โˆˆ Icc 1 (pullCount A a t ฯ‰.1), rewardByCount A R' a m ฯ‰ = sumRewards A R' a t ฯ‰.1
Type uses (3)
Body uses (3)

Actions: Source ยท Open Issue

Proof
by
  induction t with
  | zero => simp [pullCount, sumRewards]
  | succ t ht =>
    by_cases hta : A t ฯ‰.1 = a
    ยท rw [โ† hta] at ht โŠข
      rw [pullCount_action_eq_pullCount_add_one, sum_Icc_succ_top (Nat.le_add_left 1 _), ht]
      unfold sumRewards
      rw [sum_range_succ, if_pos rfl, rewardByCount_pullCount_add_one_eq_reward]
    ยท unfold sumRewards
      rwa [pullCount_eq_pullCount_of_action_ne hta, sum_range_succ, if_neg hta, add_zero]

Dependency graph

Type dependencies (3)

pullCount๐Ÿ”—

DefinitionLearning.pullCount
Details

Number of times action a was chosen up to time t (excluding t).

๐Ÿ”—def
Learning.pullCount.{u_1, u_3} {๐“ : Type u_1} {ฮฉ : Type u_3}
  [DecidableEq ๐“] (A : โ„• โ†’ ฮฉ โ†’ ๐“) (a : ๐“) (t : โ„•) (ฯ‰ : ฮฉ) : โ„•
Learning.pullCount.{u_1, u_3}
  {๐“ : Type u_1} {ฮฉ : Type u_3}
  [DecidableEq ๐“] (A : โ„• โ†’ ฮฉ โ†’ ๐“) (a : ๐“)
  (t : โ„•) (ฯ‰ : ฮฉ) : โ„•

Code

noncomputable
def pullCount (A : โ„• โ†’ ฮฉ โ†’ ๐“) (a : ๐“) (t : โ„•) (ฯ‰ : ฮฉ) : โ„• :=
  #(filter (fun s โ†ฆ A s ฯ‰ = a) (range t))
Used by (146)

Actions: Source ยท Open Issue

rewardByCount๐Ÿ”—

DefinitionLearning.rewardByCount
Details

Reward obtained when pulling action a for the m-th time. If it is never pulled m times, the reward is given by the second component of ฯ‰, which in applications will be indepedent with same law.

๐Ÿ”—def
Learning.rewardByCount.{u_1, u_2, u_3} {๐“ : Type u_1} {R : Type u_2}
  {ฮฉ : Type u_3} [DecidableEq ๐“] (A : โ„• โ†’ ฮฉ โ†’ ๐“) (R' : โ„• โ†’ ฮฉ โ†’ R)
  (a : ๐“) (m : โ„•) (ฯ‰ : ฮฉ ร— (โ„• โ†’ ๐“ โ†’ R)) : R
Learning.rewardByCount.{u_1, u_2, u_3}
  {๐“ : Type u_1} {R : Type u_2}
  {ฮฉ : Type u_3} [DecidableEq ๐“]
  (A : โ„• โ†’ ฮฉ โ†’ ๐“) (R' : โ„• โ†’ ฮฉ โ†’ R) (a : ๐“)
  (m : โ„•) (ฯ‰ : ฮฉ ร— (โ„• โ†’ ๐“ โ†’ R)) : R

Code

noncomputable
def rewardByCount (A : โ„• โ†’ ฮฉ โ†’ ๐“) (R' : โ„• โ†’ ฮฉ โ†’ R) (a : ๐“) (m : โ„•) (ฯ‰ : ฮฉ ร— (โ„• โ†’ ๐“ โ†’ R)) : R :=
  match (stepsUntil A a m ฯ‰.1) with
  | โŠค => ฯ‰.2 m a
  | (n : โ„•) => R' n ฯ‰.1
Body uses (1)
Used by (15)

Actions: Source ยท Open Issue

sumRewards๐Ÿ”—

DefinitionLearning.sumRewards
Details

Sum of rewards obtained when pulling action a up to time t (exclusive).

๐Ÿ”—def
Learning.sumRewards.{u_1, u_3} {๐“ : Type u_1} {ฮฉ : Type u_3}
  [DecidableEq ๐“] (A : โ„• โ†’ ฮฉ โ†’ ๐“) (R' : โ„• โ†’ ฮฉ โ†’ โ„) (a : ๐“) (t : โ„•)
  (ฯ‰ : ฮฉ) : โ„
Learning.sumRewards.{u_1, u_3}
  {๐“ : Type u_1} {ฮฉ : Type u_3}
  [DecidableEq ๐“] (A : โ„• โ†’ ฮฉ โ†’ ๐“)
  (R' : โ„• โ†’ ฮฉ โ†’ โ„) (a : ๐“) (t : โ„•)
  (ฯ‰ : ฮฉ) : โ„

Code

def sumRewards (A : โ„• โ†’ ฮฉ โ†’ ๐“) (R' : โ„• โ†’ ฮฉ โ†’ โ„) (a : ๐“) (t : โ„•) (ฯ‰ : ฮฉ) : โ„ :=
  โˆ‘ s โˆˆ range t, if A s ฯ‰ = a then R' s ฯ‰ else 0
Used by (44)

Actions: Source ยท Open Issue

All dependencies, transitively (1)

stepsUntil๐Ÿ”—

DefinitionLearning.stepsUntil
Details

Number of steps until action a was pulled exactly m times.

๐Ÿ”—def
Learning.stepsUntil.{u_1, u_3} {๐“ : Type u_1} {ฮฉ : Type u_3}
  [DecidableEq ๐“] (A : โ„• โ†’ ฮฉ โ†’ ๐“) (a : ๐“) (m : โ„•) (ฯ‰ : ฮฉ) : โ„•โˆž
Learning.stepsUntil.{u_1, u_3}
  {๐“ : Type u_1} {ฮฉ : Type u_3}
  [DecidableEq ๐“] (A : โ„• โ†’ ฮฉ โ†’ ๐“) (a : ๐“)
  (m : โ„•) (ฯ‰ : ฮฉ) : โ„•โˆž

Code

noncomputable
def stepsUntil (A : โ„• โ†’ ฮฉ โ†’ ๐“) (a : ๐“) (m : โ„•) (ฯ‰ : ฮฉ) : โ„•โˆž :=
  sInf ((โ†‘) '' {s | pullCount A a (s + 1) ฯ‰ = m})
Body uses (1)
Used by (46)

Actions: Source ยท Open Issue