Learning.IsAlgEnvSeq.klDiv_map_trajectory_stepKernel
Lemma
From the authors
Chain rule for trajectories. For two algorithms alg and alg' run against
two environments env and env', the divergence between the laws of the trajectories is
the series over the rounds t of the conditional divergences of the step at round t given
the first t rounds.
Types
-
๐ : Type u_1m๐ : MeasurableSpace ๐A measurable space is a space equipped with a ฯ-algebra. -
๐ : Type u_2m๐ : MeasurableSpace ๐ -
๐จ : Type u_3m๐จ : MeasurableSpace ๐จ -
ฮฉ : Type u_4mฮฉ : MeasurableSpace ฮฉ -
ฮฉ' : Type u_5mฮฉ' : MeasurableSpace ฮฉ'
Given
-
P : MeasureTheory.Measure ฮฉA measure is defined to be an outer measure that is countably additive on measurable sets, with the additional assumption that the outer measure is the canonical extension of the restricted measure.MeasureTheory.IsProbabilityMeasure PA measureฮผis called a probability measure ifฮผ univ = 1. -
P' : MeasureTheory.Measure ฮฉ'MeasureTheory.IsProbabilityMeasure P' -
O : โ โ ฮฉ โ ๐ -
A : โ โ ฮฉ โ ๐ -
Y : โ โ ฮฉ โ ๐จ -
O' : โ โ ฮฉ' โ ๐ -
A' : โ โ ฮฉ' โ ๐ -
Y' : โ โ ฮฉ' โ ๐จ -
alg : Algorithm ๐ ๐ ๐จA stochastic, sequential algorithm. -
alg' : Algorithm ๐ ๐ ๐จ -
env : Environment ๐ ๐ ๐จA stochastic environment. -
env' : Environment ๐ ๐ ๐จ
Assuming
-
h : IsAlgEnvSeq O A Y alg env PAn algorithm-environment sequence: a sequence of observations, actions and feedbacks generated by an algorithm interacting with an environment. -
h' : IsAlgEnvSeq O' A' Y' alg' env' P'
Then
InformationTheory.klDiv (MeasureTheory.Measure.map (trajectory O A Y) P)
(MeasureTheory.Measure.map (trajectory O' A' Y') P') =
โ' (t : โ),
InformationTheory.klDiv ((MeasureTheory.Measure.map (history O A Y t) P).compProd (stepKernel alg env t))
((MeasureTheory.Measure.map (history O A Y t) P).compProd (stepKernel alg' env' t))MeasurableSpace : Type u_6 โ Type u_6A measurable space is a space equipped with a ฯ-algebra.
MeasureTheory.IsProbabilityMeasure : {ฮฑ : Type u_1} โ {m0 : MeasurableSpace ฮฑ} โ MeasureTheory.Measure ฮฑ โ PropA measure `ฮผ` is called a probability measure if `ฮผ univ = 1`.
MeasureTheory.Measure : (ฮฑ : Type u_5) โ [MeasurableSpace ฮฑ] โ Type u_5A measure is defined to be an outer measure that is countably additive on measurable sets, with the additional assumption that the outer measure is the canonical extension of the restricted measure. The measure of a set `s`, denoted `ฮผ s`, is an extended nonnegative real. The real-valued version is written `ฮผ.real s`.
Nat : TypeThe natural numbers, starting at zero. This type is special-cased by both the kernel and the compiler, and overridden with an efficient implementation. Both use a fast arbitrary-precision arithmetic library (usually [GMP](https://gmplib.org/)); at runtime, `Nat` values that are sufficiently small are unboxed.
Learning.Algorithm : (๐ : Type u_5) โ
(๐ : Type u_6) โ
(๐จ : Type u_7) โ [MeasurableSpace ๐] โ [MeasurableSpace ๐] โ [MeasurableSpace ๐จ] โ Type (max (max u_5 u_6) u_7)A stochastic, sequential algorithm. At each round, it sees an observation in `๐`, then takes an action in `๐`, and finally receives feedback in `๐จ`. The action is a random function of the past rounds and the current observation.Go to its page
Learning.Environment : (๐ : Type u_5) โ
(๐ : Type u_6) โ
(๐จ : Type u_7) โ [MeasurableSpace ๐] โ [MeasurableSpace ๐] โ [MeasurableSpace ๐จ] โ Type (max (max u_5 u_6) u_7)A stochastic environment. At each round, an observation is drawn prior to the algorithm taking an action. Then the environment provides feedback based on the observation and the action.Go to its page
Learning.IsAlgEnvSeq : {๐ : Type u_1} โ
{๐ : Type u_2} โ
{๐จ : Type u_3} โ
{ฮฉ : Type u_4} โ
{m๐ : MeasurableSpace ๐} โ
{m๐ : MeasurableSpace ๐} โ
{m๐จ : MeasurableSpace ๐จ} โ
{mฮฉ : MeasurableSpace ฮฉ} โ
(โ โ ฮฉ โ ๐) โ
(โ โ ฮฉ โ ๐) โ
(โ โ ฮฉ โ ๐จ) โ
Learning.Algorithm ๐ ๐ ๐จ โโฆAn algorithm-environment sequence: a sequence of observations, actions and feedbacks generated by an algorithm interacting with an environment.Go to its page
Eq : {ฮฑ : Sort u_1} โ ฮฑ โ ฮฑ โ PropThe equality relation. It has one introduction rule, `Eq.refl`.
We use `a = b` as notation for `Eq a b`.
A fundamental property of equality is that it is an equivalence relation.
```
variable (ฮฑ : Type) (a b c d : ฮฑ)
variable (hab : a = b) (hcb : c = b) (hcd : c = d)
example : a = d :=
Eq.trans (Eq.trans hab (Eq.symm hcb)) hcd
```
Equality is much more than an equivalence relation, however. It has the important property that every assertion
respects the equivalence, in the sense that we can substitute equal expressions without changing the truth value.
That is, given `h1 : a = b` and `h2 : p a`, we can construct a proof for `p b` using substitution: `Eq.subst h1 h2`.
Example:
```
example (ฮฑ : Type) (a b : ฮฑ) (p : ฮฑ โ Prop)
(h1 : a = b) (h2 : p a) : p b :=
Eq.subst h1 h2
example (ฮฑ : Type) (a b : ฮฑ) (p : ฮฑ โ Prop)
(h1 : a = b) (h2 : p a) : p b :=
h1 โธ h2
```
The triangle in the second presentation is a macro built on top of `Eq.subst` and `Eq.symm`, and you can enter it by typing `\t`.
For more information: [Equality](https://lean-lang.org/theorem_proving_in_lean4/quantifiers_and_equality.html#equality)
Conventions for notations in identifiers:
* The recommended spelling of `=` in identifiers is `eq`.InformationTheory.klDiv : {ฮฑ : Type u_2} โ {mฮฑ : MeasurableSpace ฮฑ} โ MeasureTheory.Measure ฮฑ โ MeasureTheory.Measure ฮฑ โ ENNRealKullback-Leibler divergence between two measures.
MeasureTheory.Measure.map : {ฮฑ : Type u_4} โ
{ฮฒ : Type u_5} โ
[inst : MeasurableSpace ฮฑ] โ
[inst_1 : MeasurableSpace ฮฒ] โ (ฮฑ โ ฮฒ) โ MeasureTheory.Measure ฮฑ โ MeasureTheory.Measure ฮฒThe pushforward of a measure. If `f` is not an almost everywhere measurable function, we define it to be `0` if `ฮผ = 0`, and to be an arbitrary Dirac mass otherwise. That way we always have `map f 0 = 0`, and the push-forward of a probability measure is always a probability measure.
Learning.trajectory : {๐ : Type u_1} โ
{๐ : Type u_2} โ
{๐จ : Type u_3} โ {ฮฉ : Type u_4} โ (โ โ ฮฉ โ ๐) โ (โ โ ฮฉ โ ๐) โ (โ โ ฮฉ โ ๐จ) โ ฮฉ โ โ โ Learning.Round ๐ ๐ ๐จA random variable that gives the sequence of rounds.Go to its page
tsum : {ฮฑ : Type u_4} โ
{ฮฒ : Type u_5} โ
[AddCommMonoid ฮฑ] โ
[TopologicalSpace ฮฑ] โ (ฮฒ โ ฮฑ) โ optParam (SummationFilter ฮฒ) (SummationFilter.unconditional ฮฒ) โ ฮฑ`โ' i, f i` is the unconditional sum of `f` if it exists, or 0 otherwise. More generally, if `L` is a `SummationFilter`, `โ'[L] i, f i` is the sum of `f` with respect to `L` if it exists, and `0` otherwise. (Note that even if the unconditional sum exists, it might not be unique if the topology is not separated. When the support of `f` is finite, we make the most reasonable choice, to use the sum over the support. Otherwise, we choose arbitrarily an `a` satisfying `HasSum f a`. Similar remarks apply to more general summation filters.)
Learning.history : {๐ : Type u_1} โ
{๐ : Type u_2} โ
{๐จ : Type u_3} โ {ฮฉ : Type u_4} โ (โ โ ฮฉ โ ๐) โ (โ โ ฮฉ โ ๐) โ (โ โ ฮฉ โ ๐จ) โ (n : โ) โ ฮฉ โ Learning.Hist ๐ ๐ ๐จ nHistory of the algorithm-environment sequence before time `n`: the rounds at times `0, ..., n - 1`.Go to its page
MeasureTheory.Measure.compProd : {ฮฑ : Type u_1} โ
{ฮฒ : Type u_2} โ
{mฮฑ : MeasurableSpace ฮฑ} โ
{mฮฒ : MeasurableSpace ฮฒ} โ MeasureTheory.Measure ฮฑ โ ProbabilityTheory.Kernel ฮฑ ฮฒ โ MeasureTheory.Measure (ฮฑ ร ฮฒ)The composition-product of a measure and a kernel.
Learning.stepKernel : {๐ : Type u_1} โ
{๐ : Type u_2} โ
{๐จ : Type u_3} โ
{m๐ : MeasurableSpace ๐} โ
{m๐ : MeasurableSpace ๐} โ
{m๐จ : MeasurableSpace ๐จ} โ
Learning.Algorithm ๐ ๐ ๐จ โ
Learning.Environment ๐ ๐ ๐จ โ
(n : โ) โ ProbabilityTheory.Kernel (Learning.Hist ๐ ๐ ๐จ n) (Learning.Round ๐ ๐ ๐จ)Kernel describing the distribution of the round at time `n` given the history before `n`.Go to its page
Code
lemma IsAlgEnvSeq.klDiv_map_trajectory_stepKernel (h : IsAlgEnvSeq O A Y alg env P)
(h' : IsAlgEnvSeq O' A' Y' alg' env' P') :
klDiv (P.map (trajectory O A Y)) (P'.map (trajectory O' A' Y')) =
โ' t : โ, klDiv (P.map (history O A Y t) โโ stepKernel alg env t)
(P.map (history O A Y t) โโ stepKernel alg' env' t)Proof
by
rw [klDiv_map_trajectory_eq_iSup h.measurable_obs h.measurable_action h.measurable_feedback
h'.measurable_obs h'.measurable_action h'.measurable_feedback, ENNReal.tsum_eq_iSup_nat]
exact iSup_congr fun n โฆ h.klDiv_map_history_stepKernel h' nMeaning last changed in v4.34.0-rc2-76-g565f652 (2026-09-10), the 2th recorded change.
Self-contained, with its dependencies inlined and proofs replaced by sorry: download the raw file ยท open it in the Lean web editor.
Dependency graph
Audit surface: 8 project declarations, 36 external constants
โ Proved: no sorry anywhere in its closure
This is the tool's own reading of one build's recorded axioms, and it is not robust against an author who wants it to pass. Checking meant to be relied on should go through Comparator, which replays the proof through the kernel from an export against an explicit list of permitted axioms.