Markov kernel

In probability theory, a Markov kernel (also known as a stochastic kernel or probability kernel) is a map that in the general theory of Markov processes, plays the role that the transition matrix does in the theory of Markov processes with a finite state space. In probability theory, a Markov kernel (also known as a stochastic kernel or probability kernel) is a map that in the general theory of Markov processes, plays the role that the transition matrix does in the theory of Markov processes with a finite state space. Let ( X , A ) {displaystyle (X,{mathcal {A}})} and ( Y , B ) {displaystyle (Y,{mathcal {B}})} be measurable spaces. A Markov kernel κ : X → Y {displaystyle kappa :X o Y} with source ( X , A ) {displaystyle (X,{mathcal {A}})} and target ( Y , B ) {displaystyle (Y,{mathcal {B}})} is a map κ : B × X → [ 0 , 1 ] {displaystyle kappa :{mathcal {B}} imes X o } with the following properties: In other words it associates to each point x ∈ X {displaystyle xin X} a probability measure κ ( d y | x ) : B ↦ κ ( B , x ) {displaystyle kappa (dy|x):Bmapsto kappa (B,x)} on ( Y , B ) {displaystyle (Y,{mathcal {B}})} such that, for every measurable set B ∈ B {displaystyle Bin {mathcal {B}}} , the map x ↦ κ ( B , x ) {displaystyle xmapsto kappa (B,x)} is measurable with respect to the σ {displaystyle sigma } -algebra A . {displaystyle {mathcal {A}}.} . Take X = Y = Z {displaystyle X=Y=mathbb {Z} } , and A = B = P ( Z ) {displaystyle {mathcal {A}}={mathcal {B}}={mathcal {P}}(mathbb {Z} )} (the power set of Z {displaystyle mathbb {Z} } ). Then a Markov kernel is fully determined by the probability it assigns to a singleton set { m } {displaystyle {m}} with m ∈ Y = Z {displaystyle min Y=mathbb {Z} } for each n ∈ X = Z {displaystyle nin X=mathbb {Z} } : Now the random walk κ {displaystyle kappa } that goes to the right with probability p {displaystyle p} and to the left with probability 1 − p {displaystyle 1-p} is defined by where δ {displaystyle delta } is the Kronecker delta. The transition probabilities P ( m | n ) = κ ( { m } | n ) {displaystyle P(m|n)=kappa ({m}|n)} for the random walk are equivalent to the Markov kernel. More generally take X {displaystyle X} and Y {displaystyle Y} both countable and A = P ( X ) ,   B = P ( Y ) {displaystyle {mathcal {A}}={mathcal {P}}(X), {mathcal {B}}={mathcal {P}}(Y)} . Again a Markov kernel is defined by the probability it assigns to singleton sets for each i ∈ X {displaystyle iin X} We define a Markov process by defining a transition probability P ( j | i ) = K j i {displaystyle P(j|i)=K_{ji}} where the numbers K j i {displaystyle K_{ji}} define a (countable) stochastic matrix ( K j i ) {displaystyle (K_{ji})} i.e.