Stationary Processes and Discrete Parameter Markov Processes (Graduate Texts i..

Finite Markov Processes and Their Applications by Marius Iosifescu

Diffusions, Markov Processes, & Martingales by DAVID WILLIAMS

Stationary Processes and Discrete Parameter Markov Processes by Bhattacharya

Markov Decision Processes: Discrete Stochastic Dynamic Programming (Wiley Se...

Discrete Stochastic Processes: Tools for Machine Learning and Data Science by Ni

Foundations of non-stationary dynamic programming with discrete time parameter.

Understanding Markov Chains: Examples and by Nicolas Privault PAPERBACK