After successful completion of the course, students are able to understand the basic concepts underlying the theory of discrete-time and continuous-time Markov chains and discrete-time martingales. They are also able to apply these concepts in a variety of applications.

Stochastic processes are mathematical objects aimed at modelling random phenomena evolving in time. We will deal with two of the simplest, and at the same time most important, types of stochastic processes: Markov chains and martingales. We will also see these models at work in a wide variety of applications.

MARKOV CHAINS

• Basic theory

• (Strong) Markov property 
• Communicating classes
• Hitting times
• Recurrence and transience
• Invariant distributions
• Convergence to equilibrium
• Time reversibility
• Ergodic theorem

• Further theory in continuous time

• Generator matrices
• Jump chains and holding times
• Explosion
• Forward and backward equations

• Applications

• Random walks
• Birth-and-death processes
• PageRank algorithm
• Markov chain Monte Carlo (MCMC) methods
• Wright-Fischer model in population genetics
• Poisson processes
• Queuing systems

MARTINGALES

• Basic theory

• Filtrations and adapted processes
• Martingales, supermartingales and submartingales
• Martingale transform
• Stopped martingales
• Doob's optional stopping theorem
• Doob's decomposition
• Maximal inequalities

• Asymptotic theory

• Almost sure convergence: Doob's forward convergence theorem
• Convergence of martingales in L^2
• Uniform integrability and convergence in L^1
• Levy's upward and downward theorems
• Doob's L^p inequality and convergence in L^p

• Applications

• Branching processes
• Games and gambling strategies
• Insurance modelling
• Statistical hypothesis testing
• Filtering problems
• Strong law of large numbers
• Kolmogorov's zero-one law

Blackboard lectures

Face-to-face classes. Due to the COVID pandemic, there may be changes in the teaching modalities.

Written exam. For those who attend the course, there will be the possibility to collect additional bonus points by answering quick recap questions at the beginning of each lecture. These points will be added to the mark of their written exam (only the first time they take the exam).

If you wish to take the exam after the end of the summer semester 2023, please get in touch directly with the lecturer and another exam will be scheduled.

• Norris, J. (1997). Markov Chains. Cambridge University Press. doi:10.1017/CBO9780511810633
• Williams, D. (1991). Probability with Martingales. Cambridge University Press. doi:10.1017/CBO9780511813658

Basic probability theory, calculus and linear algebra