105.746 Theory of stochastic processes This course is in all assigned curricula part of the STEOP.\$(function(){PrimeFaces.cw("Tooltip","widget_j_id_21",{id:"j_id_21",showEffect:"fade",hideEffect:"fade",target:"isAllSteop"});});This course is in at least 1 assigned curriculum part of the STEOP.\$(function(){PrimeFaces.cw("Tooltip","widget_j_id_23",{id:"j_id_23",showEffect:"fade",hideEffect:"fade",target:"isAnySteop"});}); 2025S 2024S 2023S 2022S

2024S, VO, 3.0h, 4.5EC

Properties

• Semester hours: 3.0
• Credits: 4.5
• Type: VO Lecture
• Format: Presence

Learning outcomes

After successful completion of the course, students are able to understand the basic concepts underlying the theory of discrete-time and continuous-time Markov chains and discrete-time martingales. They are also able to apply these concepts in a variety of applications.

Subject of course

Stochastic processes are mathematical objects aimed at modelling random phenomena evolving in time. We will deal with two of the simplest, and at the same time most important, types of stochastic processes: Markov chains and martingales. We will also see these models at work in a wide variety of applications.

MARKOV CHAINS

• Basic theory

• (Strong) Markov property
• Communicating classes
• Hitting times
• Recurrence and transience
• Invariant distributions
• Convergence to equilibrium
• Time reversibility
• Ergodic theorem
• Further theory in continuous time

• Generator matrices
• Jump chains and holding times
• Explosion
• Forward and backward equations
• Applications

• Random walks
• Birth-and-death processes
• Moran model in population genetics
• PageRank algorithm
• Markov chain Monte Carlo (MCMC) methods
• Poisson processes
• Queuing systems

MARTINGALES

• Basic theory

• Martingales, supermartingales and submartingales
• Martingale transform
• Stopped martingales
• Doob's optional stopping theorem
• Doob's decomposition
• Maximal inequalities
• Asymptotic theory

• Almost sure convergence: Doob's forward convergence theorem
• Convergence of martingales in L^2
• Uniform integrability and convergence in L^1
• Levy's upward theorem
• Doob's L^p inequality and convergence in L^p
• Applications

• Games and gambling strategies
• Kolmogorov's zero-one law
• Branching processes
• Statistical hypothesis testing
• Filtering problems in statistics
• Secretary problem

Teaching methods

Blackboard lectures, in-class questions and discussions

Mode of examination

Written

In principle, this VO course should run for 6 hours per week, from the beginning of March to the beginning of May. In any case, the schedule is provisional and will be discussed with the students in the first week (for example, we may change date/time or run it for fewer weekly hours over a longer period, e.g. until the end of May). If the provisional schedule does not work for you, please drop an email to the lecturer before the start of the semester and indicate your preference.

Course dates

DayTimeDateLocationDescription
Mon09:00 - 12:0004.03.2024 - 13.05.2024EI 1 Petritsch HS Lecture
Thu08:00 - 11:0007.03.2024 - 18.04.2024Seminarraum 107/1 Lecture
Wed10:00 - 13:0008.05.2024Seminarraum 107/1 Lecture
Theory of stochastic processes - Single appointments
DayDateTimeLocationDescription
Mon04.03.202409:00 - 12:00EI 1 Petritsch HS Lecture
Thu07.03.202408:00 - 11:00Seminarraum 107/1 Lecture
Mon11.03.202409:00 - 12:00EI 1 Petritsch HS Lecture
Mon18.03.202409:00 - 12:00EI 1 Petritsch HS Lecture
Mon08.04.202409:00 - 12:00EI 1 Petritsch HS Lecture
Mon15.04.202409:00 - 12:00EI 1 Petritsch HS Lecture
Thu18.04.202408:00 - 11:00Seminarraum 107/1 Lecture
Mon22.04.202409:00 - 12:00EI 1 Petritsch HS Lecture
Mon29.04.202409:00 - 12:00EI 1 Petritsch HS Lecture
Mon06.05.202409:00 - 12:00EI 1 Petritsch HS Lecture
Wed08.05.202410:00 - 13:00Seminarraum 107/1 Lecture
Mon13.05.202409:00 - 12:00EI 1 Petritsch HS Lecture

Examination modalities

Written exam. Additional points may be awarded for class participation.

The first exam will take place after the end of the lectures, on a date to be agreed with attending students. If you wish to take the exam in a subsequent date, please get in touch directly with the lecturer and another exam will be scheduled.

Exams

DayTimeDateRoomMode of examinationApplication timeApplication modeExam
Tue09:00 - 13:0027.05.2025Seminarraum 127 written13.05.2025 08:00 - 26.05.2025 17:00TISSExam 1
Mon09:00 - 13:0016.06.2025EI 1 Petritsch HS written02.06.2025 08:00 - 13.06.2025 17:00TISSExam 2

Course registration

Begin End Deregistration end
31.01.2024 08:00 28.06.2024 19:00 29.07.2024 20:00

Curricula

Study CodeObligationSemesterPrecon.Info
066 394 Technical Mathematics Mandatory elective
066 395 Statistics and Mathematics in Economics Mandatory elective
066 453 Biomedical Engineering Not specified
066 938 Computer Engineering Mandatory elective
860 GW Optional Courses - Technical Mathematics Not specified

Literature

• Norris, J. (1997). Markov Chains. Cambridge University Press. doi:10.1017/CBO9780511810633
• Williams, D. (1991). Probability with Martingales. Cambridge University Press. doi:10.1017/CBO9780511813658

Previous knowledge

Basic probability theory, calculus and linear algebra

English