After successful completion of the course, students are able to:

• Understand the basics of generative, probabilistic (Bayesian) modeling and inference

• Construct probabilistic models via an expressive probabilistic programming language

• Explain how general purpose programming languages can be extended to support probabilistic constructs

• Understand standard inference algorithms and their implementations in probabilistic programming languages (MCMC, Variational Inference, etc.)

• Independently read literature in the probabilistic programming space

Probabilistic programming is a general framework to express probabilistic models as programs. It lies at the intersection of machine learning, statistics, and programming languages. While it has classically been seen as mechanization of Bayesian statistical inference, it has recently emerged as a candidate for next-generation AI toolchains.

This course will convey both theoretical and practical aspects of using probabilistic AI to express complex probabilistic models as programs and understand the interplay of modeling and inference to efficiently solve real world problems:

• Generative (Bayesian) Models

• Conditioning and Posterior Sampling, 

• Programmable Inference for Probabilistic Programming Languages

• Deep Probabilistic Programs (Bayesian Neural Networks), 

• Inference Methods: Markov Chain Monte Carlo (MCMC), Hamiltonian Monte Carlo, Variational Inference

• Applications of Probabilistic Programming

• Regular lectures about theoretical topics

• Individual assignments in probabilistic programming language 

• Independent group projects implemented in a probablistic programming language (Gen, Turing, Pyro, PyMC3, etc.) with final presentations

150 hours 

• 5 x 2h Lecture = 10h

• 3 x 20h Individual Assignments = 60h

• 1 x 60h Group Project = 60h

• Preparing Presentation = 10h

• Attending Final Presentations = 10h

Assignments and independent project with final presentations