After successful completion of the course, students are able to (i) apply modern regression/statistical learning methods to build predictive models, (ii) select and validate statistical learning models, (iii) assess model fit and error and (iv) use the R language for modern regression and data analysis.

• Matrix Algebra Review
• Multivariate Normal Distribution
• Review of Linear Models: 
• Simple Linear Regression, 
• Assumptions for Linear Models, 
• Ordinary Least Squares (OLS) estimators 
• Inference: 
• inference for the slope and the intercept, 
• interpretation of results, 
• prediction, 
• F-tests
• Regression Diagnostics: 
• outliers, influential points, 
• graphical diagnostics, remedies, 
• weighted least squares
• Robust regression
• Multiple linear regression: 
• estimation, 
• prediction, 
• diagnostics, 
• nested models, multi-collinearity, ridge regression, 
• qualitative predictors, mixture of continuous and categorical variables,
• model building, variable selection and model validation, LASSO
• Dimension reduction

• Principal Component Analysis and Principal Component Regression 
• Partial Least Squares
• Sufficient Dimension Reduction (lecture notes)

• Smoothing techniques 
• Kernel smoothing
• local polynomial fitting
• splines
• generalized additive models 
• regression trees
• random forests
• Neural Nets
• Regression models with binary response (binary classification), simple and multiple logistic regression
• Generalized linear model: 
• exponential family, 
• link functions, 
• parameter estimation, 
• inference and prediction

The material will be presented in lecture slides in conjunction with derivations on board. Course related information and material, including notes and details about course assignments and exams will be posted in TUWEL.

The linear regression material and some nonparametric regression material is covered in Linear Regression Analysis by Seber and Lee https://onlinelibrary.wiley.com/doi/book/10.1002/9780471722199. Other reference books are Applied Linear Statistical Models, 5th Edition by Kutner et al.; Practical Regression and ANOVA Using R at ftp://cran.r-project.org/pub/R/doc/contrib/Faraway-PRA.pdf ; Introduction to Statistical Learning with applications in R by James, Witten, Tibshirani & Hastie; Elements of Statistical Learning by Hastie, Tibshirani & Friedman.

The statistical software we will use is R. It can be downloaded from the R home page . RStudio offers a GUI R platform.

The prerequisite for the course is 

The final exam will be written and will cover all topics presented in class.

Basic probability and statistics; Linear algebra; Econometrics 1: Linear Models.