Course description
We have entered the era of big data. This deluge of data calls for automated methods of data analysis, which is what machine learning or pattern recognition provides. Machine learning is a set of methods that can automatically detect patterns in data, and then use the uncovered patterns to predict future data, or to perform other kinds of decision making under uncertainty (such as planning how to collect more data!). This introductory pattern recognition/machine learning course gives an overview of many popular models and algorithms used in modern machine learning, both statistical and non-statistical. The course gives students the basic ideas and intuition behind these methods, as well as a more formal understanding of how and why they work. Students have an opportunity to experiment with machine learning techniques and apply them to real-life problems, making extensive use of machine learning libraries.
Learning outcomes
- CLO-1: Ability to understand basic concepts of machine learning and Bayesian decision theory
- CLO-2: Ability to understand and implement different machine learning algorithms
- CLO-3: Ability to use machine learning libraries to apply machine learning algorithms and to develop models to solve problems
- CLO-4: Ability to appraise research articles in different machine learning domains
Topics
Week | Topic |
|---|---|
1 | Machine Learning/Pattern Recognition basics: Supervised and Unsupervised Learning - Classification, Clustering and Regression, Introduction to Supervised Classification – Models and features, Curse of Dimensionality, Over-fitting, Performance Measures |
2 | Review of probability theory Lab session: basic data preprocessing and visualization using pandas, numpy, sklearn and fitting a model to data |
3-4 | Bayesian Decision Theory – Bayes Theorem, ML and MAP Classifiers, Naive Bayes classifier |
5 | Normal Variables and its Discriminant Analysis |
6 | Parametric Density Estimation - MLE, Bayesian Density Estimation |
7 | Nonparametric Density Estimation - Kernel Density Estimators and KNN |
8 | Regression - Linear Regression Analysis and Bayesian Linear Regression |
9 | Logistic Regression and Multiclass Classification |
10 | Tree based Methods - Decision Trees and Random Forests, Ensemble Methods - Bagging and Boosting, Gradient boosted trees |
Linear Models for Classification - Fisher's Linear Discriminant, Support Vector Machines Introduction to Graphical Models - Bayesian Networks, Introduction to exact and approximate Inference Methods | |
11-12 | Introduction to Reinforcement Learning – MDPs, Q-learning |
Textbooks
- [ISLR] An Introduction to Statistical Learning with Applications in Python – Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani, Jonathan Taylor. Available online at https://www.statlearning.com/
- [PML] Probabilistic Machine Learning: An Introduction – Kevin Murphy, MIT Press, 2021.
- [SKL] Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition – Aurélien Géron, O'Reilly Media, Inc.
- [DM] Data Mining: Concepts and Techniques – Jiawei Han, Micheline Kamber, Jian Pei, 3rd Edition, Morgan Kaufmann
- [AI] Artificial Intelligence: A Modern Approach, 4th edition – Stuart Russell and Peter Norvig, Pearson.
References
- [PRML] Pattern Recognition and Machine Learning – Christopher M. Bishop, Springer
- [PR] Pattern Recognition – S. Theodoridis, K. Koutroumbas, 4th Edition, Academic Press.
Useful online resources and similar courses
- Introduction to Machine Learning, Dr. Rita Osadchy, University of Haifa
- Machine Learning and Data Mining
- Introduction to Neural Networks and Machine Learning