Course Description

The ever-growing influx of data allows us to develop, interpret and apply an increasing set of learning techniques. However, with this increase in data comes a challenge: how to make sense of the data and identify the components that really matter in our modeling efforts. This course gives a detailed and modern overview of statistical learning with a specific focus on high-dimensional data.

In this course we emphasize the tools that are useful in solving and interpreting modern-day analysis problems. Many of these tools are essential building blocks that are often encountered in statistical learning. We also consider the state-of-the-art in handling machine learning problems. We will not only discuss the theoretical underpinnings of supervised learning, but focus also on the skills and experience to rapidly apply these techniques to new problems.

During this course, participants will actively learn how to apply the main statistical methods in data analysis and how to use machine learning algorithms and visualization techniques, especially on high-dimensional data problems. The course has a strongly practical, hands-on focus: rather than focusing on the mathematics and background of the discussed techniques, you will gain hands-on experience in using them on real data during the course and interpreting the results.

Prerequisites

The course INFOMDA1 (or equivalent) serves as a sufficient entry requirement for this course. For information about the contents of the INFOMDA1 course, refer to its course website.

Course Objectives

At the end of this course, students are able to apply and interpret the theories, principles, methods and techniques related to contemporary data science and understand and explain different approaches to data analysis:

apply data visualization and dimension reduction techniques on high dimensional data sets
implement, understand, and explain methods and techniques that are associated with advanced data modeling, including regularized regression, principal components, correspondence analysis, neural networks, clustering, time series, text mining and deep learning.
evaluate the performance of these techniques with appropriate performance measures.
select appropriate techniques to solve specific data science problems.
motivate and explain the choice for techniques to investigate data problems.
interpret and evaluate the results of (high-dimensional) data analyses and explain these techniques in simple terminology to a broad audience.
understand and explain the principles of high-dimensional data analysis and visualization.
construct appropriate visualizations for each data analysis technique in R.

Required Readings

Freely available sections from the following books:

ISLR: James, G., Witten, D., Hastie, T., & Tibshirani, R. (2021). An introduction to statistical learning (2nd ed.). Springer. statlearning.com
SLS: Hastie, T., Tibshirani, R., & Wainwright, M. (2015). Statistical learning with sparsity CRC Press. web.stanford.edu/~hastie/StatLearnSparsity
ESL: Hastie, T., Tibshirani, R., and Friedman, J. (2001). The Elements of Statistical Learning: Data Mining, Inference and Prediction. New York: Springer Verlag. web.stanford.edu/~hastie/ElemStatLearn.
R4DS: Wickham, H., & Grolemund, G. (2016). R for data science: import, tidy, transform, visualize, and model data. O’Reilly Media, Inc. r4ds.had.co.nz
MBCC: Bouveyron, C., Celeux, G., Murphy, T., & Raftery, A. (2019). Model-based Clustering and Classification for Data Science: With Applications in R Cambridge University Press. cambridge.org
FPP3 : Hyndman, R. J. & Athanasopoulos, G. (2021). Forecasting: Principles and Practice (3rd ed.) Otexts. otexts.com/fpp3
TTMR: Silge, J., & Robinson, D. (2021). Text mining with R: A tidy approach. O’Reilly Media, Inc. tidytextmining.com
SLP3: Jurafsky, D., Martin, J.H. (2021) Speech and language processing. (3rd ed.) https://web.stanford.edu/~jurafsky/slp3/
Some freely available articles & chapters.

Required Software

In this course, we will exclusively use R & RStudio for data analysis. First, install the latest version of R for your system (see https://cran.r-project.org/). Then, install the latest (desktop open source) version of the RStudio integrated development environment (link).

We will make extensive use of the tidyverse suite of packages, which can be installed from within R using the command install.packages("tidyverse").

Course Policy

Weekly course flow

There will be a lecture and a lab session each week.
The required readings should be read before the lecture. These are not optional.
There are some take-home exercises to be done before each lab session; these will be discussed during the session.
There are some additional exercises to be done during the lab session; the answers to these will be made available after the session.

Grading policy

To develop the necessary skills for completing the assignments and the exam, 9 R practicals must be made. These exercises are not graded, but students must fulfill them to pass the course.
There are two pass/fail assignments. Successful and timely completion of these assignments will grant you a bonus point on the exam.
100% of your grade will be determined by an exam featuring both knowledge questions as well as practical data analysis skills in R. Some example questions will be made available to you so you can prepare.

Class Schedule

Key dates and deadlines

Day	Date	Time	Location	Description
Wednesday	17-11-2021	13:15 - 15:00	BBG 161	Lecture 1
Friday	19-11-2021	13:15 - 15:00	BBG 201	Q&A 1
Wednesday	24-11-2021	13:15 - 15:00	BBG 161	Lecture 2
Friday	26-11-2021	13:15 - 15:00	BBG 201	Q&A 2
Wednesday	01-12-2021	13:15 - 15:00	BBG 161	Lecture 3
Friday	03-12-2021	13:15		Deadline assignment 1
Friday	03-12-2021	13:15 - 15:00	BBG 201	Q&A 3
Wednesday	08-12-2021	13:15 - 15:00	BBG 161	Lecture 4
Friday	10-12-2021	13:15 - 15:00	BBG 201	Q&A 4
Wednesday	15-12-2021	13:15 - 15:00	BBG 161	Lecture 5
Friday	17-12-2021	13:15 - 15:00	BBG 201	Q&A 5
Wednesday	22-12-2021	13:15 - 15:00	BBG 161	Lecture 6
Friday	24-12-2021	13:15 - 15:00	BBG 201	Q&A 6
Break
Wednesday	12-01-2022	13:15 - 15:00	BBG 161	Lecture 7
Friday	14-01-2022	13:15 - 15:00	BBG 201	Q&A 7
Wednesday	19-01-2022	13:15 - 15:00	BBG 161	Lecture 8
Friday	21-01-2022	13:15		Deadline assignment 2
Friday	21-01-2022	13:15 - 15:00	BBG 201	Q&A 8
Wednesday	26-01-2022	13:15 - 15:00	BBG 161	Lecture 9
Friday	28-01-2022	13:15 - 15:00	BBG 201	Q&A 9
Friday	04-02-2022	08:30 - 11:30	Megaron	Exam
Friday	04-03-2022	TBD		Resit

Lecture 1: Introduction & betting on sparsity with the LASSO

17-11-2021 | 13:15 - 15:00

Required reading

This syllabus
SLS chapter 1
SLS sections 2.1 - 2.4.2
SLS chapter 4 until (not including) example 4.1.

Optional reading

Refresh your memory:

ISLR section 6.2 shrinkage methods
ISLR section 6.4 considerations in high dimensions
Review the tidyverse style guide

Lab session preparation

Assignments 1-5 of the first practical.

Lecture 2: Dimension reduction 1

24-11-2021 | 13:15 - 15:00

Required reading

ISLR sections 12.1-12.2
ESL section 14.5.1 on PCA
ESL section 3.4.1 from page 66, “The SVD of the centered matrix X is another way of expressing the principal components of the variables in X.” until the end of page 67.

Lab session preparation

Take-home exercises of the practical.

Lecture 3: Dimension reduction 2

01-12-2021 | 13:15 - 15:00

Required reading

ISLR section 6.3 dimension reduction methods
Abdi H. and Bera, M. (2018). Correspondence analysis. In R. Alhajj and J. Rokne (Eds.), Encyclopedia of Social Networks and Mining (2nd Edition). New York: Springer Verlag. Freely available from the website.

Lab session preparation

Take-home exercises of the practical.

Assignment 1

Partial least squares (link). Hand in on blackboard before practical 3 (03-12-2021 | 13:15).

Lecture 4: Deep learning

08-12-2021 | 13:15 - 15:00

Required reading

ISLR chapter 10, up to and including section 10.3.3.

Lab session preparation

Take-home exercises of the practical.

Lecture 5: Clustering

15-12-2020 | 13:15 - 15:00

Required reading

ISLR section 12.4
Tan, Steinbach, Karpatne, & Kumar (2019) Introduction to Data Mining section (second edition) section 7.5 cluster evaluation, available here
SLS sections 8.5.1 and 8.5.2.

Optional reading

The remainder of Introduction to Data Mining Chapter 7 Cluster analysis

Lab session preparation

TBD

Lecture 6: Model-based clustering

22-12-2020 | 13:15 - 15:00

Required reading

Mixture models: latent profile and latent class analysis by Daniel Oberski (2016) link
MBCC sections 2.1 and 2.2

Optional reading

MBCC sections 2.3, 2.4, 2.8
MBCC chapter 8 (not freely available)

Lab session preparation

TBD

Winter break

Lecture 7: Time series

12-01-2021 | 13:15 - 15:00

Required reading

FPP3 sections TBD

Lab session preparation

TBD

Lecture 8: Text mining 1

19-01-2021 | 13:15 - 15:00

Required reading

TTMR sections TBD
SLP3 sections TBD

Lab session preparation

TBD

Assignment 2

TBD. Hand in on blackboard before practical 8 (21-01-2022 | 13:15).

Lecture 9: Text mining 2

26-01-2021 | 13:15 - 15:00

Required reading

TTMR sections TBD
SLP3 sections TBD

Lab session preparation

TBD

Exam

04-02-2021 | 8:30 - 11:30

Resit

Target date: 05-03-2021, to be confirmed.