Dimensionality reduction: principal component analysis
Summarising many variables in a few axes: the curse of dimensionality, then PCA (variance, eigenvalues, scree plot, correlation circle, factorial plane); t-SNE and UMAP are only named.
- 25 cards
- 6 topics
- Free
The outline, topic by topic
Each topic is weighted the way the exam weighs it.
- Concepts
- Questions
- The numbers
- The curves
- Scenarios
- Understand
A few cards from the deck
Principal component analysis (PCA): what is it?
Show the answer
A method that replaces correlated variables with new perpendicular axes, ranked by how much of the data's variance they keep.
Each axis is a combination of the original variables. You keep the first ones and lose little information; the method is linear.
A PCA is run on 10 standardised variables. What is the sum of its 10 eigenvalues?
- 10, a variance of 1 per variable
- 1, the total variance being brought to 100%
- 100, that is 10 times 10
- The number of individuals in the dataset
Show the answer
10, a variance of 1 per variable
Each standardised variable has a variance of 1, so the total variance is 10; PCA redistributes it among the axes without losing any, hence a sum of 10. That is why an eigenvalue divided by 10 gives a share of variance.
Before a PCA on variables with different units, you … each column, otherwise the first axis follows the variable with the largest scale.
Show the answer
Before a PCA on variables with different units, you standardise each column, otherwise the first axis follows the variable with the largest scale.
Variance depends on the unit: a weight in grams has a million times the variance of the same weight in kilograms. Standardising gives each variable a variance of 1.
Dimensionality reduction: what is it?
Show the answer
Going from many variables to a few that keep the essential information, in order to visualise, speed up or denoise.
PCA is linear. t-SNE and UMAP are non-linear and mostly used to draw data in two dimensions: the long distances in the picture cannot be read as in a PCA.