PCA

What is PCA?

Principal Component Analysis (PCA) is an unsupervised machine learning technique used for dimensionality reduction and data visualization. It works by transforming a large set of correlated quantitative variables into a smaller set of uncorrelated variables called "principal components" while retaining as much of the original variation (info) in the data as possible. The first principal component captures the max variance, and each additional component captures the remaining variance under the constraint that it's orthogonal to the previous ones. This process allows data scientists to simplify complex datasets, identify dominant patterns or features, and visualize high-dimensional data in 2D or 3D space without losing significant meaning.

Data Before PCA Transformation

The input dataset was constructed by pivoting the cleaned soviet_breakdown_dataset.csv so that each row represents one country and each column represents a year's GDP Per Capita value. This produced a 15 × 33 matrix (15 countries, 33 years from 1988–2020) of purely quantitative values with no labels. The data was then normalized using StandardScaler so that each column had a mean of 0 and a standard deviation of 1.

After PCA Transformation

After applying PCA with n_components=3, the 33-dimensional dataset was reduced to just 3 columns representing the three principal components. Each row still represents one country, but the columns are no longer interpretable years they are abstract mathematical directions that capture the maximum variance in the data.

Percent of data remaining in the 2D dataset

The 2D dataset retains about 97% of the original information. The first principal component captures the vast majority of the variance (general economic growth trends), while the second component captures secondary nuances (like transition speed or crash depth). 

Percentage of information remaining in the 3D dataset:

The 3D dataset retains roughly 97.8% of the original information. Adding the third dimension only captures an additional fraction of a percent of the variance, indicating that the third dimension is largely unnecessary for explaining the differences between these countries. 

Dataset dimensions needed (after using PCA) to retain at least 95% of the data:

To retain at least 95% of the data's variance, we only need 2 dimensions. The original dataset had 33 dimensions (representing 33 years of economic data). However, because national economic trajectories are highly correlated across time, reducing the 33 years into just 2 principal components still safely preserves over 95% of the total mathematical variance. 

The top three eigenvalues of this data:

Link to code used