PCA

Matrix Basics

Identity Matrix

Matrix Multiplication

Matrix Determinants

Example

Matrix Inverse

Eigen values and vectors

Normalizing the vector

PCA

Introduction

Imagine you have a collection of data with many different measurements or features, like the height, weight, and age of a group of people. You want to understand this data better and make it simpler to work with. PCA is like finding the most important things that describe this data and focusing on them.

Maths Behind PCA

The mathematical part of PCA involves some linear algebra and statistics:

• Covariance Matrix: PCA starts by calculating how different measurements (e.g., height and weight) relate to each other using something called the covariance matrix. This matrix tells us if two measurements tend to go up or down together or in opposite directions.
• Eigenvalues and Eigenvectors: Next, we find the "eigenvalues" and "eigenvectors" of this covariance matrix. Think of eigenvectors as the main directions we talked about earlier, and eigenvalues as numbers telling us how important each direction is.
• Choosing Principal Components: We choose the eigenvectors with the largest eigenvalues because they represent the most important directions in the data. These are our principal components.
• Projection: We project or transform our data onto these principal components to get a new, simplified representation of the data.

How to perform PCA

To see how to perofrm PCA on a dataset, best see the video explaining the whole process.

References:

1. Finding Eigen values and Eigen vectors
2. PCA the math
3. Inverse Matrices and Their Properties

Note: Parts of the article are developed by using ChatGPT