Return to Answer

Although there are myriad examples given to provide an intuitive understanding of PCA, that fact can almost make it more difficult to grasp at the outset, at least it was for me.

"What was the one thing about PCA that all these different examples from different disciplines have in common??"

What helped me intuitively understand were a couple of math parallels I found, since it's apparent the maths is the easy part for you, although this doesn't help explain it to your grandmother...

Think of a regularization problem, trying to get

$$|| XB - Y || = 0$$

Or in English, break down your data $Y$ into two other matrices which will somehow shed light on the data? If those two matrices work well, then the error between them and $Y$ shouldn't be too much.

PCA gives you a useful factorizaton of $Y$, for all the reasons other people have said. It breaks the matrix of data you have, $Y$, down into two other useful matrices. In this case, $X$ would be a matrix where the columns are first $k$ PCs you kept, and $B$ is a matrix giving you a recipe to reconstruct the columns of matrix $Y$ using the columns of $X$. $B$ is the first $k$ rows of $S$, and all of $V$ transpose. The eigenvalues on the diagonal of $S$ basically weights which PCs are most important. That is how the math explicitly tells you which PCs are the most important: they are each weighted by their eigenvalues. Then, the matrix $V^\mathrm{T}$ tells the PCs how to combine.

I think people gave many intuitive examples, so I just wanted to share that. Seeing that helped me understand how it works. There are a world of interesting algorithms and methods which do similar things as PCA. Sparse coding is a subfield of machine learning which is all about factoring matrix $A$ into two other useful and interesting ones that reflect patterns in $A$.

Although there are many examples given to provide an intuitive understanding of PCA, that fact can almost make it more difficult to grasp at the outset, at least it was for me.

"What was the one thing about PCA that all these different examples from different disciplines have in common??"

What helped me intuitively understand were a couple of math parallels, since it's apparent the maths is the easy part for you, although this doesn't help explain it to your grandmother...

Think of a regularization problem, trying to get

$$|| XB - Y || = 0$$

Or in English, break down your data $Y$ into two other matrices which will somehow shed light on the data? If those two matrices work well, then the error between them and $Y$ shouldn't be too much.

PCA gives you a useful factorizaton of $Y$, for all the reasons other people have said. It breaks the matrix of data you have, $Y$, down into two other useful matrices. In this case, $X$ would be a matrix where the columns are first $k$ PCs you kept, and $B$ is a matrix giving you a recipe to reconstruct the columns of matrix $Y$ using the columns of $X$. $B$ is the first $k$ rows of $S$, and all of $V$ transpose. The eigenvalues on the diagonal of $S$ basically weights which PCs are most important. That is how the math explicitly tells you which PCs are the most important: they are each weighted by their eigenvalues. Then, the matrix $V^\mathrm{T}$ tells the PCs how to combine.

I think people gave many intuitive examples, so I just wanted to share that. Seeing that helped me understand how it works. There are a world of interesting algorithms and methods which do similar things as PCA. Sparse coding is a subfield of machine learning which is all about factoring matrix $A$ into two other useful and interesting ones that reflect patterns in $A$.

Although there are myriad examples given to provide an intuitive understanding of PCA, that fact can almost make it more difficult to grasp at the outset, at least it was for me.

"What was the one thing about PCA that all these different examples from different disciplines have in common??"

What helped me intuitively understand were a couple of math parallels I found, since it's apparent the maths is the easy part for you, although this doesn't help explain it to your grandmother...

Think of a regularization problem, trying to get

$$|| XB - Y || = 0$$

Or in English, break down your data $Y$ into two other matrices which will somehow shed light on the data? If those two matrices work well, then the error between them and $Y$ shouldn't be too much.

PCA gives you a useful factorizaton of $Y$, for all the reasons other people have said. It breaks the matrix of data you have, $Y$, down into two other useful matrices. In this case, $X$ would be a matrix where the columns are first $k$ PCs you kept, and $B$ is a matrix giving you a recipe to reconstruct the columns of matrix $Y$ using the columns of $X$. $B$ is the first $k$ rows of $S$, and all of $V$ transpose. The eigenvalues on the diagonal of $S$ basically weights which PCs are most important. That is how the math explicitly tells you which PCs are the most important: they are each weighted by their eigenvalues. Then, the matrix $V^\mathrm{T}$ tells the PCs how to combine.

I think people gave many intuitive examples, so I just wanted to share that. Seeing that helped me understand how it works. There are a world of interesting algorithms and methods which do similar things as PCA. Sparse coding is a subfield of machine learning which is all about factoring matrix $A$ into two other useful and interesting ones that reflect patterns in $A$.

Anyhow have fun!

Although there are myriad examples given to provide an intuitive understanding of PCA, that fact can almost make it more difficult to grasp at the outset, at least it was for me.

"What was the one thing about PCA that all these different examples from different disciplines have in common??"

What helped me intuitively understand were a couple of math parallels I found, since it's apparent the maths is the easy part for you, although this doesn't help explain it to your grandmother...

Think of a regularization problem, trying to get

$$|| XB - Y || = 0$$

Or in English, break down your data $Y$ into two other matrices which will somehow shed light on the data? If those two matrices work well, then the error between them and $Y$ shouldn't be too much.

PCA gives you a useful factorizaton of $Y$, for all the reasons other people have said. It breaks the matrix of data you have, $Y$, down into two other useful matrices. In this case, $X$ would be a matrix where the columns are first $k$ PCs you kept, and $B$ is a matrix giving you a recipe to reconstruct the columns of matrix $Y$ using the columns of $X$. $B$ is the first $k$ rows of $S$, and all of $V$ transpose. The eigenvalues on the diagonal of $S$ basically weights which PCs are most important. That is how the math explicitly tells you which PCs are the most important: they are each weighted by their eigenvalues. Then, the matrix $V^\mathrm{T}$ tells the PCs how to combine.

I think people gave many intuitive examples, so I just wanted to share that. Seeing that helped me understand how it works. There are a world of interesting algorithms and methods which do similar things as PCA. Sparse coding is a subfield of machine learning which is all about factoring matrix $A$ into two other useful and interesting ones that reflect patterns in $A$.

Although there are myriad examples given to provide an intuitive understanding of PCA, that fact can almost make it more difficult to grasp at the outset, at least it was for me.

"What was the one thing about PCA that all these different examples from different disciplines have in common??"

What helped me intuitively understand were a couple of math parallels I found, since it's apparent the maths the easy part for you, although this doesn't help explain it to your grandmother...

Think of a regularization problem, trying to get

|| XB - Y || = 0

Or in english, break down your data Y into two other matrices which will somehow shed light on the data? If those two matrices work well, then the error between them and Y shouldn't be too much.

PCA gives you a useful factorizaton of Y, for all the reasons other people have said. It breaks the matrix of data you have, Y, down into two other useful matrices. In this case, X would be a matrix where the columns are first k PCs you kept, and B is a matrix giving you a recipe to reconstruct the columns of matrix Y using the columns of X. B is the first k rows of S, and all of V transpose. The eigenvalues on the diagonal of S basically weights which PCs are most important. That is how the math explicitly tells you which PCs are the most important: they are each weighted by their eigenvalues. Then, the matrix V^T tells the PCs how to combine.

I think people gave many intuitive examples, so I just wanted to share that. Seeing that helped me understand how it works. There are a world of interesting algorithms and methods which do similar things as PCA. Sparse coding is a subfield of machine learning which is all about factoring matrix A into two other useful and interesting ones that reflect patterns in A.

Anyhow have fun!

Although there are myriad examples given to provide an intuitive understanding of PCA, that fact can almost make it more difficult to grasp at the outset, at least it was for me.

"What was the one thing about PCA that all these different examples from different disciplines have in common??"

What helped me intuitively understand were a couple of math parallels I found, since it's apparent the maths is the easy part for you, although this doesn't help explain it to your grandmother...

Think of a regularization problem, trying to get

$$|| XB - Y || = 0$$

Or in English, break down your data $Y$ into two other matrices which will somehow shed light on the data? If those two matrices work well, then the error between them and $Y$ shouldn't be too much.

PCA gives you a useful factorizaton of $Y$, for all the reasons other people have said. It breaks the matrix of data you have, $Y$, down into two other useful matrices. In this case, $X$ would be a matrix where the columns are first $k$ PCs you kept, and $B$ is a matrix giving you a recipe to reconstruct the columns of matrix $Y$ using the columns of $X$. $B$ is the first $k$ rows of $S$, and all of $V$ transpose. The eigenvalues on the diagonal of $S$ basically weights which PCs are most important. That is how the math explicitly tells you which PCs are the most important: they are each weighted by their eigenvalues. Then, the matrix $V^\mathrm{T}$ tells the PCs how to combine.

I think people gave many intuitive examples, so I just wanted to share that. Seeing that helped me understand how it works. There are a world of interesting algorithms and methods which do similar things as PCA. Sparse coding is a subfield of machine learning which is all about factoring matrix $A$ into two other useful and interesting ones that reflect patterns in $A$.

Anyhow have fun!