1
$\begingroup$

I struggle to understand how to get further in my PCA analysis after computing the eigenvectors of the covariance matrix.

I choose the number of principal components which contain 95% of the information and reduced the number of dimensions from 168 to 8.

But now I end up with 8 eigenvectors. How do I use these? I don't know how to continue from this point.

$\endgroup$
  • 3
    $\begingroup$ What is your ultimate goal? Do you want to just reduce the dimensionality of the data set or do you want to extract some meaningful information after performing PCA? $\endgroup$ – camillejr Aug 29 at 15:05
  • 1
    $\begingroup$ I would like to exctract some meaningful data. What im trying to acomplish with the analysis is to "predict" normal behavior of electricity consumption. So i have constructed an input matrix where each row represent weeks, and colunms represent hours in a week. The matrix then contains hourly values for 168 weeks. 1) I have standardized the matrix by subtracting the mean of each row and dividing with the standard deviation. 2) finding the covariance of the standardized matrix 3) Finding eigenvectors with the higest eigenvalue and picked 8 principal componentes. 4) Now im stuck $\endgroup$ – Mads Andersen Aug 30 at 7:48
  • $\begingroup$ I see. So I assume your matrix is 168 x 168? One thing I would suggest is to plot your 8 eigenvectors and see if what you see makes sense. Each eigenvector has 168 entries, right? Each of those entries should correspond to an hour in a week. So if you plot, you see some shape, a "mode", that spans over 168 hours in a week. Then your real electricity consumption in any given week is some linear combination of all the modes, but the first modes are the most important. $\endgroup$ – camillejr Aug 30 at 20:49
  • $\begingroup$ So within the first modes you should see the hints for what the normal behaviour, or the mean behaviour is. (You've probably already looked at the eigenvalues to select the 8 eigenvectors, but you can also look specifically if perhaps the first eigenvalue is much higher than the next 7. That would mean that the first eigenvector (mode) is by far the most important "shape" for your electricity consumption in any given week.) $\endgroup$ – camillejr Aug 30 at 20:50
  • $\begingroup$ Yes exactly! The first eigenvector with the higest eigenvalue does has the wanted shape, but does that mean that i can discard the rest? .... And how can i apply this shape from the first eigenvector, to get an estimate of actual consumption? All eigenvectors is centered around zero on the x-axis and are nowhere near the actual result. Sorry for all my questions - but i feel like im almost there, but i just need a little help. $\endgroup$ – Mads Andersen Sep 3 at 12:58
2
$\begingroup$

Your data is $k=168$ dimensional, so each of your eigenvectors and your data vectors are $k$ dimensional. Eigenvectors represent directions of change in your data. They're like your new x,y,z axes. When your select $m=8$ of them, it means you select $m$ of these axes. Then, the data is projected onto these axes. The projection is done via dot products. For example, if the data vector is $d=[2,1,3]$ and we project this vector onto $x$ axis (i.e. $\hat{x}=[1,0,0]$), we calculate the dot product $<\hat{x},d>=2$, and get the $x$ coordinate of your data vector $d$. By projecting the data onto new axes, i.e. onto the eigenvectors, we get our new coordinates. So, each of your $k$ dimensional original data points will map to $m$ dimensional new data points.

$\endgroup$
0
$\begingroup$

Then you multiply the matrix of original features (let’s say $X$) by the matrix whose columns are the k chosen eigenvectors (let’s say $V$) corresponding to the k largest eigenvalues (the 8 representing 95% of the total variance) to get to the reduced-set of transformed uncorrelated features: $$X^{*}=XV$$ Which is a matrix with 8 columns and a number of rows equal to the number of samples that you have, as this matrix represents the 8 PCA-transformed features observed for each sample (row of $X^{*} $). Now you can use this new set of features for multiple purposes, including (which is very common!) the estimation a model for the prediction of a certain dependent variable, with the desiderabile properties that, compared to the previous features, the new transformed ones are not correlated and they have reduced the number of predictors from 168 to 8 while retaining a large percentage (95%) of their original total variance.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.