Hot answers tagged

8

The first principal component describes the direction of maximal variation. The three points lie along a straight line. The direction of maximal variation is along that straight line. Consequently, the first principal component is the unit vector at either $45$ degrees or $225$ degrees from the x-axis (along the line). It will help to draw this out, but I ...


5

If I understood your data correctly, one method would be calculating feature importances for each model and maybe plotting them. Below is an example beeswarm plot and code in R for 198 different models that predict wind speed of storms using four features, with the features in decreasing order by average importances. Each point represents one feature ...


4

The PCA is not good or bad. The values you have, where two principal components do not explain a big part of variance, mean that the data is far from being near a 2 dimensional subspace. I can understand your data have many dimensions, as the variance of the second component only explains 6% and rest of components must explain even less each. Here is an ...


3

The reason you are not breaking PCA is because your data is still "simple" and have strong "linear properties". In your first example, the line example, we can summarize data as follows: the regression target will be larger, with respect x and y, i.e., in original feature space, the upper right corner. In your second example, the S shaped example, we can ...


3

Not a full answer so long, but trying to answer the literal question in the title what to do with so many regressions. So, you have 200 similar regression models for some parallel data. Each response is "the same" variable measured at different points. So presumably, the estimated coefficients ought to be similar. So make some plots: For each estimated ...


3

Multiplying $X$ by $W$ gives you transformed data. Multiplying $X$ by $P$ gives you $X$ again ($P$ is the identity matrix because the eigenvectors are orthogonal). Observe that $$ \text{Var}\left[W^T X\right] = W^T\text{Var}\left[X\right]W \approx W^T\frac{1}{n}X^TXW $$ You're writing the spectral decomposition as $$ X^TX = V \begin{bmatrix} \lambda_1 ...


2

Principal Component Analysis is an unsupervised method, with the resulting latent variables depending only on the values in the supplied X matrix. Linear Discriminant Analysis is a supervised method, where the resultant latent variables are selected to maximise the separation of the samples into classes provided in a second target matrix. See here for a ...


1

In a deep autoencoder, you can freeze all components of the bottleneck layer and then perturb them one by one to see how each of them affects the output. This may lead to interesting insights. However, as you increase the value of a specific component, do not expect the output to only increase its intensity, because the model is not linear. Also, do not ...


1

You can't enter $n$ dimensional vectors into a NN with input dimension $k$. Even if you could, it wouldn't make sense because the interpretations of dimensions changed. What is sensible here is to project your test set onto new axes defined by your training set, and use them for testing.


1

What could be the reason? This means that the first two PCs don't separate the groups. PCA is an unsupervised-learning algorithm. This means that PCA has no information about the groups you want to separate. By contrast, supervised-learning is explicitly designed to separate groups according to their features. You would probably have more success using a ...


1

Ok I try my best to explain some of terms, the eigenvectors are stored under pca.components_. In the post you linked, they took the explained_variance_ to scale the length of the eigenvector by its eigenvalue, but for plotting you most likely don't need that. So I use the iris data: import pandas as pd import seaborn as sns from sklearn.decomposition import ...


1

You probably had several ordinal variables, so your objects were described by several traits and you wanted to collapse this long description into few new traits. You succeeded an now your objects are described by 3 new traits. This description is in object scores. Component loadings tell you how original variables were transformed to new ones. So you ...


1

If the goal is dimensionality reduction, PCA may not be a good option in this case according to the variances you found. Some non-linear coordinate transformation may be much more effective.


1

sklean has no fault in this. The numpy array you use has data type int64, so when you divide by a number and save back to the same numpy array, the values are converted to integers. To prevent it, just use the following casting: xy = houses[['GrLivArea', 'SalePrice']].values.astype(np.float)


1

Yes. If the variables were standardized (that is alias to say that the PCA was based on correlation matrix), then loading values are correlations between the PCs and the variables. (All the p X m loadings are the correlations.) Likewise: Yes. If the variables were centered (that is alias to say that the PCA was based on covariance matrix), then loading ...


1

The other answers use a different terminology than what the author may be familiar with. Below, I refer to the scores matrix and use principal components to refer to the unit variance eigenvectors. If you consider the answer as applied to the general case of in-sample and out-of-sample regression, then knowing the principal components matrix is sufficient ...


Only top voted, non community-wiki answers of a minimum length are eligible