Tag Info

7

There is not a formal proof because the assertion is false: autoencoders do not perform non-linear PCA. PCA is defined as a (reversible) linear transformation into a space where variables are now orthogonal that captures maximal variance. Autoencoders do not do that in general. Linear autoencoders with $k$-dimensional bottlenecks will often converge to the ...

5

They are very different concepts, that unfortunately share a common term. For convolutions, the kernel is a tensor that indicates how the value at a particular index is calculated from a neighborhood of that index. For example, you can have a kernel that calculates the average of the 9 pixel neighborhood (for an image) by using the kernel [[1/9, 1/9, 1/9], [...

3

It is basically OK to use the criteria from PCA as part of a guide to selecting the number of factors. Most of the times, FA and PCA results will be in agreement. Parallel analysis and Velicer's minimum average partial (MAP) are the most reliable and accurate techniques to assess the number of components or factors to retain, according to Zwick & Velicer....

2

This is a scaling issue, as is often the case when using these methods. You asked to scale the variables when calling pcr (scale=T), but your manual computation makes use of the raw (unscaled) data. > pcr1 = pcr(y ~ ., ncomp = k, data = d) > betahat = coef(pcr1, ncomp = k, intercept = T) --%<---------------------------------------- > data.frame(...

2

Note that PCA is an unsupervised method. In most cases, when we say the data is imbalanced, we are talking about the prediction label has skewed distribution. We can run PCA on any data, even the data is not 'evenly distributed'. PCA just rotate data, and optionally, maps data into a lower dimensional space. It is not clear on what is the performance of ...

1

PCA doesn't predict anything so I think you may be committing a category error here. PCA is an optimal low-rank approximation method. You can use it to reduce the number of dimensions in a dataset amongst other things. Sometimes people apply PCA to a dataset prior to using other algorithms in order to reduce collinearity between variables because the PCA ...

1

Yes. Because you have a fairly small amount of features already (15), it makes sense if you weren't able to reduce the dimensionality much further without reducing the explanation for variance. PCA is often done on datasets with hundreds or thousands of features to reduce the dataset. Although, to note, if you did have highly-redundant features, it is also ...

1

This is probably a comment rather than an answer and I'm not sure I'm getting it right. Let's try to compare the output of a linear model on two nearly colinear variables before and after PCA: set.seed(1234) x1 <- 1:10 x2 <- x1 + rnorm(n= length(x1), sd= 0.0001) # x2 is nearly colinear to x1 y <- rowMeans(cbind(x1, x2)) + rnorm(n= length(x1)) # A ...

1

We have a dedicated thread for that very specific purpose: Using principal component analysis (PCA) for feature selection. Just a few points regarding the interpretation of those visual displays, and some reflexions on the question at hand: This graphical output is a visual aid to see which variable contribute the most to the definition of the principal ...

1

I think you will find this paper helpful: D. Kunin et al., "Loss Landscapes of Regularized Linear Autoencoders (2019). The authors discuss the convergence of linear autoencoders to the principal subspace and how regularization enables the recovery of the actual principal components (not just the subspace). You might also find this paper helpful: E. ...

1

Canonical correlation analysis (CCA) is mainly concerned with characterizing the linear association between two blocks of variables (or more blocks in generalized CCA). We can think of it as an extension of the use of a correlation matrix to summarize correlations in a multivariate dataset. In this respect, no single block plays the role of a response block ...

1

PCA is not designed for noise removal purpose. It is designed to REDUCE DIMENSIONS. As a large number of features are difficult to handle. PCA just lets you to approximate your data. Think PCA as a tuning knob. You can smoothly decide how much approximation you want by tuning it and which is impossible to achieve if you work directly with original given ...

1

My suggestion is to concatenate all 4 matrices into a single one with 11264 + 120 + 89 + 128 measures for each of the 284 individuals. And then perform the PCA on this resulting matrix. in this approach, if you select the PCA to keep k dimensions of the combined matrix, you will have the best k uncorrelated, derived (though a linear combinations of the ...

Only top voted, non community-wiki answers of a minimum length are eligible