9k views

### Why are we using a biased and misleading standard deviation formula for $\sigma$ of a normal distribution?

It came as a bit of a shock to me the first time I did a normal distribution Monte Carlo simulation and discovered that the mean of $100$ standard deviations from $100$ samples, all having a sample ...
• 13.2k
30k views

### What is the relationship between regression and linear discriminant analysis (LDA)?

Is there a relationship between regression and linear discriminant analysis (LDA)? What are their similarities and differences? Does it make any difference if there are two classes or more than two ...
• 861
3k views

### Practical usefulness of PCA

I asked a similar question in the past, but I've thought about the message I am trying to convey a bit more and feel I can articulate it better. For context, I am on an introductory course in machine ...
24k views

### How to whiten the data using principal component analysis?

I want to transform my data $\mathbf X$ such that the variances will be one and the covariances will be zero (i.e I want to whiten the data). Furthermore the means should be zero. I know I will get ...
• 1,631
12k views

### Why is Python's scikit-learn LDA not working correctly and how does it compute LDA via SVD?

I was using the Linear Discriminant Analysis (LDA) from the scikit-learn machine learning library (Python) for dimensionality reduction and was a little bit curious ...
7k views

### Optimization with orthogonal constraints

I am working on computer vision, and have to optimize an objective function involves matrix $X$ and matrix $X$ is an orthogonal matrix. $$maximize \ \ f(X)$$ $$s.t \ \ X^T X=I$$ Where $I$ is the ...
• 311
6k views

### How to evaluate estimated principal components?

I am interested in evaluating estimated principal components. The components are estimated from a sample and I want to evaluate how good my estimated principal components are. Are there any commonly ...
• 161
1k views

### Is CCA between two identical datasets equivalent to PCA on this dataset?

Reading Wikipedia about canonical correlation analysis (CCA) for two random vectors $X$ and $Y$, I was wondering if principal component anslysis (PCA) is the same as CCA when $X=Y$?
• 19.6k
3k views

### Why do deep learning practitioners forego PCA for ZCA?

I have an understanding of PCA and ZCA, read a similar question on the subject which, unfortunately, does not have the specific answer to my question. I understand the benefits of data whitening: ...
• 990
1k views

### How to choose the regularization parameter in ZCA whitening?

ZCA whitening can use regularization, as in $$\tilde{X} = L\sqrt{(D + \epsilon)^{-1}}L^{-1}X,$$ where $LDL^\top$ is an eigendecomposition of the sample covariance matrix. What's a good choice for ...
• 1,337
715 views

• 1,636
1 vote
245 views

### When to use PCA of features and when of samples?

I am learning now about the PCA and ZCA applications for the machine learning problems of classification and clustering. I would like to apply PCA and ZCA mostly, but not only, to image data. From ...
• 121
1 vote