Tagged Questions

-1
votes
1answer
52 views

Singular Value Decomposition and PCR

Can anyone guide me to understand the relation between Singular Value Decomposition (SVD) and Principle Component Regression (PCR)? I know that we can construct the principle components (PCs) using ...
8
votes
1answer
339 views

For a random matrix, shouldn't a SVD explain nothing at all? What am I doing wrong?

If I construct a 2-D matrix composed entirely of random data, I would expect the PCA and SVD components to essentially explain nothing. Instead, it seems like the the first SVD column appears to ...
0
votes
1answer
174 views

Solving PCA with correlation matrix of a dataset and its singular value decomposition

Suppose I have a $d \times n$ matrix $\mathbf X$ (each entry point has $d$ dimensions) and after some manipulation of data (i.e. summarizing the data $\mathbf X$) I get its $d \times d$ symmetric, ...
2
votes
1answer
391 views

PCA and SVD, R prcomp

I am trying to replicate Shalizi's NY Times PCA example found in his Advanced Data Analysis with an Elementary Point of View book. I found sample code and data here ...
1
vote
1answer
57 views

Adversarial noise in PCA

PCA is known to be quite sensitive to outlier noise (and this is why several Robust PCA techniques exists.) However, I am looking for a concrete example of sensitivity of PCA to adversarial noise that ...
0
votes
0answers
85 views

PCA and using SVD [closed]

Suppose we have a data matrix $\textbf{X}$. The SVD of $\textbf{X}$ would be $\textbf{X} = \textbf{W} \Sigma \textbf{V}^{T}$ where $\textbf{W}$ is the matrix of eigenvectors of the covariance matrix ...
2
votes
0answers
162 views

Clear description of PCA using SVD

After reading thousands of articles on PCA and SVD, using them in a number of programming frameworks and even implementing similar techniques (like Random Indexing) I found out that I still have ...
7
votes
1answer
844 views

Dimensionality reduction (SVD or PCA) on a large, sparse matrix

I have a large, sparse Matrix of features I would like to use in a machine learning algorithm: ...
4
votes
0answers
609 views

What is the correct way to calculate the explained variance of each EOF as calculated from a gappy data set?

I am trying to determine the correct amount of variance explained by each mode of an Empirical Orthogonal Function (EOF) analysis (similar to "PCA") as applied to a gappy data set. (i.e., containing ...
5
votes
3answers
497 views

PCA of non-Gaussian data

I have a couple of quick questions about PCA: Does the PCA assume that the dataset is Gaussian? What happens when I apply a PCA to inherently non-linear data? Given a dataset, the process is to ...
7
votes
3answers
491 views

PCA when the dimensionality is greater than the number of samples

I've come across a scenario where I have 10 signals/person for 10 people (so 100 samples) containing 14000 data points (dimensions) that I need to pass to a classifier. I would like to reduce the ...
4
votes
1answer
166 views

SVD of a data matrix after smoothing

Let's say I have a (n x m) centered data matrix $A$ with SVD $A = U \Sigma V^{T}$. For example, m=50 columns (measurements) that are spectra with n=100 different frequencies. The matrix is centered ...
4
votes
1answer
243 views

Does the sign of the principal component become meaningless with centered variables?

Does the sign of the vectors returned by PCA have any significance after I center my variables?
5
votes
2answers
235 views

One component in PCA is always the mean vector in two-dimensions but not three

I've been testing PCA via SVD to decompose a simple time series data matrix, $X$. I have two signals $x_1(t)$ and $x_2(t)$ in a data matrix where $M$ rows represents each timepoint sample and each ...
1
vote
0answers
64 views

Comparing original variables with characteristic values of diagonalized variance-covariance matrix

If I have a reference data set comprising repeated measurements of 3 variables of a system in state $A$. Given new observations of these variables for a different system I would like to classify ...
5
votes
2answers
489 views

Simulated annealing and k-means

One of my problems http://stackoverflow.com/questions/7783933/clustering-data-outputs-irregular-plot-graph suffers from the curse of dimensionality, which also makes it infeasible for exhaustive ...
0
votes
0answers
109 views

princomp relevant information matrix

I want to get $ relative error = \frac{x-x_{0}}{x}$ between the results of R and mine with pca, to do so I am doing ...
7
votes
2answers
362 views

When do we combine dimensionality reduction with clustering?

I am trying to perform document-level clustering. I constructed the term-document frequency matrix and I am trying to cluster these high dimensional vectors using k-means. Instead of directly ...
10
votes
2answers
2k views

“Normalizing” variables for SVD / PCA

Suppose we have $N$ measurable variables, $(a_1, a_2, \ldots, a_N)$, we do a number $M > N$ of measurements, and then wish to perform singular value decomposition on the results to find the axes of ...
8
votes
1answer
321 views

How to interpret results of dimensionality reduction/multidimensional scaling?

I performed both a SVD decomposition and a multidimensional scaling of a 6-dimensional data matrix, in order to get a better understanding of the structure of the data. Unfortunately, all the ...
7
votes
3answers
923 views

What are the differences among latent semantic analysis (LSA), latent semantic indexing (LSI), and singular value decomposition (SVD)?

These terms get thrown around together a lot, but I would like to know what you think the differences are, if any. Thanks