# Questions tagged [svd]

Singular value decomposition (SVD) of a matrix $\mathbf{A}$ is given by $\mathbf{A} = \mathbf{USV}^\top$ where $\mathbf{U}$ and $\mathbf{V}$ are orthogonal matrices and $\mathbf{S}$ is a diagonal matrix.

407 questions
Filter by
Sorted by
Tagged with
1 vote
29 views

### Variable weighted PCA

I have seen a lot of "weighted PCA" but they are really all on "observations". For example Weighted principal components analysis if you have K variables, N observations, the ...
1 vote
33 views

1 vote
20 views

### How to compare different clusters of different size, rotation, scale and translation?

Assume that you have a matrix $X$ that contains the data inside the left image. The data inside $X$ is not classified. The matrix $X$ also contains outliers/noise. On the right, we can se the template ...
• 425
188 views

### Difference between conducting PCA on $XX^\top$ vs $X^\top X$?

PCA: For a given set of centered data $\mathscr D =\{x_i\}_{i=1}^N \subset \mathbb R^d$, i.e. the data has $N$ examples with dimension $d$. Then the principal directions of PCA can be obtained from ...
156 views

### PCA to reconstruct Binary Data

I'm working with binary 3D matrices. I calculate their PCA (or REOF or SVD) and as a test I would like to reconstruct these matrices from the PCA results. However I realized that because I only keep ...
• 143
1 vote
84 views

### Which features corresponds to which eigenvalues when use SVD in PCA?

Today, after learning about performing $PCA$ using $SVD$, I know $PCA$ will choose $K$ components that have the highest eigenvalues. I have a question which feature will correspond to which ...
• 113
1 vote
81 views

### Is it correct to do SVD from the latent space of an autoencoder?

Is it correct to do SVD from the latent space of an autoencoder? I am asking because I think that by performing SVD from a latent space, and plotting the singular values, it is possible to know the ...
• 11
32 views

### Is there an alternate estimator for a sample covariance matrix when n < p such that the estimator is not singular

Let's say I have $n$ samples which are vectors of length $p$. I know that the $p \times p$ sample covariance matrix is singular if $n \leq p$. Is there another estimator for the covariance that ...
• 147
66 views

### Relationship between SVD of Matrix and SVD of same Matrix with deleted entries (Matrix can be Adjacency Matrix of a Graph)

Could somebody direct to me to some literature dealing with this issue. So we have $X = U\Sigma V^{T}$ and we have $M \odot X = U^{'}\Sigma^{'}V^{'^{T}}$ with M_{i,j} = \begin{cases} ...
1 vote
22 views

### Prediction of Multiple Linear Regression With Constant

Let $X$ be a matrix with $n$ rows and $d$ columns. We know that there exists matrices $U, S, V$, with $U$ of dimensions $(n, d)$, $S$ of dimensions $(d, d)$ and $V$ of dimensions $(d, d)$, which form ...
178 views

### Why is the Scaling Matrix in LDA unnormalized?

I was carrying out LDA (linear Discriminant Analysis) and noticed that the Scaling matrix produced by R is not normalized. Here is an example: ...
• 516
76 views

### Latent Semantic Indexing vs. PCA

I am trying to understand how Latent Semantic Analysis works, reading demonstrations based on singular value decomposition. Let's denote $X$ a $N \times D$ document-term matrix. The $D$ rows of $X$ ...
• 319
1 vote
1k views

### Understanding Leverage Score Sampling to get representative sample

I was reading about Leverage Score Sampling. If I am not wrong then what I know that Leverage Score Sampling help us to select representative sample. But I didn't understand how the whole process is ...
• 131
1 vote
62 views

### is it fair to use a subset of eigenvalues to evaluate the multidimensional variance

I want to find a single metric to assess how spread (or how much variance) a multidimensional dataset (a large number of features) is. I learned that the determinant (or pseudo-determinant) of the ...
• 21
21 views

### Should I do documents transformation at once or a pair at a time for auto grading with cosine similarity?

I'm developing auto grading essay that compares the similarity between the answer key and student answer with cosine similarity. This one is written in php. Let's say in a course there are 30 - 100 ...
1 vote
60 views

### In principal component regression, how to show that $X \beta_\mathrm{PCR}=U\: \mathrm{diag}\left\{1,\ldots, 1, 0, \ldots 0\right\}U^Ty$ if $X=USV^T$

Regarding the third equation in this answer, I'm struggling to figure out how $X \beta_\mathrm{PCR} = U\: \mathrm{diag}\left\{1,\ldots, 1, 0, \ldots 0\right\} U^\top y$. Given $X = USV^T$ where $X$ ...
• 1,189
26 views

### Time-Continuity of PCA/SVD [duplicate]

I have a daily set of observations (Jacard similarity matrix) that I want to embed in a much lower dimensional space. Thus I run PCA or SVD for each day and plot the projection on the top 2 dimensions ...
• 2,246
296 views

### Procrustes Problem for rank-deficient input

It is well known that the solution to the orthogonal Procrustes problem $$\textrm{arg min}_{\Omega \in \text{SO}(n)} ||Y - \Omega X||^2_2,$$ can be expressed in terms of an SVD of the covariance ...
• 301
78 views

### Why do we multiply the centered data with eigen vector instead of taking inverse of the eigen vector while performing PCA

The question may sound stupid but I really don't understand the logic behind this. Whenever we do a PCA, we take a covariance matrix on the centered data and do eigen decomposition. In order to ...
1 vote
52 views

• 23
239 views

### How do ‘arpack’, ‘propack’ and ‘lobpcg’ compare to each other?

I want to implement singular value decomposition on a sparse matrix using the Scipy implementation. The documentation shows three solvers: ‘arpack’, ‘propack’ and ‘lobpcg’. How do these solvers ...
• 361
1 vote
23 views

1 vote
12 views

### PCA: understanding how to use loading vectors of X to recapture the geometry of X

If we suppose that X is an n x d matrix, and then perform PCA to obtain loading vectors ...
1 vote
83 views

### Does PCA centering guarantee projection optimality onto an affine set subject to dimensionality constraint?

Say I have $m$ points in $R^n$, not necessarily with sum 0, and I want to project them onto an affine set $S$ with $\dim(S)=d$ for a given $d$ so as to minimize the sum of the squared distances ...
83 views

### Dot product and linear combination of Basis

I am currently going through the SVD intuition provided here. In the section "From intuition to definition", It says that, First, note that any vector $\textbf{x}$ can be described using ...
1 vote
478 views

### Use K-means clustering on SVD/PCA of data

In an assignment I was suppose to perform K-means clustering on the MNIST dataset (just the 0's and the 1's) and then use SVD/PCA to visualize the data in two dimensions. I missunderstood this and ...
• 129
33 views

• 1,005
1 vote