Questions tagged [svd]

Singular value decomposition (SVD) of a matrix $\mathbf{A}$ is given by $\mathbf{A} = \mathbf{USV}^\top$ where $\mathbf{U}$ and $\mathbf{V}$ are orthogonal matrices and $\mathbf{S}$ is a diagonal matrix.

Filter by
Sorted by
Tagged with
1 vote
0 answers
43 views

is it fair to use a subset of eigenvalues to evaluate the multidimensional variance

I want to find a single metric to assess how spread (or how much variance) a multidimensional dataset (a large number of features) is. I learned that the determinant (or pseudo-determinant) of the ...
  • 11
0 votes
0 answers
18 views

Should I do documents transformation at once or a pair at a time for auto grading with cosine similarity?

I'm developing auto grading essay that compares the similarity between the answer key and student answer with cosine similarity. This one is written in php. Let's say in a course there are 30 - 100 ...
0 votes
1 answer
30 views

In principal component regression, how to show that $X \beta_\mathrm{PCR}=U\: \mathrm{diag}\left\{1,\ldots, 1, 0, \ldots 0\right\}U^Ty$ if $X=USV^T$

Regarding the third equation in this answer, I'm struggling to figure out how $ X \beta_\mathrm{PCR} = U\: \mathrm{diag}\left\{1,\ldots, 1, 0, \ldots 0\right\} U^\top y$. Given $X = USV^T$ where $X$ ...
  • 915
3 votes
0 answers
20 views

Time-Continuity of PCA/SVD [duplicate]

I have a daily set of observations (Jacard similarity matrix) that I want to embed in a much lower dimensional space. Thus I run PCA or SVD for each day and plot the projection on the top 2 dimensions ...
  • 2,126
5 votes
1 answer
114 views

Procrustes Problem for rank-deficient input

It is well known that the solution to the orthogonal Procrustes problem $$ \textrm{arg min}_{\Omega \in \text{SO}(n)} ||Y - \Omega X||^2_2, $$ can be expressed in terms of an SVD of the covariance ...
  • 237
0 votes
0 answers
17 views

Why do we multiply the centered data with eigen vector instead of taking inverse of the eigen vector while performing PCA

The question may sound stupid but I really don't understand the logic behind this. Whenever we do a PCA, we take a covariance matrix on the centered data and do eigen decomposition. In order to ...
1 vote
0 answers
30 views

SVD to find the common factors

Let's say I have a time series $R\in \mathbb{R}^{T\times M}$ where $T$ shows the dates and $M$ is the number of variables. I do an SVD on it and obtain $R=U\Sigma V^\top$, where $U\in \mathbb{R}^{T\...
  • 591
0 votes
1 answer
24 views

Prove that P = A†A is orthogonal projection where A† is pseudo-inverse of A

I am new to this but I was wondering how to prove this. I can reduce A†A to (VΣ†UT)(UΣVT). Would I have to reduce the UT and U to the identity matrix and just continue to simplify? I would get ...
  • 1
0 votes
0 answers
28 views

Does invariance of PCA under orthogonal transformation hold for data that is not centered?

I read the proof in the top answer to this question, but that page assumes that $\overline{A} = 0$. If the data instead has some nonzero mean $\mu$, I'm not sure if the same logic applies: ...
2 votes
0 answers
22 views

PCA on X to capture the most variance of Y

PCA maximises the variation in $X$. Now, suppose $X$ is causal to $Y$. Is there then any analogous way to decompose $X$ into the principal components that cause the most variation in $Y$. For an ...
1 vote
0 answers
20 views

Eckart–Young–Mirsky theorem for $n \gg m$

It has been proven that the best reconstruction error in the $k$ rank matrix estimation problem in terms of Frobenius or $L2$ norm is given by the $k$-truncated SVD as shown here. I've read in ...
1 vote
1 answer
62 views

How to derive the three matrices of SVD from eigenvalue decomposition in Kernel PCA?

Kernel PCA is usually done via eigenvalue decomposition of the Kernel Matrix $\mathbf{K}$ and standard PCA via SVD of the input $\mathbf{X}$. In standard PCA as far as I know we can derive $\mathbf{S}...
  • 23
0 votes
0 answers
84 views

Scaling before PCA & SVD - Iris Dataset [duplicate]

I've seen differing approaches from various resources online, on the necessity of scaling data before doing PCA/SVD. PCA Resource 1: CMU slides Center the means, does not divide by standard devitaion ...
1 vote
0 answers
95 views

How do ‘arpack’, ‘propack’ and ‘lobpcg’ compare to each other?

I want to implement singular value decomposition on a sparse matrix using the Scipy implementation. The documentation shows three solvers: ‘arpack’, ‘propack’ and ‘lobpcg’. How do these solvers ...
  • 311
0 votes
0 answers
25 views

SVD on the item-user interaction matrix

Factor-based recommenders (typically) apply SVD on the user-item interaction matrix (u-i matrix). If we apply SVD on the transpose of the u-i matrix (i-u matrix), will we obtain the same embedding ...
  • 21
1 vote
0 answers
19 views

quasi-PCA reconstruction of the matrix by orthogonal basis

Let's say I have a "data" matrix $X$ of $N$ rows and $p$ cols with $N \gg p$. Now PCA with $L$ components can be formulated as $$ X_L = \underset{Y:rank(Y) = L}{\text{argmin}} ||X- Y||^2_F, ...
0 votes
0 answers
34 views

Is it logical to perform PCA and reduction on observations instead of features?

I am currently working with a set of code called GPMSA that was published by LANL. The code serves to create a Gaussian process model of some simulator and perform regression with experimental data. I ...
0 votes
1 answer
36 views

Rotation-sensitivity of SVD

Suppose I perform a truncated SVD on a symmetric, PSD matrix $A \in R^{N \times d}$ (lowering the dimensionality from $d$ to $k$). Further suppose that there is a rotation matrix $Q$ such that some of ...
0 votes
0 answers
20 views

Are "weights", "loading scores", and "singular values" all synonyms?

I'm currently learning to use "eigenfaces" for facial image classification. Unfortunately, I've encountered some confusion with the following lines of code: ...
  • 631
1 vote
0 answers
118 views

Optimal truncation in SVD

I am working with SVD on a matrix $$Y_{m,n} = T_{m,m} \Sigma D^T_{n,n} $$ where $T$ and $D$ describe the row and the column entities of Y, respectively. The truncated SVD takes the first $r$ ...
1 vote
0 answers
10 views

PCA: understanding how to use loading vectors of X to recapture the geometry of X

If we suppose that X is an n x d matrix, and then perform PCA to obtain loading vectors ...
1 vote
0 answers
40 views

Does PCA centering guarantee projection optimality onto an affine set subject to dimensionality constraint?

Say I have $m$ points in $R^n$, not necessarily with sum 0, and I want to project them onto an affine set $S$ with $\dim(S)=d$ for a given $d$ so as to minimize the sum of the squared distances ...
0 votes
0 answers
31 views

calculate correlation coefficient with singular values of covariance matrix

Given a normal distribution where the covariance matrix $\Sigma$ has known singular values $s_1$, $s_2$, ... $s_m$, what are the Pearson's correlation coefficient values?
0 votes
0 answers
16 views

How to calculate RMSE with missing values except filling them with zeros? [duplicate]

I have a real and predicted matrix of the form np.array and I calculate the RMSE using Sklearn: ...
  • 1
0 votes
0 answers
48 views

Dot product and linear combination of Basis

I am currently going through the SVD intuition provided here. In the section "From intuition to definition", It says that, First, note that any vector $\textbf{x}$ can be described using ...
0 votes
0 answers
84 views

what is the difference between factor analysis and SVD? factanal() vs svd()

I am doing factor analysis. Some sources tell me that I should use factanal() to do (exploratory) factor analysis; my goal is to find common sources of latent ...
  • 93
1 vote
1 answer
311 views

Use K-means clustering on SVD/PCA of data

In an assignment I was suppose to perform K-means clustering on the MNIST dataset (just the 0's and the 1's) and then use SVD/PCA to visualize the data in two dimensions. I missunderstood this and ...
0 votes
0 answers
27 views

Why does reversing PCA from SVD follow this formula?

In this answer the author writes: PCA is very closely related to singular value decomposition (SVD), see Relationship between SVD and PCA. How to use SVD to perform PCA? for more details. If a $n\...
  • 195
0 votes
1 answer
34 views

Practical significance of number of singular values in SVD

I am working on a binary classification problem. SVD is used for dimensionality reduction and the vector with reduced dimension is used as the feature vector. DNN is used as the classifier. There are ...
0 votes
0 answers
165 views

The discrepancy of results of PCA via Eigendecomposition vs via SVD in Python with scipy.linalg [duplicate]

I recently learned about different methods of PCA. I decided to manually implement PCA in Python with Eigendecomposition of cov(X) and the Singular Value ...
  • 175
1 vote
0 answers
15 views

What does the SVD of the numerical derivative matrix of paired instances tell us about our data? [closed]

Let's say we have real-valued random variables $X$ and $Y$, and that under simple random sampling we obtain paired values $\{(x_1,y_1), \cdots, (x_i,y_i), \cdots, (x_m,y_m) \}$. From this sample we ...
  • 5,606
1 vote
0 answers
90 views

How do I get the stationary distribution of a Markov chain matrix from SVD?

I have a matrix that represents a Markov chain. ...
  • 1,337
1 vote
0 answers
85 views

Should you train/test split when using softimpute/matrix completion?

I am trying to impute missing values on a large dataset. After reading the paper(s) introducing matrix completion via soft-SVD thresholding, as well as the softImpute R package vignetter by Hastie (...
  • 11
1 vote
0 answers
54 views

What is the difference between two forms of matrix coherence?

I am stuck on the definition of coherence of a matrix. Let $x_1, \dots, x_p \in \mathbb{R}^p$ be the columns of the matrix $X$, which are assumed to be normalized such that $x'_i x_i = 1$. Wikipedia ...
  • 122
1 vote
1 answer
89 views

Why is the first canonical direction equal to the left singular vector i.e. why is $w_1 = a = u_1$ in CCA (Canonical Correlation Analysis)?

I want to understand why the canonical direction $a$ is equal to the left singular value of $M = \Sigma^{-1/2}_X \Sigma_{X, Y} \Sigma^{-1/2}_Y$ and not $a = \sigma_1 u_1$. My calculation tell me that ...
0 votes
0 answers
169 views

Singular value decomposition for a matrix with missing entries

Suppose $A$ is a matrix consisting of real numbers and nan's. What are some of the robust formulations and the associated algorithms for estimating its singular ...
  • 905
1 vote
0 answers
24 views

Relation between low-rank approximation, nuclear norm of a matrix and Singular Value Decomposition

I'm reading the following paper https://arxiv.org/pdf/2005.10203.pdf which proposes improvements on robustness of large graphs to defend against adversarial attacks that are nothing but slight ...
0 votes
0 answers
56 views

how to understand the implications of the SVD matrix of the covariance $C_{XY} = X Y^T$

Given an $m \times n$ data matrix $X$, the SVD of its covariance matrix $$C = XX^T = ULU^T$$ provides the orthogonal unit vectors that maximize the variance in these directions. In the case of an $m \...
  • 101
3 votes
0 answers
45 views

Do someone understand what the authors mean? - a very strange notation

I'm reading the paper Estimation of (near) low-rank matrices with noise and high-dimensional scaling and came across a very very odd notation. I'll quote the entire passage: Any matrix $\Theta^*\in \...
  • 41
2 votes
1 answer
75 views

Numerically PCA implements SVD or SVD implements PCA

How do we numerically implement SVD? I confused the numerically implementations between PCA and SVD (who implements who). Since we know that PCA can be numerically implemented by ...
  • 1,129
1 vote
0 answers
92 views

What is the PLSR implementation in sklearn?

Having a look to the source code from sklear implementation of PLSRegression I see two differences between what they cite as ...
1 vote
0 answers
151 views

Adding explicit user info to matrix factorization

In the paper Matrix Factorization Techniques for Recommender Systems, it is claimed that we can incorporate extra user information into our recommender model by doing something like this: $$ \hat{r}_{...
0 votes
0 answers
33 views

How can I interpret singular value decomposition analysis?

I am trying to understand singular value decomposition analysis. I compared two gridded atmospheric data. The Mode 1 has 79.5% squared covariance fraction. Modes 2 and 3 have 3% and 2%, respectively. ...
  • 135
1 vote
1 answer
111 views

Understanding the decomposed matrices in Singular Value Decomposition [duplicate]

I am trying to get some intuition regarding the U and V matrices in SVD ($M=UEV^T$). I think these are orthonormal basis vectors, but I am struggling to get an intuition if they represent anything ...
  • 239
1 vote
0 answers
43 views

Determining ML Approach for calculating SVD using neural networks

I am currently working on a project where I need to perform SVD (Singular Value Decomposition) computation on a noisy data using neural networks. It doesn't have to be exact SVD, certain degree of ...
  • 113
0 votes
0 answers
280 views

Singular value decomposition on a polynomial

I'm messing around with the SVD to find a best fit solution. The way I understand (never taking a stat class, only linear alg.) is that that the SVD captures the data variation by its projection onto ...
  • 41
1 vote
0 answers
54 views

Does it make sense to use SVD to do a sort of "lossy compression"?

So - I know if you perform SVD to a matrix $X$, you can then use Echkart Young theorem to get the best rank $r$ approximation $\overline{X}$ to $X$ possible. Since the resultant ${\overline{X}}$ will ...
1 vote
0 answers
54 views

Computational advantage for soft-impute method over other methods

I am reading in the soft-imputing paper for low-rank-based matrix completion. They suggested another solution for $$\hat{Z} = \text{argmin}_Z\lVert X - Z \rVert_F^2 + \lambda \lVert Z \rVert_*$$ ...
  • 291
0 votes
0 answers
29 views

PCA with sample-specific prior information about principal components

Suppose I have some noisy dataset $\mathbf{X} \in \mathbb{R}^{N \times p}$ that I want to perform PCA on. Obtaining the (trimmed) SVD $\mathbf{X} = \mathbf{UDV}$ infers the q-dimensional principal ...
0 votes
1 answer
161 views

Variable Selection : Removing Linear Dependency by SVD using the Condition number and then eliminating the variable causing multicollinearity

I am trying to perform regression with over 5000 feature variables(X) and I would like to eliminate multicollinearity. Incremental VIF computation is expensive. Incremental PCA works but I might lose ...

1
2 3 4 5
8