Questions tagged [svd]

Singular value decomposition (SVD) of a matrix $\mathbf{A}$ is given by $\mathbf{A} = \mathbf{USV}^\top$ where $\mathbf{U}$ and $\mathbf{V}$ are orthogonal matrices and $\mathbf{S}$ is a diagonal matrix.

Filter by
Sorted by
Tagged with
0 votes
0 answers
16 views

Does invariance of PCA under orthogonal transformation hold for data that is not centered?

I read the proof in the top answer to this question, but that page assumes that $\overline{A} = 0$. If the data instead has some nonzero mean $\mu$, I'm not sure if the same logic applies: ...
2 votes
0 answers
17 views

PCA on X to capture the most variance of Y

PCA maximises the variation in $X$. Now, suppose $X$ is causal to $Y$. Is there then any analogous way to decompose $X$ into the principal components that cause the most variation in $Y$. For an ...
1 vote
0 answers
16 views

Eckart–Young–Mirsky theorem for $n \gg m$

It has been proven that the best reconstruction error in the $k$ rank matrix estimation problem in terms of Frobenius or $L2$ norm is given by the $k$-truncated SVD as shown here. I've read in ...
1 vote
1 answer
41 views

How to derive the three matrices of SVD from eigenvalue decomposition in Kernel PCA?

Kernel PCA is usually done via eigenvalue decomposition of the Kernel Matrix $\mathbf{K}$ and standard PCA via SVD of the input $\mathbf{X}$. In standard PCA as far as I know we can derive $\mathbf{S}...
  • 23
0 votes
0 answers
25 views

Scaling before PCA & SVD - Iris Dataset [duplicate]

I've seen differing approaches from various resources online, on the necessity of scaling data before doing PCA/SVD. PCA Resource 1: CMU slides Center the means, does not divide by standard devitaion ...
1 vote
0 answers
34 views

How do ‘arpack’, ‘propack’ and ‘lobpcg’ compare to each other?

I want to implement singular value decomposition on a sparse matrix using the Scipy implementation. The documentation shows three solvers: ‘arpack’, ‘propack’ and ‘lobpcg’. How do these solvers ...
  • 301
0 votes
0 answers
12 views

SVD on the item-user interaction matrix

Factor-based recommenders (typically) apply SVD on the user-item interaction matrix (u-i matrix). If we apply SVD on the transpose of the u-i matrix (i-u matrix), will we obtain the same embedding ...
  • 21
1 vote
0 answers
18 views

quasi-PCA reconstruction of the matrix by orthogonal basis

Let's say I have a "data" matrix $X$ of $N$ rows and $p$ cols with $N \gg p$. Now PCA with $L$ components can be formulated as $$ X_L = \underset{Y:rank(Y) = L}{\text{argmin}} ||X- Y||^2_F, ...
0 votes
0 answers
32 views

Is it logical to perform PCA and reduction on observations instead of features?

I am currently working with a set of code called GPMSA that was published by LANL. The code serves to create a Gaussian process model of some simulator and perform regression with experimental data. I ...
0 votes
1 answer
29 views

Rotation-sensitivity of SVD

Suppose I perform a truncated SVD on a symmetric, PSD matrix $A \in R^{N \times d}$ (lowering the dimensionality from $d$ to $k$). Further suppose that there is a rotation matrix $Q$ such that some of ...
0 votes
0 answers
15 views

Are "weights", "loading scores", and "singular values" all synonyms?

I'm currently learning to use "eigenfaces" for facial image classification. Unfortunately, I've encountered some confusion with the following lines of code: ...
  • 211
1 vote
0 answers
66 views

Optimal truncation in SVD

I am working with SVD on a matrix $$Y_{m,n} = T_{m,m} \Sigma D^T_{n,n} $$ where $T$ and $D$ describe the row and the column entities of Y, respectively. The truncated SVD takes the first $r$ ...
1 vote
0 answers
10 views

PCA: understanding how to use loading vectors of X to recapture the geometry of X

If we suppose that X is an n x d matrix, and then perform PCA to obtain loading vectors ...
1 vote
0 answers
27 views

Does PCA centering guarantee projection optimality onto an affine set subject to dimensionality constraint?

Say I have $m$ points in $R^n$, not necessarily with sum 0, and I want to project them onto an affine set $S$ with $\dim(S)=d$ for a given $d$ so as to minimize the sum of the squared distances ...
0 votes
0 answers
26 views

calculate correlation coefficient with singular values of covariance matrix

Given a normal distribution where the covariance matrix $\Sigma$ has known singular values $s_1$, $s_2$, ... $s_m$, what are the Pearson's correlation coefficient values?
0 votes
0 answers
16 views

How to calculate RMSE with missing values except filling them with zeros? [duplicate]

I have a real and predicted matrix of the form np.array and I calculate the RMSE using Sklearn: ...
  • 1
0 votes
0 answers
36 views

Dot product and linear combination of Basis

I am currently going through the SVD intuition provided here. In the section "From intuition to definition", It says that, First, note that any vector $\textbf{x}$ can be described using ...
0 votes
0 answers
62 views

what is the difference between factor analysis and SVD? factanal() vs svd()

I am doing factor analysis. Some sources tell me that I should use factanal() to do (exploratory) factor analysis; my goal is to find common sources of latent ...
  • 93
1 vote
1 answer
194 views

Use K-means clustering on SVD/PCA of data

In an assignment I was suppose to perform K-means clustering on the MNIST dataset (just the 0's and the 1's) and then use SVD/PCA to visualize the data in two dimensions. I missunderstood this and ...
0 votes
0 answers
25 views

Why does reversing PCA from SVD follow this formula?

In this answer the author writes: PCA is very closely related to singular value decomposition (SVD), see Relationship between SVD and PCA. How to use SVD to perform PCA? for more details. If a $n\...
  • 133
0 votes
1 answer
31 views

Practical significance of number of singular values in SVD

I am working on a binary classification problem. SVD is used for dimensionality reduction and the vector with reduced dimension is used as the feature vector. DNN is used as the classifier. There are ...
0 votes
0 answers
94 views

The discrepancy of results of PCA via Eigendecomposition vs via SVD in Python with scipy.linalg [duplicate]

I recently learned about different methods of PCA. I decided to manually implement PCA in Python with Eigendecomposition of cov(X) and the Singular Value ...
  • 165
1 vote
0 answers
14 views

What does the SVD of the numerical derivative matrix of paired instances tell us about our data? [closed]

Let's say we have real-valued random variables $X$ and $Y$, and that under simple random sampling we obtain paired values $\{(x_1,y_1), \cdots, (x_i,y_i), \cdots, (x_m,y_m) \}$. From this sample we ...
  • 4,560
0 votes
0 answers
14 views

Proof of SVD generates equal eigenvectors as PCA [duplicate]

Figure link: https://people.cs.pitt.edu/~milos/courses/cs3750-Fall2014/lectures/class9.pdf In process of PCA, we either decompose covariance matrix, or do SVD on X. $$ C = \frac{1}{n-1} X^T X = \frac{...
  • 1
0 votes
0 answers
69 views

By multiplying data by eigenvectors of PCA, do we actually recover part of the left singular vector?

In software like sklearn or matlab, often we project data through first $r$ principal components of PCA. For example if we have column data matrix $X$ (mean centered) and principal component $v_r$, to ...
1 vote
0 answers
66 views

How do I get the stationary distribution of a Markov chain matrix from SVD?

I have a matrix that represents a Markov chain. ...
  • 1,327
1 vote
0 answers
66 views

Should you train/test split when using softimpute/matrix completion?

I am trying to impute missing values on a large dataset. After reading the paper(s) introducing matrix completion via soft-SVD thresholding, as well as the softImpute R package vignetter by Hastie (...
  • 11
0 votes
0 answers
27 views

Adding New Regularization Terms to Closed Form LSE

I am trying to align the input space X to the output space Y by using least squares method in a closed form solution. To do that I use svd for finding the rotation matrix (W). And I find a solution ...
1 vote
0 answers
44 views

What is the difference between two forms of matrix coherence?

I am stuck on the definition of coherence of a matrix. Let $x_1, \dots, x_p \in \mathbb{R}^p$ be the columns of the matrix $X$, which are assumed to be normalized such that $x'_i x_i = 1$. Wikipedia ...
  • 122
0 votes
0 answers
20 views

What is the principal axis means in PCA? [duplicate]

I want to know the main trend of data by PCA method. There was a question that explained what the PCA method works. In this page. And there was also a question that told how to implement this method. ...
1 vote
1 answer
72 views

Why is the first canonical direction equal to the left singular vector i.e. why is $w_1 = a = u_1$ in CCA (Canonical Correlation Analysis)?

I want to understand why the canonical direction $a$ is equal to the left singular value of $M = \Sigma^{-1/2}_X \Sigma_{X, Y} \Sigma^{-1/2}_Y$ and not $a = \sigma_1 u_1$. My calculation tell me that ...
0 votes
0 answers
112 views

Singular value decomposition for a matrix with missing entries

Suppose $A$ is a matrix consisting of real numbers and nan's. What are some of the robust formulations and the associated algorithms for estimating its singular ...
  • 905
1 vote
0 answers
23 views

Relation between low-rank approximation, nuclear norm of a matrix and Singular Value Decomposition

I'm reading the following paper https://arxiv.org/pdf/2005.10203.pdf which proposes improvements on robustness of large graphs to defend against adversarial attacks that are nothing but slight ...
0 votes
0 answers
48 views

how to understand the implications of the SVD matrix of the covariance $C_{XY} = X Y^T$

Given an $m \times n$ data matrix $X$, the SVD of its covariance matrix $$C = XX^T = ULU^T$$ provides the orthogonal unit vectors that maximize the variance in these directions. In the case of an $m \...
3 votes
0 answers
44 views

Do someone understand what the authors mean? - a very strange notation

I'm reading the paper Estimation of (near) low-rank matrices with noise and high-dimensional scaling and came across a very very odd notation. I'll quote the entire passage: Any matrix $\Theta^*\in \...
  • 41
2 votes
1 answer
67 views

Numerically PCA implements SVD or SVD implements PCA

How do we numerically implement SVD? I confused the numerically implementations between PCA and SVD (who implements who). Since we know that PCA can be numerically implemented by ...
  • 1,117
1 vote
0 answers
83 views

What is the PLSR implementation in sklearn?

Having a look to the source code from sklear implementation of PLSRegression I see two differences between what they cite as ...
1 vote
0 answers
134 views

Adding explicit user info to matrix factorization

In the paper Matrix Factorization Techniques for Recommender Systems, it is claimed that we can incorporate extra user information into our recommender model by doing something like this: $$ \hat{r}_{...
0 votes
0 answers
31 views

How can I interpret singular value decomposition analysis?

I am trying to understand singular value decomposition analysis. I compared two gridded atmospheric data. The Mode 1 has 79.5% squared covariance fraction. Modes 2 and 3 have 3% and 2%, respectively. ...
  • 135
1 vote
1 answer
83 views

Understanding the decomposed matrices in Singular Value Decomposition [duplicate]

I am trying to get some intuition regarding the U and V matrices in SVD ($M=UEV^T$). I think these are orthonormal basis vectors, but I am struggling to get an intuition if they represent anything ...
  • 209
1 vote
0 answers
40 views

Determining ML Approach for calculating SVD using neural networks

I am currently working on a project where I need to perform SVD (Singular Value Decomposition) computation on a noisy data using neural networks. It doesn't have to be exact SVD, certain degree of ...
  • 113
0 votes
0 answers
200 views

Singular value decomposition on a polynomial

I'm messing around with the SVD to find a best fit solution. The way I understand (never taking a stat class, only linear alg.) is that that the SVD captures the data variation by its projection onto ...
  • 41
1 vote
0 answers
38 views

Does it make sense to use SVD to do a sort of "lossy compression"?

So - I know if you perform SVD to a matrix $X$, you can then use Echkart Young theorem to get the best rank $r$ approximation $\overline{X}$ to $X$ possible. Since the resultant ${\overline{X}}$ will ...
1 vote
0 answers
45 views

Computational advantage for soft-impute method over other methods

I am reading in the soft-imputing paper for low-rank-based matrix completion. They suggested another solution for $$\hat{Z} = \text{argmin}_Z\lVert X - Z \rVert_F^2 + \lambda \lVert Z \rVert_*$$ ...
  • 261
0 votes
0 answers
24 views

PCA with sample-specific prior information about principal components

Suppose I have some noisy dataset $\mathbf{X} \in \mathbb{R}^{N \times p}$ that I want to perform PCA on. Obtaining the (trimmed) SVD $\mathbf{X} = \mathbf{UDV}$ infers the q-dimensional principal ...
0 votes
1 answer
136 views

Variable Selection : Removing Linear Dependency by SVD using the Condition number and then eliminating the variable causing multicollinearity

I am trying to perform regression with over 5000 feature variables(X) and I would like to eliminate multicollinearity. Incremental VIF computation is expensive. Incremental PCA works but I might lose ...
1 vote
0 answers
159 views

Why are the directions of eigenvectors in SVD and Eigen-Decomposition for PCA opposite? [duplicate]

As you may know, scikit-learn library utilizes singular value decomposition (SVD) of data matrix X to produce eigenvectors for PCA. I decided to code PCA by using ...
  • 517
0 votes
0 answers
71 views

Principal Component analysis - SVD U matrix projection

I have a bit mathematical question I am interested in. Principal component analysis (PCA) has mathematically multiple solutions. One way is to use SVD. I have prepared an example bellow. I am curious ...
  • 131
1 vote
0 answers
41 views

Fastest way to find Leading singular value and vector (power iteration, rsvd etc)

I want to know the fastest way to find out the leading singular value and vector of a large rectangular matrix. I have seen 2 suggestions and have questions on both of them : Power Method : For this ...
2 votes
1 answer
83 views

Largest singular values

Given the positive semi-definite, symmetric matrix $A = bb^T + \sigma^2I$ where b is a column vector is it possible to find the singular values and singular vectors of the matrix analytically? I know ...
's user avatar

1
2 3 4 5
8