Questions tagged [svd]

Singular value decomposition (SVD) of a matrix $\mathbf{A}$ is given by $\mathbf{A} = \mathbf{USV}^\top$ where $\mathbf{U}$ and $\mathbf{V}$ are orthogonal matrices and $\mathbf{S}$ is a diagonal matrix.

Filter by
Sorted by
Tagged with
1 vote
0 answers
23 views

Optimal truncation in SVD

I am working with SVD on a matrix $$Y_{m,n} = T_{m,m} \Sigma D^T_{n,n} $$ where $T$ and $D$ describe the row and the column entities of Y, respectively. The truncated SVD takes the first $r$ ...
user avatar
1 vote
0 answers
9 views

PCA: understanding how to use loading vectors of X to recapture the geometry of X

If we suppose that X is an n x d matrix, and then perform PCA to obtain loading vectors ...
user avatar
1 vote
0 answers
18 views

Does PCA centering guarantee projection optimality onto an affine set subject to dimensionality constraint?

Say I have $m$ points in $R^n$, not necessarily with sum 0, and I want to project them onto an affine set $S$ with $\dim(S)=d$ for a given $d$ so as to minimize the sum of the squared distances ...
user avatar
0 votes
0 answers
14 views

calculate correlation coefficient with singular values of covariance matrix

Given a normal distribution where the covariance matrix $\Sigma$ has known singular values $s_1$, $s_2$, ... $s_m$, what are the Pearson's correlation coefficient values?
user avatar
0 votes
0 answers
15 views

How to calculate RMSE with missing values except filling them with zeros? [duplicate]

I have a real and predicted matrix of the form np.array and I calculate the RMSE using Sklearn: ...
user avatar
  • 1
0 votes
0 answers
20 views

Dot product and linear combination of Basis

I am currently going through the SVD intuition provided here. In the section "From intuition to definition", It says that, First, note that any vector $\textbf{x}$ can be described using ...
user avatar
0 votes
0 answers
40 views

what is the difference between factor analysis and SVD? factanal() vs svd()

I am doing factor analysis. Some sources tell me that I should use factanal() to do (exploratory) factor analysis; my goal is to find common sources of latent ...
user avatar
  • 93
1 vote
1 answer
73 views

Use K-means clustering on SVD/PCA of data

In an assignment I was suppose to perform K-means clustering on the MNIST dataset (just the 0's and the 1's) and then use SVD/PCA to visualize the data in two dimensions. I missunderstood this and ...
user avatar
0 votes
0 answers
25 views

Why does reversing PCA from SVD follow this formula?

In this answer the author writes: PCA is very closely related to singular value decomposition (SVD), see Relationship between SVD and PCA. How to use SVD to perform PCA? for more details. If a $n\...
user avatar
  • 111
0 votes
1 answer
27 views

Practical significance of number of singular values in SVD

I am working on a binary classification problem. SVD is used for dimensionality reduction and the vector with reduced dimension is used as the feature vector. DNN is used as the classifier. There are ...
user avatar
0 votes
0 answers
50 views

The discrepancy of results of PCA via Eigendecomposition vs via SVD in Python with scipy.linalg [duplicate]

I recently learned about different methods of PCA. I decided to manually implement PCA in Python with Eigendecomposition of cov(X) and the Singular Value ...
user avatar
  • 155
1 vote
0 answers
13 views

What does the SVD of the numerical derivative matrix of paired instances tell us about our data? [closed]

Let's say we have real-valued random variables $X$ and $Y$, and that under simple random sampling we obtain paired values $\{(x_1,y_1), \cdots, (x_i,y_i), \cdots, (x_m,y_m) \}$. From this sample we ...
user avatar
  • 3,403
0 votes
0 answers
14 views

Proof of SVD generates equal eigenvectors as PCA [duplicate]

Figure link: https://people.cs.pitt.edu/~milos/courses/cs3750-Fall2014/lectures/class9.pdf In process of PCA, we either decompose covariance matrix, or do SVD on X. $$ C = \frac{1}{n-1} X^T X = \frac{...
user avatar
  • 1
0 votes
0 answers
27 views

By multiplying data by eigenvectors of PCA, do we actually recover part of the left singular vector?

In software like sklearn or matlab, often we project data through first $r$ principal components of PCA. For example if we have column data matrix $X$ (mean centered) and principal component $v_r$, to ...
user avatar
1 vote
0 answers
50 views

How do I get the stationary distribution of a Markov chain matrix from SVD?

I have a matrix that represents a Markov chain. ...
user avatar
  • 1,298
1 vote
0 answers
47 views

Should you train/test split when using softimpute/matrix completion?

I am trying to impute missing values on a large dataset. After reading the paper(s) introducing matrix completion via soft-SVD thresholding, as well as the softImpute R package vignetter by Hastie (...
user avatar
  • 11
0 votes
0 answers
17 views

Adding New Regularization Terms to Closed Form LSE

I am trying to align the input space X to the output space Y by using least squares method in a closed form solution. To do that I use svd for finding the rotation matrix (W). And I find a solution ...
user avatar
1 vote
0 answers
38 views

What is the difference between two forms of matrix coherence?

I am stuck on the definition of coherence of a matrix. Let $x_1, \dots, x_p \in \mathbb{R}^p$ be the columns of the matrix $X$, which are assumed to be normalized such that $x'_i x_i = 1$. Wikipedia ...
user avatar
  • 112
0 votes
0 answers
20 views

What is the principal axis means in PCA? [duplicate]

I want to know the main trend of data by PCA method. There was a question that explained what the PCA method works. In this page. And there was also a question that told how to implement this method. ...
user avatar
1 vote
1 answer
66 views

Why is the first canonical direction equal to the left singular vector i.e. why is $w_1 = a = u_1$ in CCA (Canonical Correlation Analysis)?

I want to understand why the canonical direction $a$ is equal to the left singular value of $M = \Sigma^{-1/2}_X \Sigma_{X, Y} \Sigma^{-1/2}_Y$ and not $a = \sigma_1 u_1$. My calculation tell me that ...
user avatar
0 votes
0 answers
74 views

Singular value decomposition for a matrix with missing entries

Suppose $A$ is a matrix consisting of real numbers and nan's. What are some of the robust formulations and the associated algorithms for estimating its singular ...
user avatar
  • 885
1 vote
0 answers
20 views

Relation between low-rank approximation, nuclear norm of a matrix and Singular Value Decomposition

I'm reading the following paper https://arxiv.org/pdf/2005.10203.pdf which proposes improvements on robustness of large graphs to defend against adversarial attacks that are nothing but slight ...
user avatar
0 votes
0 answers
40 views

how to understand the implications of the SVD matrix of the covariance $C_{XY} = X Y^T$

Given an $m \times n$ data matrix $X$, the SVD of its covariance matrix $$C = XX^T = ULU^T$$ provides the orthogonal unit vectors that maximize the variance in these directions. In the case of an $m \...
user avatar
0 votes
0 answers
6 views

Correspondence Analysis GSVD (generalised single value decomposition) proof

I'm not able to found a simple proof or just a normal detailed proof of: \begin{aligned} \mathcal{X}^{2} &=n \text { trace }\left(\left(\mathbf{F}-\mathbf{r c}^{\prime}\right)^{\prime} \mathbf{D}_{...
user avatar
0 votes
0 answers
28 views

SVD on demeaned matrix

I'm trying to understand the effect of de-meaning with SVD. Suppose I have matrix $WW^T = \sum_{i=1}^n w_iw_i^T$ where $W$ is $n \times m$ and $w_i$ are its columns. Running SVD on this yields ...
user avatar
  • 141
0 votes
0 answers
18 views

Linear auto-encoders and PCA with unequal input-output

It is a well-known fact that linear auto-encoders are equivalent to PCA, i.e. for the data matrx $X\in {\mathbb R}^{n\times N}$ the task $$ \min_{W\in {\mathbb R}^{n\times k}}||X-WW^TX|| $$ has a ...
user avatar
  • 101
0 votes
0 answers
354 views

Getting the slope of the first principal component from PCA

"Given an Nx2 dataset get the first principle component then it's slope."I'm working through how to get the slope of the first principal component in PCA using Numpy ...
user avatar
0 votes
0 answers
20 views

Purpose of SVD in Rank-Deficient Matrices

Suppose $A\in\mathbb{R}^{m\times n}$ with $m>n$. Solving the least square problem is equivalent to solving the normalized system $A^{T}Ax=A^{T}b$ and a unique solution exists if $\operatorname{rank}...
user avatar
0 votes
0 answers
25 views

Relation between first left-singular vector and row-wise mean

I am working on some imaging data: let $X$ be a matrix of size $t\times v$ ($v$ is the number of pixels of my images, $t$ a time dimension), centered and scaled to unit variance. I am interested in ...
user avatar
  • 161
3 votes
0 answers
44 views

Do someone understand what the authors mean? - a very strange notation

I'm reading the paper Estimation of (near) low-rank matrices with noise and high-dimensional scaling and came across a very very odd notation. I'll quote the entire passage: Any matrix $\Theta^*\in \...
user avatar
  • 41
0 votes
0 answers
22 views

Using Singular Value Decomposition to Compute Variance Covariance Matrix where n < p

The problem is the same as the following:- Using Singular Value Decomposition to Compute Variance Covariance Matrix from linear regression model In R, this works. ...
user avatar
2 votes
1 answer
64 views

Numerically PCA implements SVD or SVD implements PCA

How do we numerically implement SVD? I confused the numerically implementations between PCA and SVD (who implements who). Since we know that PCA can be numerically implemented by ...
user avatar
1 vote
0 answers
71 views

What is the PLSR implementation in sklearn?

Having a look to the source code from sklear implementation of PLSRegression I see two differences between what they cite as ...
user avatar
1 vote
0 answers
120 views

Adding explicit user info to matrix factorization

In the paper Matrix Factorization Techniques for Recommender Systems, it is claimed that we can incorporate extra user information into our recommender model by doing something like this: $$ \hat{r}_{...
user avatar
0 votes
0 answers
26 views

How can I interpret singular value decomposition analysis?

I am trying to understand singular value decomposition analysis. I compared two gridded atmospheric data. The Mode 1 has 79.5% squared covariance fraction. Modes 2 and 3 have 3% and 2%, respectively. ...
user avatar
  • 135
1 vote
1 answer
60 views

Understanding the decomposed matrices in Singular Value Decomposition [duplicate]

I am trying to get some intuition regarding the U and V matrices in SVD ($M=UEV^T$). I think these are orthonormal basis vectors, but I am struggling to get an intuition if they represent anything ...
user avatar
  • 209
1 vote
0 answers
39 views

Determining ML Approach for calculating SVD using neural networks

I am currently working on a project where I need to perform SVD (Singular Value Decomposition) computation on a noisy data using neural networks. It doesn't have to be exact SVD, certain degree of ...
user avatar
  • 113
0 votes
0 answers
147 views

Singular value decomposition on a polynomial

I'm messing around with the SVD to find a best fit solution. The way I understand (never taking a stat class, only linear alg.) is that that the SVD captures the data variation by its projection onto ...
user avatar
  • 41
1 vote
0 answers
35 views

Does it make sense to use SVD to do a sort of "lossy compression"?

So - I know if you perform SVD to a matrix $X$, you can then use Echkart Young theorem to get the best rank $r$ approximation $\overline{X}$ to $X$ possible. Since the resultant ${\overline{X}}$ will ...
user avatar
1 vote
0 answers
42 views

Computational advantage for soft-impute method over other methods

I am reading in the soft-imputing paper for low-rank-based matrix completion. They suggested another solution for $$\hat{Z} = \text{argmin}_Z\lVert X - Z \rVert_F^2 + \lambda \lVert Z \rVert_*$$ ...
user avatar
  • 231
0 votes
0 answers
20 views

PCA with sample-specific prior information about principal components

Suppose I have some noisy dataset $\mathbf{X} \in \mathbb{R}^{N \times p}$ that I want to perform PCA on. Obtaining the (trimmed) SVD $\mathbf{X} = \mathbf{UDV}$ infers the q-dimensional principal ...
user avatar
0 votes
1 answer
99 views

Variable Selection : Removing Linear Dependency by SVD using the Condition number and then eliminating the variable causing multicollinearity

I am trying to perform regression with over 5000 feature variables(X) and I would like to eliminate multicollinearity. Incremental VIF computation is expensive. Incremental PCA works but I might lose ...
user avatar
1 vote
0 answers
109 views

Why are the directions of eigenvectors in SVD and Eigen-Decomposition for PCA opposite? [duplicate]

As you may know, scikit-learn library utilizes singular value decomposition (SVD) of data matrix X to produce eigenvectors for PCA. I decided to code PCA by using ...
user avatar
  • 517
0 votes
0 answers
60 views

Principal Component analysis - SVD U matrix projection

I have a bit mathematical question I am interested in. Principal component analysis (PCA) has mathematically multiple solutions. One way is to use SVD. I have prepared an example bellow. I am curious ...
user avatar
  • 131
1 vote
0 answers
40 views

Fastest way to find Leading singular value and vector (power iteration, rsvd etc)

I want to know the fastest way to find out the leading singular value and vector of a large rectangular matrix. I have seen 2 suggestions and have questions on both of them : Power Method : For this ...
user avatar
2 votes
1 answer
69 views

Largest singular values

Given the positive semi-definite, symmetric matrix $A = bb^T + \sigma^2I$ where b is a column vector is it possible to find the singular values and singular vectors of the matrix analytically? I know ...
user avatar
3 votes
1 answer
38 views

Using SVD to write the least squares fitted vector

Elements of statistics p.66 Please I know the least squares solution for $\hat\beta = (X^TX)^{-1}X^Ty$ but I don't know how they were able to get $X\hat\beta= X(X^TX)^{-1}X^Ty = UU^Ty$ These are the ...
user avatar
  • 299
1 vote
1 answer
47 views

How to compute the left singular eigenvector matrix (U) from the output of prcomp() for PCA in R?

I am examining the output of the prcomp() function in R for PCA in light of the singular value decomposition equation: $X = U \cdot \Sigma \cdot V^{T}$, where: $X$: ...
user avatar
  • 1,065
2 votes
0 answers
112 views

Singular value decomposition used for dimensionality reduction in brain signal topographic data

I am trying to replicate the localizer method described in this paper (page 4). I am stuck on a step which I don't completely understand, and I would like your input and interpretation to progress. ...
user avatar
  • 121
4 votes
0 answers
127 views

Physical interpretation of $U$ and $V$ matrices in SVD

I have a question about the physical interpretation of $U$ and $V$ matrices in SVD. I collect measurements at multiple devices across time are collected into an $m$ × $T$ matrix $M$, where m is the ...
user avatar
  • 51

1
2 3 4 5
8