Questions tagged [svd]

Singular value decomposition (SVD) of a matrix $\mathbf{A}$ is given by $\mathbf{A} = \mathbf{USV}^\top$ where $\mathbf{U}$ and $\mathbf{V}$ are orthogonal matrices and $\mathbf{S}$ is a diagonal matrix.

Filter by
Sorted by
Tagged with
0
votes
0answers
8 views

Adding New Regularization Terms to Closed Form LSE

I am trying to align the input space X to the output space Y by using least squares method in a closed form solution. To do that I use svd for finding the rotation matrix (W). And I find a solution ...
1
vote
0answers
32 views

What is the difference between two forms of matrix coherence?

I am stuck on the definition of coherence of a matrix. Let $x_1, \dots, x_p \in \mathbb{R}^p$ be the columns of the matrix $X$, which are assumed to be normalized such that $x'_i x_i = 1$. Wikipedia ...
2
votes
0answers
27 views

How to use SVD to perform PCA for complex data matrix?

Most researches I have learnt are using SVD to perform PCA for real data mtrix. If the data is complex, will the corresponding solution be different? How to use SVD to perform PCA for complex data ...
0
votes
0answers
19 views

What is the principal axis means in PCA? [duplicate]

I want to know the main trend of data by PCA method. There was a question that explained what the PCA method works. In this page. And there was also a question that told how to implement this method. ...
1
vote
1answer
61 views

Why is the first canonical direction equal to the left singular vector i.e. why is $w_1 = a = u_1$ in CCA (Canonical Correlation Analysis)?

I want to understand why the canonical direction $a$ is equal to the left singular value of $M = \Sigma^{-1/2}_X \Sigma_{X, Y} \Sigma^{-1/2}_Y$ and not $a = \sigma_1 u_1$. My calculation tell me that ...
0
votes
0answers
50 views

Singular value decomposition for a matrix with missing entries

Suppose $A$ is a matrix consisting of real numbers and nan's. What are some of the robust formulations and the associated algorithms for estimating its singular ...
1
vote
0answers
13 views

Relation between low-rank approximation, nuclear norm of a matrix and Singular Value Decomposition

I'm reading the following paper https://arxiv.org/pdf/2005.10203.pdf which proposes improvements on robustness of large graphs to defend against adversarial attacks that are nothing but slight ...
0
votes
0answers
31 views

how to understand the implications of the SVD matrix of the covariance $C_{XY} = X Y^T$

Given an $m \times n$ data matrix $X$, the SVD of its covariance matrix $$C = XX^T = ULU^T$$ provides the orthogonal unit vectors that maximize the variance in these directions. In the case of an $m \...
0
votes
0answers
6 views

Correspondence Analysis GSVD (generalised single value decomposition) proof

I'm not able to found a simple proof or just a normal detailed proof of: \begin{aligned} \mathcal{X}^{2} &=n \text { trace }\left(\left(\mathbf{F}-\mathbf{r c}^{\prime}\right)^{\prime} \mathbf{D}_{...
0
votes
0answers
21 views

SVD on demeaned matrix

I'm trying to understand the effect of de-meaning with SVD. Suppose I have matrix $WW^T = \sum_{i=1}^n w_iw_i^T$ where $W$ is $n \times m$ and $w_i$ are its columns. Running SVD on this yields ...
0
votes
0answers
17 views

Linear auto-encoders and PCA with unequal input-output

It is a well-known fact that linear auto-encoders are equivalent to PCA, i.e. for the data matrx $X\in {\mathbb R}^{n\times N}$ the task $$ \min_{W\in {\mathbb R}^{n\times k}}||X-WW^TX|| $$ has a ...
0
votes
0answers
79 views

Getting the slope of the first principal component from PCA

"Given an Nx2 dataset get the first principle component then it's slope."I'm working through how to get the slope of the first principal component in PCA using Numpy ...
0
votes
0answers
18 views

Purpose of SVD in Rank-Deficient Matrices

Suppose $A\in\mathbb{R}^{m\times n}$ with $m>n$. Solving the least square problem is equivalent to solving the normalized system $A^{T}Ax=A^{T}b$ and a unique solution exists if $\operatorname{rank}...
0
votes
0answers
20 views

Relation between first left-singular vector and row-wise mean

I am working on some imaging data: let $X$ be a matrix of size $t\times v$ ($v$ is the number of pixels of my images, $t$ a time dimension), centered and scaled to unit variance. I am interested in ...
3
votes
0answers
41 views

Do someone understand what the authors mean? - a very strange notation

I'm reading the paper Estimation of (near) low-rank matrices with noise and high-dimensional scaling and came across a very very odd notation. I'll quote the entire passage: Any matrix $\Theta^*\in \...
0
votes
0answers
19 views

Using Singular Value Decomposition to Compute Variance Covariance Matrix where n < p

The problem is the same as the following:- Using Singular Value Decomposition to Compute Variance Covariance Matrix from linear regression model In R, this works. ...
1
vote
1answer
54 views

Numerically PCA implements SVD or SVD implements PCA

How do we numerically implement SVD? I confused the numerically implementations between PCA and SVD (who implements who). Since we know that PCA can be numerically implemented by ...
1
vote
0answers
49 views

What is the PLSR implementation in sklearn?

Having a look to the source code from sklear implementation of PLSRegression I see two differences between what they cite as ...
0
votes
0answers
7 views

How to standardize semi-exponential data before applying SVD for PCA?

I am working with a set of semi-exponential decays measured during an experiment. Each decay consists on 20 values registered over time and my dataset consists on 250 decays. Some of them present ...
1
vote
0answers
119 views

Adding explicit user info to matrix factorization

In the paper Matrix Factorization Techniques for Recommender Systems, it is claimed that we can incorporate extra user information into our recommender model by doing something like this: $$ \hat{r}_{...
0
votes
0answers
25 views

How can I interpret singular value decomposition analysis?

I am trying to understand singular value decomposition analysis. I compared two gridded atmospheric data. The Mode 1 has 79.5% squared covariance fraction. Modes 2 and 3 have 3% and 2%, respectively. ...
1
vote
1answer
35 views

Understanding the decomposed matrices in Singular Value Decomposition [duplicate]

I am trying to get some intuition regarding the U and V matrices in SVD ($M=UEV^T$). I think these are orthonormal basis vectors, but I am struggling to get an intuition if they represent anything ...
1
vote
0answers
37 views

Determining ML Approach for calculating SVD using neural networks

I am currently working on a project where I need to perform SVD (Singular Value Decomposition) computation on a noisy data using neural networks. It doesn't have to be exact SVD, certain degree of ...
0
votes
0answers
26 views

PCA and SVD not effective at capturing variance in fewer dimensions. Any good alternatives?

I have four columns, which I have z-scaled and tried PCA and SVD on, hoping that I can obtain one dimension that explains the majority of the variance. However, with PCA there is approximately 25% ...
0
votes
0answers
14 views

SVD - vectors in matrix A

In SVD we have $A = U \Sigma V^T$. When applying it for ML, e.g. to calculate Moore-Penrose pseudoinverse for linear regression, I have seen that we take columns of $A$ as vectors. Typically in ML I ...
0
votes
0answers
13 views

How to fit a constrained harmonic fit in R to mean daily temperature data?

I am trying to reproduce the statistics procedures for creating daily normal temperatures according to this paper. I was provided code in IDL (a Fortran based scientific language), but it is so ...
0
votes
0answers
74 views

Singular value decomposition on a polynomial

I'm messing around with the SVD to find a best fit solution. The way I understand (never taking a stat class, only linear alg.) is that that the SVD captures the data variation by its projection onto ...
1
vote
0answers
32 views

Does it make sense to use SVD to do a sort of "lossy compression"?

So - I know if you perform SVD to a matrix $X$, you can then use Echkart Young theorem to get the best rank $r$ approximation $\overline{X}$ to $X$ possible. Since the resultant ${\overline{X}}$ will ...
0
votes
0answers
31 views

Correct use of S.V.D. regression on my data

All I(barley) know is electrical engineering. Only ever taken one basic stat class and a linear alg. class. So please use layman terms. I'm working on a wireless power transfer system. (Like those ...
1
vote
0answers
39 views

Computational advantage for soft-impute method over other methods

I am reading in the soft-imputing paper for low-rank-based matrix completion. They suggested another solution for $$\hat{Z} = \text{argmin}_Z\lVert X - Z \rVert_F^2 + \lambda \lVert Z \rVert_*$$ ...
0
votes
0answers
19 views

PCA with sample-specific prior information about principal components

Suppose I have some noisy dataset $\mathbf{X} \in \mathbb{R}^{N \times p}$ that I want to perform PCA on. Obtaining the (trimmed) SVD $\mathbf{X} = \mathbf{UDV}$ infers the q-dimensional principal ...
0
votes
0answers
8 views

Physical significance of non-negative factors of a matrix?

I was trying to make a recommender system using matrix factorization techniques on rating data. I came across 2 algorithms - SVD and NMF. While the basic difference is very clear , I was wondering ...
0
votes
1answer
56 views

Variable Selection : Removing Linear Dependency by SVD using the Condition number and then eliminating the variable causing multicollinearity

I am trying to perform regression with over 5000 feature variables(X) and I would like to eliminate multicollinearity. Incremental VIF computation is expensive. Incremental PCA works but I might lose ...
1
vote
0answers
55 views

Why are the directions of eigenvectors in SVD and Eigen-Decomposition for PCA opposite? [duplicate]

As you may know, scikit-learn library utilizes singular value decomposition (SVD) of data matrix X to produce eigenvectors for PCA. I decided to code PCA by using ...
0
votes
0answers
50 views

Principal Component analysis - SVD U matrix projection

I have a bit mathematical question I am interested in. Principal component analysis (PCA) has mathematically multiple solutions. One way is to use SVD. I have prepared an example bellow. I am curious ...
1
vote
0answers
33 views

Fastest way to find Leading singular value and vector (power iteration, rsvd etc)

I want to know the fastest way to find out the leading singular value and vector of a large rectangular matrix. I have seen 2 suggestions and have questions on both of them : Power Method : For this ...
2
votes
1answer
51 views

Largest singular values

Given the positive semi-definite, symmetric matrix $A = bb^T + \sigma^2I$ where b is a column vector is it possible to find the singular values and singular vectors of the matrix analytically? I know ...
2
votes
1answer
30 views

Using SVD to write the least squares fitted vector

Elements of statistics p.66 Please I know the least squares solution for $\hat\beta = (X^TX)^{-1}X^Ty$ but I don't know how they were able to get $X\hat\beta= X(X^TX)^{-1}X^Ty = UU^Ty$ These are the ...
1
vote
1answer
35 views

How to compute the left singular eigenvector matrix (U) from the output of prcomp() for PCA in R?

I am examining the output of the prcomp() function in R for PCA in light of the singular value decomposition equation: $X = U \cdot \Sigma \cdot V^{T}$, where: $X$: ...
2
votes
0answers
44 views

Singular value decomposition used for dimensionality reduction in brain signal topographic data

I am trying to replicate the localizer method described in this paper (page 4). I am stuck on a step which I don't completely understand, and I would like your input and interpretation to progress. ...
4
votes
0answers
103 views

Physical interpretation of $U$ and $V$ matrices in SVD

I have a question about the physical interpretation of $U$ and $V$ matrices in SVD. I collect measurements at multiple devices across time are collected into an $m$ × $T$ matrix $M$, where m is the ...
1
vote
0answers
43 views

Kernel matrix decomposition

I had a look at the sklearn.kernel_approxiamtion.Nystroem implementation, which is also described in this post: Nystroem Method for Kernel Approximation Here, a ...
0
votes
0answers
68 views

How to predict for test set when training a recommender by decomposing the utility matrix X=UV?

This probably sounds stupid but I don't get the workflow of building a recommending system by the utility matrix: X[i,j] = how much the ith user likes the jth object. For practical issues I refer to ...
1
vote
1answer
50 views

Question about the Proof of PCA in "Learning from Data" by Shwartz and Ben-David, P. 280-281

Does anyone know how to justify the red and blue line in the attached proof of PCA? Red line: $B \in \mathbb{R}^{ d \times n}$, arrange $B = [B_{j,1} | B_{j,2} | \cdots | B_{j,n}]$, then $B^\top B = \...
1
vote
0answers
29 views

What is so special about the least norm solution in case of an undetermined system of equations

In particular this is the go to approach in case of solving a least squares problem that lacks a unique solution, how does being the closest point to the origin among all the solutions make it any ...
0
votes
0answers
40 views

Principal components with and without centering [duplicate]

Suppose that I am given various samples of a vector random variable as the columns $v_1,v_2,\dots,v_n$ of a certain matrix $A$. Is there a relation between the SVD $A = USV^T$ the SVD $\hat{A} = \hat{...
1
vote
0answers
40 views

SVD for a complex data matrix — what is the meaning of the columns of $V$?

I've read this wonderful explanation of SVD, where the writer mentions that the columns of $V$ are the principal directions (Summary, #1). Is this also true when the data matrix $X$ is complex? If I'm ...
0
votes
0answers
48 views

Computing Pairwise Distances Through PCA or SVD

What should I do to reduce an mxn (m=17, n=650,000) matrix, where m are samples and n are features of these samples, into a matrix of pairwise distances (which I will then use to generate a dendrogram)...
0
votes
0answers
20 views

Which form of SVD is most commonly in stats/ML (e.g., PCA, least squares)?

SVM comes in several forms: e.g., the full form and the reduced forms https://en.wikipedia.org/wiki/Singular_value_decomposition#Reduced_SVDs. Is there one that is more commonly used in stats? If so, ...
3
votes
1answer
283 views

Human intuition behind SVD in case of recommendation system

This does not answer my question. I struggled very hard to understand the SVD from a linear-algebra point of view. But in some cases I failed to connect the dots. So, I started to see all the ...

1
2 3 4 5
8