Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [svd]

Singular value decomposition (SVD) of a matrix $\mathbf{A}$ is given by $\mathbf{A} = \mathbf{USV}^\top$ where $\mathbf{U}$ and $\mathbf{V}$ are orthogonal matrices and $\mathbf{S}$ is a diagonal matrix.

2
votes
0answers
20 views

Factorizing a matrix of distributions [on hold]

Let's say we have a matrix $X \in \mathbb{R}^{m \times n}$, then the (R-truncated) SVD allows to approximate: $X_{i,j} \approx \sum\limits_{r=1}^{R} \sigma_r \times U_{r,i} \times V_{r, j}$ Now I ...
5
votes
0answers
168 views

Why do PCA and PCoA give the same components but different explained variances?

I'm quite familiar with Principal Component Analysisis, as I use it to study genetic structure. Lately, I was revisiting some of the functions I was using in R (...
1
vote
0answers
32 views

Calculating PCA coefficients using SVD, PCA (sklearn) and Covariance Matrix

I am trying to understand PCA implemented in different methods on python. I am failing to get equal PCA coefficients in each of the methods. By PCA coefficients I mean data projected in the principle ...
0
votes
0answers
21 views

Singular values for a latent-factor model

Suppose we build a latent-factor model using alternating least squares (ALS) or stochastic gradient descent (SGD). Can we calculate weights for each latent factor, in a similar way to how the singular-...
1
vote
0answers
19 views

How SVD factorisation -based recomendation algos deal with new user interaction

Classic SVD and SVD++ alogritms generate predictions based on a current known ratings only for known users and known items. But I need to make prediction for some new user on the old items. In the ...
0
votes
1answer
65 views

SVD matrixes do not coincide with Eigen decomposition for covariance matrix [duplicate]

I am comparing the output from the singular value decomposition with the eigendecomposition of the covariance matrix (symmetric matrix). I am expecting that the Eigenvector and a non-diagonal matrix ...
0
votes
1answer
13 views

preparing free text column for regression

I have a column X which contains occupation/profession as an independent variable as free text, which is very much correlated with a continuous dependent variable. What techniques do you usually use ...
0
votes
2answers
61 views

What is the meaning of these principal components?

I have a matrix of data. I computed the principal components of my matrix using SVD (code shown below): subtract mean...then $$[U,S,V] = SVD({\rm matrix})$$ for $V$, which is the principal ...
1
vote
1answer
75 views

SVD PCA reconstruction of data [duplicate]

I have some data about the $\{noise,~ size,~ speed,~ length,~ width\}$ of cars. I have performed SVD, and I want to reconstruct my data using only the first 2 principal components. I subtracted mean ...
0
votes
0answers
31 views

Rank 1 SVD with constraint on U

I need to perform a particular rank 1 decomposition of a sparse matrix $\mathbf{A} \in \mathbb{R}^{n\times n}$. In particular I am looking for the positive vector $\mathbf{u} \in \mathbb{R}^{+n}$ ...
3
votes
1answer
301 views

Why is computing ridge regression with a Cholesky decomposition much quicker than using SVD?

By my understanding, for a matrix with n samples and p features: Ridge regression using Cholesky decomposition takes O(p^3) time Ridge regression using SVD takes O(p^3) time Computing SVD when only ...
3
votes
0answers
43 views

Why is it much quicker to compute ridge regression than regular linear regression?

By my understanding, for a matrix with n samples and p features: Ridge regression using cholesky takes O(p^3) time Ordinary linear regression takes O(p^3) time Singular value decomposition if u, v ...
0
votes
0answers
26 views

Using numpy SVD to calculate factor loadings [duplicate]

I'm doing PCA (Principal Component Analysis) in Python using the numpys Singular Value Decomposition. Effectively extracting the principal components like so: ...
0
votes
0answers
15 views

Meaning of matrix U in LSA?

LSA uses SVD algorithm i.e. the terms-documents matrix $=U \Sigma V^T$ What is the exact meaning of matrix U here: I know that it is a rotation but rotation of what? Is it a concepts space? If yes ...
0
votes
0answers
29 views

Can anyone help me with step by step procedure in MATLAB for missing data imputation using Singular Value Decomposition (SVD [duplicate]

I need step by step procedure for imputing data using SVD in Matlab. Please provide resources if any !
0
votes
0answers
7 views

Explanation of SVD in relation to specific research question (building vector on multiple 3D coordinates)

unfortunately I'm completely out of my comfort zone when it comes to the math involved in my question, appologies in advance. Background: I am evaluating the presicion of electrode placement in the ...
2
votes
1answer
45 views

Reference point in projection axis of SVD (singular value decomposition)

I am watching a YouTube video on SVD, and attempting to recreate some of its examples to better understand the internal machinery of the algorithm. In one of the slides, the instructor mentions that ...
0
votes
0answers
26 views

Additional Property of Singular Value Decomposition

I am new to SVD so forgive me if the question is trivial. Following is my question. If I have two sets of linear equations, Y1 = T1.X Y2 = T2.X where T1 and T2 are mxn rectangular matrices. Now let'...
0
votes
0answers
32 views

Idea behind change of basis and how it relates to projecting your points onto principal components

I would like to clarify if my understanding is correct. In the traditional X-Y coordinate system, our choice of basis vectors are $\vec{i} = (1, 0)$ and $\vec{j} = (0, 1)$ and when you I have a point $...
0
votes
0answers
37 views

SVD for simultaneous row and column reduction of a squared matrix

I have an n x n similarity matrix which I'd like to reduce to a smaller square matrix. I am aware of this answer: How to use SVD for dimensionality reduction to reduce the number of columns (features)...
1
vote
0answers
20 views

Decomposing SVD of this dataset by hand (perspective)

I have the following data matrix $\left[\begin{array}{ccccc} 1 & 1 & 1 & 0 & 0\\ -3 & -3 & -3 & 0 & 0\\ 2 & 2 & 2 & 0 & 0\\ 0 & 0 & 0 & -1 &...
1
vote
0answers
32 views

Is low rank finite-iteration manifold identification possible?

In sparse optimization, I am trying to solve the problem $$ \min_{x\in \mathbb R^{n}} \quad f(x) + \|x\|_1 $$ and at optimality, $x^*$ may be sparse. If I define the sparse manifold as $\mathcal M = ...
0
votes
0answers
34 views

Eigenvalue decomposition/SVD and the filtering perspective

I have been studying the SVD algorithm recently and I can understand how it might be used for compression but I am trying to figure out if there is a perspective of SVD where it can be seen as a low ...
2
votes
0answers
48 views

Why truncated SVD can denoise images

There are a lot of empirical results about that truncated SVD (TSVD) can help denoise the noises of images, but I wonder what is the theoretical support behind that? We know that TSVD is the best low-...
0
votes
0answers
64 views

Does Column ordering matter in QR decomposition?

I am trying to understand if the ordering of columns matters in QR decompsoition. In general it seems that column ordering won't matter. I guess for SVD or any matrix factorization the way columns ...
4
votes
2answers
133 views

Eckart-Young-Mirsky theorem: rank $≤k$ or rank $=k$

The Eckart-Young-Mirsky theorem is sometimes stated with rank $\le k$ and sometimes with rank $= k$. Why? More specifically, given a matrix $X \in \mathbb{R}^{n \times d}$, and a natural number $k \...
1
vote
0answers
51 views

Extracting latent vectors from autoencoder similar to SVD

I have read that there is an equivalency between a linear autoencoder and performing SVD. SVD can be used in collaborative filtering, for example, factorization of a user-movies matrix $\mathbf{M}$ ...
0
votes
1answer
252 views

PCA in psych package with more columns than rows

Why is it impossible to do a PCA in R using principal from psych package without warnings with a matrix, which has more columns ...
1
vote
0answers
31 views

PCA for three-dimensional linear fit on time-resolved trajectory

I study the behavior of organisms that are able of self-locomotion and that show directed motion toward one another. This directed motion occurs through the detection of chemical trails released by ...
4
votes
1answer
66 views

Derivation of Procrustes transformation

I'm curious about the derivation for the Procrustes transformation. I'm following ESLII: see figure 14.25 and problem 14.8 (Procrustes distance with scaling). Given matrices $\mathbf{X}_1$ and $\...
1
vote
0answers
55 views

matrix factorization with non-negative constraint only on one of the factors

I have a 2D spectral data time series with a wavelength dimension and a time dimension, and I'd like to decompose it to the time evolution ($SV^T$ for SVD and $H$ for NNMF) of several spectral ...
2
votes
0answers
43 views

Is there any sort of higher-order SVD (quadratic and above) for dimensionality reduction?

X-Posted on math.stackexchange, apologies, though I thought this was equally relevant to both communities. I'm wondering if there exists any higher-order SVD for dimensionality reduction. Note that ...
0
votes
0answers
86 views

How SVD is used for dimensionality reduction? [duplicate]

Given a $M \times N$ data matrix $ D = (x_1, x_2, \cdots, x_M)^{T}$. Applying singular value decomposition to $D$ yields $$D = USV^{T} = (u_1, u_2, \cdots, u_M) \begin{pmatrix} s_1 &&0\\...
0
votes
0answers
124 views

Why is the size of fitted truncated svd model is so big?

I have a dataset with tfidf matrix of shape (200000, 565000). I am fitting truncated svd of 500 dimensions from sklearn onto it and pickling the resulting svd object for later use. The pickle file is ...
4
votes
2answers
162 views

The Curse of high Dimension And Distance

For extracting features from video frames (2 sample/sec) I use keras framework in python and load VGG16 that input size is (150,150,3) and output size is (4,4,512). After the feature extraction step I ...
5
votes
1answer
81 views

Evaluate approximation of PCA from randomized algorithms

I have been comparing different PCA implementations (some via explicit calculation of the covariance matrix, some with randomized/truncated SVD) in terms of speed, and now wanted to compare how good ...
2
votes
1answer
100 views

Latent variable and Factor analysis ICA

While I was going through the factor analysis for Independent component analysis, I got stuck in one statement. How does it come to co-variance of S* is I? Is A* =ART ? Following is what I was going ...
0
votes
0answers
73 views

How to choose reduced features using SVD from a dataset?

How svd reduces features from a matrix. suppose, Matrix A(m,n). if we apply svd to A then we will find matrix U(m,m), S(m,n), v(n,n) matrix. S is the strength/ diagonal matrix as I know. We can ignore ...
1
vote
1answer
250 views

Orthogonality as found by the Gram-Schmidt process vs. uncorrelated basis vectors

I have a data matrix $Y$ of size $n \times p$, a basis vector in $\mathbb{R}^p$ $v_1$, and a potential basis vector in $\mathbb{R}^p$ $v_2'$. Then, if I use the Gram-Schmidt process on $[v_1, v_2']$ ...
0
votes
0answers
25 views

How to return the variance size of a recommendation system using SVD

One problem that many people have when making a recommendation system is the reasonableness of the suggestion / prediction, so I wanted to know how we can calculate the variance size of a generic ...
2
votes
1answer
48 views

Has anyone seen Gibbs phenomenon in SVD?

I read the notes on online about Regularized matrix computation. It said The truncated SVD solution has “ringing,” e.g., Gibbs’s phenomenon in truncated Fourier series I haven't seen any work ...
4
votes
2answers
4k views

How does eigenvalues measure variance along the principal components in PCA? [duplicate]

I understand that eigenvalues measure variance along the principal components. Questions How are eigenvalues and variance same for PCA? What is the intuition behind this being the same? What is ...
0
votes
1answer
309 views

Relationship between Alternating Least Squares and SVD

I have been assuming that ALS is simply an alternative algorithm for doing matrix decomposition that is more efficient, but in the end produces the same $U$ & $V$ matrices that SVD does. Is this ...
2
votes
1answer
444 views

PCA - Reconstruction from a “clean” set of eigenvectors?

This is a question related to the explanation here on how to reconstruct data from PCs found here: How to reverse PCA and reconstruct original variables from several principal components? I have two ...
0
votes
0answers
107 views

Singular value decomposition

Can singular value decomposition used to impute missing values in highly nonlinear process under multiple input and multiple output behavior?
2
votes
2answers
1k views

Explaining dimensionality reduction using SVD (without reference to PCA)

I have seen Dimensionality reduction mentioned as one of the practical usages of SVD. However, the explanation for me has always been Let me find the directions in which the variance of the data is ...
0
votes
1answer
185 views

TruncatedSVD always reduces dataset to 1D

I know that I have a large sparse matrix which I'm using TruncatedSVD to condense into a smaller number of dimensions. Here is my code: ...
1
vote
0answers
57 views

Understanding Diagonal Matrix of SVD [closed]

I have a matrix A with dimensions 4x3. I performed SVD on the matrix using numpy (np.linalg.svd) on matrix A. The output dimensions of U, V, and S are (4x4),(3x3),(3,). So V comprise orthonormal ...
0
votes
0answers
126 views

How do I assign meaning or give names to the latent variables in exploratory factor analysis?

I read the book Latent Variable Models, and in the chapter dealing with exploratory factor analysis the author shows a way to learn latent variables (factors) from ...
0
votes
1answer
325 views

What is the correct way to scale data, apply PCA and fit a Multivariate Normal Distribution for anomaly detection?

I want to train an anomaly detection model in python. I have a training data set with some 30,000 observations, 700 of which are anomalies, and I can distinguish between normal and anomalous cases (I ...