# Questions tagged [svd]

Singular value decomposition (SVD) of a matrix $\mathbf{A}$ is given by $\mathbf{A} = \mathbf{USV}^\top$ where $\mathbf{U}$ and $\mathbf{V}$ are orthogonal matrices and $\mathbf{S}$ is a diagonal matrix.

125 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
173 views

### What is the fastest way to compute PC1 scores, without performing the whole PCA?

I want to compute only the first principal component's scores $t_1$ of a large number $n$ of data points x with a high dimensionality $p$. Assume the data has been centered about zero. Data points ...
78 views

### Physical interpretation of $U$ and $V$ matrices in SVD

I have a question about the physical interpretation of $U$ and $V$ matrices in SVD. I collect measurements at multiple devices across time are collected into an $m$ × $T$ matrix $M$, where m is the ...
264 views

### Why PMI + SVD works for similarities arithmetics?

Recently Julia Silge blogged here and here, quoting blog entry by Chris Moody, who suggested that the similarities arithmetic in word2vec can be approximated by using PMI indexes followed by SVD ...
1k views

### Why would SVD be 'unstable' if you don't standardize your data first?

I'm reading an article about Direct Linear Transformation which processes data using SVD, and the data set is standardized so that it has zero mean and unit standard deviation (n.b., some people call ...
1k views

### Prediction using SVD and Fisher's linear discriminant

Where can I get an explanation of the procedure used when making a prediction using SVD? Let me elaborate a bit more. Suppose you have data in a matrix $A$ containing two classes. In particular, you ...
3k views

### What is the correct way to calculate the explained variance of each EOF as calculated from a gappy data set?

I am trying to determine the correct amount of variance explained by each mode of an Empirical Orthogonal Function (EOF) analysis (similar to "PCA") as applied to a gappy data set. (i.e., containing ...
218 views

### Why is it much quicker to compute ridge regression than regular linear regression?

By my understanding, for a matrix with n samples and p features: Ridge regression using cholesky takes O(p^3) time Ordinary linear regression takes O(p^3) time Singular value decomposition if u, v ...
120 views

### Why truncated SVD can denoise images

There are a lot of empirical results about that truncated SVD (TSVD) can help denoise the noises of images, but I wonder what is the theoretical support behind that? We know that TSVD is the best low-...
64 views

### Has anyone seen Gibbs phenomenon in SVD?

I read the notes on online about Regularized matrix computation. It said The truncated SVD solution has “ringing,” e.g., Gibbs’s phenomenon in truncated Fourier series I haven't seen any work ...
739 views

### Imputing missing values and SVD

Similar questions have been asked a lot of times but I have not found an answer that gives an intuitive explanation as to why this works. For reference I have read the answers here and here. As I ...
433 views

### Identifying variables contributing to near multicollinearty in linear regression using VIF's and multiple R squared's

When trying to detect collinear columns in $X$ a high proportion of cases give a $R_k^2$ close to 1 for independent columns (see figure). When near multicollinearity arise in a $n\times m$ data ...
22 views

### Singular value decomposition used for dimensionality reduction in brain signal topographic data

I am trying to replicate the localizer method described in this paper (page 4). I am stuck on a step which I don't completely understand, and I would like your input and interpretation to progress. ...
71 views

### Orthonormalization to use closed form Lasso solution

Given the Lasso problem $$min_\beta (Y-X\beta)^\top(Y-X\beta) \quad s.t. \|\beta\|_1\leq\lambda,$$ and assuming that X is orthonormal such that $X^\top X=I$, we know that the closed form solution ...
16 views

### How to approximate a Hermitian matrix with a transposed cross product of a single matrix?

I have a complex square matrix, and wish to learn latent factors (equally weighted latent factors, so not SVD) from this matrix. Given a Hermitian matrix A of ...
108 views

96 views

### matrix factorization with non-negative constraint only on one of the factors

I have a 2D spectral data time series with a wavelength dimension and a time dimension, and I'd like to decompose it to the time evolution ($SV^T$ for SVD and $H$ for NNMF) of several spectral ...
65 views

### Is there any sort of higher-order SVD (quadratic and above) for dimensionality reduction?

X-Posted on math.stackexchange, apologies, though I thought this was equally relevant to both communities. I'm wondering if there exists any higher-order SVD for dimensionality reduction. Note that ...
549 views

### SVD item similarity calculation

I am performing SVD on a rating matrix of Users and Items and I get 3 matrices out of which Vt provides latent feature for items. How do I compute similarities between a pair of items using these ...
225 views

### Data compression using either Singular Values or Eigenvalues

In many applications, an SVD of a matrix is used to determine which features are important and which ones less important. For example, in image compression, the smallest singular values are often ...
430 views

### SVD based recommender system C#

I'm trying to reduce the number of dimensions in my dataset for a movie recommender system using SVD. I'm using the 'MovieLens 1M Dataset' from GroupLens.org. I've used the MathNet library for ...
107 views

### SVD of a matrix normal: practical applications?

What are some practical applications of the distributions of the components of an SVD of a matrix of normals? In particular, assume $Y \sim N_{n \times p}({\bf 0}, \Sigma \otimes I)$, i.e. the rows ...
1k views

### What is the difference of learning Latent features using SVD and using embedding vectors in deep network

Traditionally, singular value decomposition (SVD) can be used to learn latent feature of user and items according to user-item rating matrix. Recently, researchers use embedding layers as the input ...
154 views

### Why 1 norm of reconstruction error is not used/minimized for low rank approximation using PCA?

In pca, I see reconstruction error is calculated in terms of either frobenius norm or spectral norm. And I also saw they have a closed bound in terms of singular values. My question is why 1 norm of ...
285 views

### Is there a supervised/semi supervised version of pca for dimensionality reduction?

PCA can give me the proper result if "Large variances have important dynamics" holds true for the data. In other words if I want to know along which components the variance of my data is maximized, ...
2k views

### Different order and signs of eigenvectors when doing PCA via eig() or svd() functions in Matlab

Assume we have a matrix X = randn(5,3). I am doing two things: 1) [S D1 V1] = svd(X); 2) ...
224 views

### LSA projections of documents and terms

I am trying to understand how Latent Semantic Analysis works, reading demonstrations based on singular value decomposition. Let's denote $X$ a $D \times W$ document-term matrix. The $D$ rows of $X$ ...
596 views

### How to go from sparse matrix to linear regression model (using SVD)?

I am trying to replicate the Kosinski, Stillwell, & Graepel (2013) study about predicting private traits and attributes from Facebook like data for study purposes. First I have admit, however, ...
155 views

### Truncated singular value decomposition

Is it possible to get a "truncated SVD"-regularized solution for L1 norm min errors problem? $$min\|Ax-b\|_{1}$$ In L2 universe results are derived easily analytically. I want to formulate a problem ...
628 views

### How would you preprocess data for SVD?

I am computing SVD on a matrix which is the empirical version of $E[XY^{\top}]$ for some $X \in \mathbb{R}^{m \times 1}$ and $Y \in \mathbb{R}^{n \times 1}$. I am wondering if there are standard ways ...
249 views

### Rank-one nonnegative matrix factorization

For non-negative matrix factorization with Frobenius norm: $$\min\limits_{U\in\mathbb{R}_+^{m\times r}, V\in\mathbb{R}_+^{r\times n}}||A-UV||_F^2, A\in\mathbb{R}_+^{m\times n}$$ $r=1$ is a very ...
107 views

20 views

### Principal Component Analysis on Numerical Predictors alone for Dimension Reduction

I'm trying to reduce the number of dimensions for this 'Network Anamoly Detection' dataset: https://www.kaggle.com/anushonkar/network-anamoly-detection The dataset has a total of 40 features out of ...
35 views

### Can I combine independent components from different models using PCA?

I have a set of independent components for each subject in my dataset (i.e. an ica model was generated for each subject). The samples used to generate each set of ICs are aligned across subjects, and ...
497 views

### What do the matrix (S, U, V) returned by singular value decomposition represent (in terms of variation)?

I believe SVD on a matrix A returns three matrices: U, S, and V. Let's imagine A is a data matrix with training examples/records/whatever you call them as its rows and attributes as its columns. I ...
59 views

### SVD versus RSVD

In the so-called incremental SVD used for collaborative filtering: http://www.machinelearning.org/proceedings/icml2007/papers/407.pdf http://www2.research.att.com/~volinsky/papers/ieeecomputer.pdf ...
259 views

### Why does kmeans after SVD result in ideal clusters

I am clustering tweets which are related to eye fashion and they are extracted using keywords like mascara, eyeliner, eyeshadow, etc from twitter. I constructed a Tf-idf matrix (tweets x words) ...