SVD refers to the singular value decomposition of a matrix. Given a matrix $\mathbf{A}$, the SVD is given by $\mathbf{A} = \mathbf{USV}^\top$ where $\mathbf{U}$ and $\mathbf{V}$ are orthogonal matrices and $\mathbf{S}$ is a diagonal matrix.

learn more… | top users | synonyms

1
vote
0answers
14 views

What fast algorithms exist for computing a _Truncated_ SVD?

Possibly off topic here, but there exist several related questions already. Poking around in the literature (or a google search for Truncated SVD Algorithms) turns up a lot of papers that use ...
0
votes
1answer
17 views

Can PCA explained variance be computed from the components (or from SVD matrices)?

I'm looking to use Spark to calculate PCAs. However I need to get the explained variance for each component and the PCAModel class doesn't appear to provide that. Is there a way to calculated the ...
0
votes
0answers
21 views

Too small baseline predictors

I'm trying to implement a recommender system, based on SVD-algorithm. I have a matrix with binary rates, i.e. 0 and 1. This matrix is very sparse. I'm using a formula for learning process: ...
0
votes
1answer
270 views

SVD interpretation of Climate data

I was going through my homework. I have a question to interpret the plot which i obtain from first and second left singular vectors after computing SVD. What can i exactly interpret, is it specific to ...
1
vote
1answer
20 views

Projecting a new entry onto the two largest eigenvectors of a PCA model

I have a bit of problems understanding how PCA and SVD works, as most materials focus on calculating the factors rather than the classification of new entries. In order to provide some context of my ...
0
votes
0answers
37 views

Using SVD or PCA for reducing dimensionality [duplicate]

I have always heard that I can reduce dimensionality of a matrix using SVD. So, I'd like to ask something hypothetically. Suppose that the following matrix A has a high dimensionality and I want to ...
0
votes
0answers
47 views

Sparse features and dimension reduction

Let sparse feature be a feature which values are subsets of some set. For example, the set of countries from which user logged to server is a sparse feature, because for each user we've got the set of ...
2
votes
2answers
53 views

Singular value and eigen-decomposition of a square symmetric matrix should be identical, but differ in sign

As far as I know, singular value decomposition (SVD) and eigendecomposition give the same result for symmetric square matrices. But when I check the results in R, that's not what I see. Please see ...
1
vote
0answers
40 views

PCA on a rank-deficient matrix using SVD of the covariance matrix

I have a high-dimensional data matrix X where sample size is smaller than the variable size. I want to use PCA as a dimensionality reduction method, but I cannot ...
1
vote
1answer
65 views

Latent Semantic Indexing and Data Centering

In PCA it's common to center the data, i.e. preprocess the data matrix such that the columns have zero mean. PCA can be done via SVD, but in this case the data matrix also has to be mean-centered. If ...
0
votes
0answers
18 views

SVD of a large sparse matrix in R [duplicate]

I have a huge matrix with roughly around half a billion entries. However, the matrix is hugely sparse. If I store it as a matrix in R, it will be around 6 gb and if I store it in spare matrix format, ...
1
vote
0answers
41 views

Which SVD matrix do we use for cosine similarity

In Latent Semantic Analysis, we get 3 matrices from the singular value decomposition (SVD), but I am confused - which matrix do we use for cosine similarity?
1
vote
1answer
40 views

What is the relationship between SVD and UV decomposition?

I am aware that both are Matrix decomposition techniques. SVD decomposes the Matrix as $\mathbf C = \mathbf U_1 \mathbf L \mathbf V_1^\top$. UV decomposes the Matrix as $\mathbf C = \mathbf ...
1
vote
2answers
116 views

Treating missing data in voting pattern analysis

I'm trying to analyze voting patterns of Ukraine's parliament deputies. I scraped all the data on their voting during last session. Each data entry has following information: Deputy name, date, bill ...
0
votes
0answers
35 views

Single value decomposition applied to Lee Carter in R

I have a problem regarding the svd in R. It is applied to matrix m, 101 rows by 69 columns, which is the demeaned log mortality rates. When I apply svd(m) (I left nv and nu out), I get d which is a ...
0
votes
0answers
27 views

Getting Rotation Matrix from SVD

I'd like to use SVD to get the rotation of ellipsoid data. When I create ellipsoid data with a known rotation, I can see that there is some kind of relationship between the $V'$ and the original ...
1
vote
0answers
30 views

how to solve weighted low rank approximation problem with diagonal weight matrix

Without weights the low rank approximation problem can be solved in terms of svd of the original matrix.But is there any way to solve the problem following problem $$\min_C{\sum_{i=1}^n \sum_{j=1}^d ...
2
votes
0answers
53 views

Fastest algorithm for rank-one approximation of a matrix [closed]

I need to compute the best (in the sense of sum of squares) rank-one approximation to a matrix, i.e. a one-component SVD, $$ \mathbf{X} \approx \sigma\mathbf{u}\mathbf{v}^T $$ such that $$ ...
0
votes
0answers
35 views

minimization of weighted frobenius norm for pca

So my problem is i like to derive pca solution as the maximum likelihood estimate for the true data.So basically i am assuming that my measured data has two component one is low rank component and ...
0
votes
1answer
69 views

Singular Value decomposition positive components

I am using Singular Value Decomposition (SVD) applied to Singular Spectrum Analysis (SSA) of a timeseries. ...
1
vote
0answers
86 views

Rotation in PCA and Factor Analysis [duplicate]

I want to know what elements are (varimax-)rotated when I rotate after PCA and after Factor Analysis. Let’s assume a standardized data vector $X$ of dimension $N \times q$. In PCA, I have the ...
1
vote
0answers
20 views

PCA using prcomp and princomp in R [duplicate]

I have been trying to do Principal Component Analysis (PCA) via R. The data set is available at https://www.dropbox.com/s/s3jstl8pu1e1xcp/Cars.csv?dl=0 I tried to do PCA via 2 different methods - ...
4
votes
2answers
93 views

Is dimensionality reduction almost always useful for classification?

Is singular value decomposition almost always useful in practice for enhancing the predicative power of a trained classification model? E.x. A dataset for classification has 20,000 features. Run SVD ...
8
votes
1answer
432 views

PCA and Correspondence analysis in their relation to Biplot

Biplot is often used to display results of principal component analysis (and of related techniques). It is a dual or overlay scatterplot showing component loadings and component scores simultaneously. ...
1
vote
1answer
48 views

Clustering singular vectors [closed]

I have a $n\times m$ matrix $A$ that I have done SVD on and picked off $k$ out of $n$ highest singular values and vectors. That is, I have decomposition for the $k$ rank approximation of $A$: $A^{k} ...
7
votes
1answer
476 views

Positioning the arrows on a PCA biplot

I am looking to implement a biplot for principal component analysis (PCA) in JavaScript. My question is, how do I determine the coordinates of the arrows from the $U,V,D$ output of the singular vector ...
3
votes
2answers
243 views

Does a correlation matrix of two variables always have the same eigenvectors?

I wanted to conduct a total least squares regression on two variables. My statistical programme does not provide TLS, but TLS luckily equals Principal Component Analysis, as far as I know. Since all ...
15
votes
3answers
885 views

Weird correlations in the SVD results of random data; do they have a mathematical explanation or is it a LAPACK bug?

I observe a very weird behaviour in the SVD outcome of random data, which I can reproduce in both Matlab and R. It looks like some numerical issue in the LAPACK library; is it? I draw $n=1000$ ...
0
votes
0answers
12 views

Interpolating singular values

So I have the singular values associated with a data matrix and I would like to interpolate them and then find the maximum curvature of the interpolation in order to decide how many singular values to ...
1
vote
1answer
61 views

Matrix Factorization in Recommender Systems: Multiple solutions?

I have implemented a recommender system for predicting user ratings based on the matrix factorization approach. $$ r_{ui}=μ+b_u+b_i+q_i^T p_u $$ Where q and p are found by mimization of the squared ...
26
votes
1answer
2k views

Relationship between SVD and PCA. How to use SVD to perform PCA?

Principal component analysis (PCA) is usually explained via an eigen-decomposition of the covariance matrix. However, it can also be performed via singular value decomposition (SVD) of the data matrix ...
1
vote
2answers
95 views

Maximally reducing the rank of a matrix by removing some rows or columns

I have a $N \times M$ matrix, and the rank of matrix, $r$, is near $\min(M,N)$. I want to minimize the rank by removing some of the rows or columns to get $r \ll \min(M,N)$. The goal is to achieve ...
1
vote
1answer
84 views

How to verify implementation of SVD in Javascript

I have implemented the SVD algortihm for my Node.js project for collaborative filtering of a sparse dataset based on this paper by GroupLens. For calculating the SVD, I am using the package node-svd ...
0
votes
0answers
99 views

Fisher information matrix with negative eigenvalues, fix through singular value decomposition?

I have the following Fisher information matrix (just an example): ...
2
votes
0answers
63 views

What does it mean to compute eigenvectors of a covariance matrix if the data were not centered first? [duplicate]

Say $\mathbf{X} \in \mathbb{R}^{n \times p}$ and $\boldsymbol{\Sigma} = \frac{1}{n}\mathbf{X}'\mathbf{X}$. The eigenvector decomposition of $\boldsymbol{\Sigma}$ gives $\boldsymbol{\Sigma} = ...
0
votes
0answers
21 views

Singular Value Decomposition (data reduction) on non-numerical data

I have a large amount of data where each datapoint contains string valued attributes, for example: ...
1
vote
0answers
74 views

Using SVD on features before SVM classification, when p >> N

So I am going through Hastie's Elements of Statistical computing, and in section 18.3.5 which deals with computational shortcuts when the number of dimensions $p$ is much larger that the number of ...
4
votes
1answer
146 views

Why are the singular values of a standardized data matrix not equal to the eigenvalues of its correlation matrix? [duplicate]

Conceptually, aren't the eigenvalues of a correlation matrix and the singular values of the associated scaled data matrix supposed to be the same? The below illustration is saying that it isn't so. ...
0
votes
0answers
175 views

Incremental SVD Recommendation System

I have the following Octave/Matlab code to compute an SVD-like matrix-decomposition: ...
1
vote
1answer
30 views

svd adds value before an elastic net model?

I learned that SVD eliminates redundancies. If you use an elastic net model, is it still greedy as stepwise models in general? or the fact that it has penalization factors reduces the greedy ...
9
votes
1answer
639 views

Is there any advantage of SVD over PCA?

I know how to calculate PCA and SVD mathematically and I know that both can be applied to Linear Least Squares regression. The main advantage of SVD mathematically seems to be that it can be applied ...
3
votes
1answer
2k views

K-means on cosine similarities vs. Euclidean distance (LSA)

I am using latent semantic analysis to represent a corpus of documents in lower dimensional space. I want to cluster these documents into two groups using k-means. Several years ago, I did this using ...
1
vote
1answer
46 views

The bias of users and items in SVD++

I'm reading Yehuda Koren's paper: "Factorization Meets the Neighborhood: a Multifaceted Collaborative Filtering Model" SIGKDD 2008. I notice that in the traditional neighborhood method, say the ...
0
votes
0answers
39 views

Choosing between different methods when the first one raises error message for linear regression

I have a linear regression problem $$Ax=b$$ My initial approach that helped to solve some of my questions was using SVD and obtaining the chi-square and some other values that I am interested but it ...
1
vote
0answers
65 views

Estimating a vector from a rank-one symmetric matrix plus scaled identity

I have a problem regarding estimating a $M\times 1$ vector from a given $M\times M$ symmetric matrix. The known matrix is a scaled identity matrix with a rank-one update. I have some idea how to ...
2
votes
1answer
137 views

PCA vs SVD - understanding difference and preference of SVD over PCA [duplicate]

I understand that PCA and SVD are similar - PCA removes the mean and SVD doesn't? I think I have an understanding of PCA - you would use it to reduce dimensions of data and separate it out into ...
1
vote
0answers
78 views

should one always perform svd before doing KNN?

I am trying to perform a Collaborative filtering for recommendation of products to customers in fashion industry.I am using the usual KNN approach to bring similarities among products. I have seen ...
1
vote
1answer
85 views

Updating SVD in Recommender Systems for change in ratings

I have read that there are projection based methods to accomodate for new user's ratings or for the ratings for a new item in SVD. However, I want to know how to update my feature space for change in ...
1
vote
2answers
245 views

How to compare the outputs of two algorithms computing SVD?

For example we have 2 algorithms from R: SVD and irlba and I want to compare them int terms of speed,memory and precision. But I don't understand how to compare output of algorithms, they must be ...
4
votes
0answers
183 views

Understanding Singular Value Decomposition in the context of LSI

My question is generally on Singular Value Decomposition (SVD), and particularly on Latent Semantic Indexing (LSI). Say, I have $ A_{word \times document} $ that contains frequencies of 5 words for ...