SVD refers to the singular value decomposition of a matrix. Given a matrix $\mathbf{A}$, the SVD is given by $\mathbf{A} = \mathbf{USV}^\top$ where $\mathbf{U}$ and $\mathbf{V}$ are orthogonal matrices and $\mathbf{S}$ is a diagonal matrix.

learn more… | top users | synonyms

3
votes
2answers
165 views

Does a correlation matrix of two variables always have the same eigenvectors?

I wanted to conduct a total least squares regression on two variables. My statistical programme does not provide TLS, but TLS luckily equals Principal Component Analysis, as far as I know. Since all ...
15
votes
3answers
779 views

Weird correlations in the SVD results of random data; do they have a mathematical explanation or is it a LAPACK bug?

I observe a very weird behaviour in the SVD outcome of random data, which I can reproduce in both Matlab and R. It looks like some numerical issue in the LAPACK library; is it? I draw $n=1000$ ...
0
votes
0answers
8 views

Interpolating singular values

So I have the singular values associated with a data matrix and I would like to interpolate them and then find the maximum curvature of the interpolation in order to decide how many singular values to ...
1
vote
1answer
30 views

Matrix Factorization in Recommender Systems: Multiple solutions?

I have implemented a recommender system for predicting user ratings based on the matrix factorization approach. $$ r_{ui}=μ+b_u+b_i+q_i^T p_u $$ Where q and p are found by mimization of the squared ...
17
votes
1answer
617 views

Relationship between SVD and PCA. How to use SVD to perform PCA?

Principal component analysis (PCA) is usually explained via an eigen-decomposition of the covariance matrix. However, it can also be performed via singular value decomposition (SVD) of the data matrix ...
1
vote
2answers
61 views

Maximally reducing the rank of a matrix by removing some rows or columns

I have a $N \times M$ matrix, and the rank of matrix, $r$, is near $\min(M,N)$. I want to minimize the rank by removing some of the rows or columns to get $r \ll \min(M,N)$. The goal is to achieve ...
1
vote
1answer
46 views

How to verify implementation of SVD in Javascript

I have implemented the SVD algortihm for my Node.js project for collaborative filtering of a sparse dataset based on this paper by GroupLens. For calculating the SVD, I am using the package node-svd ...
0
votes
0answers
36 views

Fisher information matrix with negative eigenvalues, fix through singular value decomposition?

I have the following Fisher information matrix (just an example): ...
2
votes
0answers
62 views

What does it mean to compute eigenvectors of a covariance matrix if the data were not centered first? [duplicate]

Say $\mathbf{X} \in \mathbb{R}^{n \times p}$ and $\boldsymbol{\Sigma} = \frac{1}{n}\mathbf{X}'\mathbf{X}$. The eigenvector decomposition of $\boldsymbol{\Sigma}$ gives $\boldsymbol{\Sigma} = ...
0
votes
0answers
17 views

Singular Value Decomposition (data reduction) on non-numerical data

I have a large amount of data where each datapoint contains string valued attributes, for example: ...
1
vote
0answers
35 views

Using SVD on features before SVM classification, when p >> N

So I am going through Hastie's Elements of Statistical computing, and in section 18.3.5 which deals with computational shortcuts when the number of dimensions $p$ is much larger that the number of ...
4
votes
1answer
106 views

Why are the singular values of a standardized data matrix not equal to the eigenvalues of its correlation matrix? [duplicate]

Conceptually, aren't the eigenvalues of a correlation matrix and the singular values of the associated scaled data matrix supposed to be the same? The below illustration is saying that it isn't so. ...
0
votes
0answers
109 views

Incremental SVD Recommendation System

I have the following Octave/Matlab code to compute an SVD-like matrix-decomposition: ...
1
vote
1answer
25 views

svd adds value before an elastic net model?

I learned that SVD eliminates redundancies. If you use an elastic net model, is it still greedy as stepwise models in general? or the fact that it has penalization factors reduces the greedy ...
8
votes
1answer
418 views

Is there any advantage of SVD over PCA?

I know how to calculate PCA and SVD mathematically and I know that both can be applied to Linear Least Squares regression. The main advantage of SVD mathematically seems to be that it can be applied ...
1
vote
1answer
775 views

K-means on cosine similarities vs. Euclidean distance (LSA)

I am using latent semantic analysis to represent a corpus of documents in lower dimensional space. I want to cluster these documents into two groups using k-means. Several years ago, I did this using ...
1
vote
1answer
28 views

The bias of users and items in SVD++

I'm reading Yehuda Koren's paper: "Factorization Meets the Neighborhood: a Multifaceted Collaborative Filtering Model" SIGKDD 2008. I notice that in the traditional neighborhood method, say the ...
0
votes
0answers
26 views

Choosing between different methods when the first one raises error message for linear regression

I have a linear regression problem $$Ax=b$$ My initial approach that helped to solve some of my questions was using SVD and obtaining the chi-square and some other values that I am interested but it ...
1
vote
0answers
39 views

Estimating a vector from a rank-one symmetric matrix plus scaled identity

I have a problem regarding estimating a $M\times 1$ vector from a given $M\times M$ symmetric matrix. The known matrix is a scaled identity matrix with a rank-one update. I have some idea how to ...
0
votes
1answer
91 views

PCA vs SVD - understanding difference and preference of SVD over PCA [duplicate]

I understand that PCA and SVD are similar - PCA removes the mean and SVD doesn't? I think I have an understanding of PCA - you would use it to reduce dimensions of data and separate it out into ...
1
vote
0answers
55 views

should one always perform svd before doing KNN?

I am trying to perform a Collaborative filtering for recommendation of products to customers in fashion industry.I am using the usual KNN approach to bring similarities among products. I have seen ...
1
vote
1answer
65 views

Updating SVD in Recommender Systems for change in ratings

I have read that there are projection based methods to accomodate for new user's ratings or for the ratings for a new item in SVD. However, I want to know how to update my feature space for change in ...
1
vote
2answers
213 views

How to compare the outputs of two algorithms computing SVD?

For example we have 2 algorithms from R: SVD and irlba and I want to compare them int terms of speed,memory and precision. But I don't understand how to compare output of algorithms, they must be ...
4
votes
0answers
106 views

Understanding Singular Value Decomposition in the context of LSI

My question is generally on Singular Value Decomposition (SVD), and particularly on Latent Semantic Indexing (LSI). Say, I have $ A_{word \times document} $ that contains frequencies of 5 words for ...
0
votes
0answers
37 views

After applying SVD, how do I find which features from my original dataset were most significant?

I'm using MathNet.Numerics library in C# to find the SVD but the Sigma matrix gives no indication of which values correspond to which features. It simply lists them in the most significant order. ...
-1
votes
1answer
26 views

What is the range of values that can be expected in the result of Principal Component Analysis (PCA)?

I want to normalize all of my preprocessing techniques between 0 and 1 so I want to know what the PCA range of values is so that I can apply a proper normalization to it. I applied PCA by using the ...
3
votes
2answers
162 views

How to use SVD for dimensionality reduction to reduce the number of columns (features) of the data matrix? [duplicate]

My original data has many more columns (features) than rows (users). I am trying to reduce the features of my SVD (I need all of the rows). I found one method of doing so in a book called "Machine ...
1
vote
1answer
52 views

Are SVD (Singular Value Decomposition) values always positive? Is there a relation between the maximum SVD value and the original data?

Assuming it's the standard SVD (no variation of it) with $A = USV^T$, would the $A$ matrix always have positive values (0 to $\infty$)? I noticed that the $U$ and $V^T$ matrices had some negative ...
0
votes
1answer
119 views

Different results for Singular Value Decomposition (SVD) using different tools

I am currently implementing Latent Semantic Analysis in Java using the EJML library for the preliminary Singular Value Decomposition (SVD). I am testing my code against the original term frequency ...
0
votes
0answers
20 views

Can LSA find correlations between multiple words?

I need to find correlations between multiple terms (say, 3 or 4) in a single-term search index. I'm trying to figure out if LSA fits to the problem. Am I right that LSA is no more than a term-to-term ...
0
votes
0answers
54 views

PCA eigenvalues meaning

When projecting the data set on the Eigen vectors of the co-variance matrix , the eigenvalues represent how much each example varies away from the mean of the data set in the projected direction , ...
0
votes
1answer
49 views

Question about PCA data recovery equation

In PCA , consider a 4 x 3 data matrix ( 4 examples each with 3 features ). After getting the 3 eigenvectors (a/b/c) and projecting data on the first 2 vectors, the equation looks like this : [ first ...
1
vote
0answers
97 views

SVD application for a Boolean sparse Matrix

Basically, I am trying to have a recommender system based on SVD for a Boolean utility matrix. ie If at all some entries are present in the utility matrix, they will be 1 (I made it pseudo-implicit ...
1
vote
0answers
53 views

Is there a way to perform SVD in a sequential manner?

My neurology experiment has a spike detector outputting 40 sample long spike waveforms. I'm using a dictionary method for sorting the spikes in real time. To ...
0
votes
0answers
52 views

PCA: When using eigendecompostion instead of single value decompostion [duplicate]

If you want to perform a PCA, I guess that using SVD will always work. Eigendecomposition on the covariance matrix only works when your data is not high dimensional(so n > p). But I'm wonder if there ...
0
votes
1answer
77 views

Singular Value Decomposition more columns than rows

I am confused by the singular value decomposition of a matrix. This may just be a misunderstanding of what singular value decomposition does so please be gentle with me. If I do a singular value ...
3
votes
3answers
248 views

Understanding the output of SVD when used for PCA [duplicate]

I'm doing principal components analysis (PCA) on quite a bit of data (3000 variables, 100079 data points). I'm doing this mostly for fun; data analysis is not my day job. Normally, to do a PCA I ...
0
votes
0answers
169 views

What's wrong with my Kernel algorithm (Kernel SVD)?

I have a user-item matrix $A$ as data input, which is a sparse matrix containing a large number of missing values (as zeros). Each row is a user, and each column is an item. Generally, I am conducting ...
0
votes
0answers
39 views

Best way to classify a set through a single feature

I need to classify a single dataset through a numeric value. I added below a simple dataset to explain what I need. Restriction: Category has two values: 1 or ...
4
votes
1answer
335 views

What are the advantages of kernel PCA over standard PCA?

I want to implement an algorithm in a paper which uses kernel SVD to decompose a data matrix. So I have been reading materials about kernel methods and kernel PCA etc. But it still is very obscure to ...
1
vote
0answers
60 views

How to optimize a SVD recommender regarding number of factors?

I am using R to do recommendation based on pureSVD. It is basically to choose the number of factors and then SVD the user-item matrix and then restore the matrix and provide top-N recommendations for ...
0
votes
0answers
15 views

SVD Down to One Dimension - K=1

I ran an analysis on a very sparse 40K x 40K customer-item rating matrix for recommendations; I first ran SVD on this matrix using many different reduced rank sizes, k=20,30,40... I used the results ...
0
votes
1answer
152 views

Identify original features corresponding to high singular/principal component values [duplicate]

In MATLAB, while SVD gives a diagonal matrix S of decreasing singular values, PCA gives a column vector LATENT of decreasing principal component values. How can S be used to obtain a subset of ...
3
votes
1answer
173 views

Multiple linear regression with normalization - how to get non-scaled full covariance matrix?

I am doing a quite complicated multiple regression modelling in physics and have a problem how to got back to covariance matrix for non-normalized parameters. I don't know how to calculate the error ...
2
votes
1answer
191 views

SVD & ICA — or why doesn't the other rotation matrix in SVD solve for independent components?

When data are a linear mixture of non-gaussian sources, it can be shown that with a rotation, an independent rescaling of each of the rotated axes, and a second rotation you can recover the original, ...
0
votes
0answers
178 views

The difference between SVD and SVD++

What if any is the connection between SVD (the one you learn about in your linear algebra course) and SVD++ (the one from the Netflix prize)? I know they both want to find latent factor spaces. But ...
0
votes
2answers
155 views

Compute the user and item features in SVD++

I have a sparse matrix. There is lots of missing data. Hence, I can't use SVD naively. I read Koren's SVD++ paper. I'm confused as to how the $q_i$ and $p_u$ vectors are determined. $q_i^Tp_u$ is ...
0
votes
1answer
139 views

Truncated SVD: how do I go from [Uk, Sk, Vk'] to low-dimension matrix? [duplicate]

I have a large word-frequency matrix (~6m unique words X ~4k documents) and I'm trying to use truncated singular value decomposition (SVD) to project it onto a matrix with fewer dimensions. I know how ...
1
vote
1answer
153 views

How to separate groups using PCA?

I have two groups: A) controls: $25\times2000$, B) patients: $12\times2000$. Group A has ...
3
votes
1answer
623 views

Eigenvalues of correlation matrices exhibit exponential decay

I have a data-set of $P$ samples of size $N$, and noticed that the eigenvalues of the correlation matrices $A^TA$, when presented in descending order, can in many cases be described as an exponential ...