SVD refers to the singular value decomposition of a matrix. Given an r x c matrix $\mathbf{A}$, the SVD is given by $\mathbf{A} = \mathbf{USV'}$ where $\mathbf{U}$ and $\mathbf{V}$ are orthogonal matrices and $\mathbf{S}$ is a diagonal matrix.

learn more… | top users | synonyms

0
votes
0answers
14 views

Choosing between different methods when the first one raises error message for linear regression

I have a linear regression problem $$Ax=b$$ My initial approach that helped to solve some of my questions was using SVD and obtaining the chi-square and some other values that I am interested but it ...
0
votes
0answers
10 views

Estimating a vector from a rank-one symmetric matrix plus scaled identity

I have a problem regarding estimating a $M\times 1$ vector from a given $M\times M$ symmetric matrix. The known matrix is a scaled identity matrix with a rank-one update. I have some idea how to ...
0
votes
1answer
51 views

PCA vs SVD - understanding difference and preference of SVD over PCA [duplicate]

I understand that PCA and SVD are similar - PCA removes the mean and SVD doesn't? I think I have an understanding of PCA - you would use it to reduce dimensions of data and separate it out into ...
0
votes
0answers
14 views

should one always perform svd before doing KNN?

I am trying to perform a Collaborative filtering for recommendation of products to customers in fashion industry.I am using the usual KNN approach to bring similarities among products. I have seen ...
0
votes
0answers
26 views

Updating SVD in Recommender Systems for change in ratings

I have read that there are projection based methods to accomodate for new user's ratings or for the ratings for a new item in SVD. However, I want to know how to update my feature space for change in ...
1
vote
2answers
145 views

How to compare the outputs of two algorithms computing SVD?

For example we have 2 algorithms from R: SVD and irlba and I want to compare them int terms of speed,memory and precision. But I don't understand how to compare output of algorithms, they must be ...
2
votes
0answers
38 views

Understanding Singular Value Decomposition in the context of LSI

My question is generally on Singular Value Decomposition (SVD), and particularly on Latent Semantic Indexing (LSI). Say, I have $ A_{word \times document} $ that contains frequencies of 5 words for ...
0
votes
0answers
18 views

Is subtracting the mean from PCA necessary when using an SVD result that is feature scaled?

I've applied SVD to the original data matrix and eliminated insignificant columns and rows from U and V^T respectively using the Sigma values. I multiplied together my optimized U, Sigma, and V^T ...
0
votes
0answers
19 views

After applying SVD, how do I find which features from my original dataset were most significant?

I'm using MathNet.Numerics library in C# to find the SVD but the Sigma matrix gives no indication of which values correspond to which features. It simply lists them in the most significant order. ...
0
votes
1answer
20 views

What is the range of values that can be expected in the result of Principal Component Analysis (PCA)?

I want to normalize all of my preprocessing techniques between 0 and 1 so I want to know what the PCA range of values is so that I can apply a proper normalization to it. I applied PCA by using the ...
0
votes
1answer
33 views

How to use SVD for dimensionality reduction to reduce columns specifically?

My original data has many more columns (features) than rows (users). I'm trying to reduce the features of my SVD (I need all of the rows). I found one method of doing so in a book called "Machine ...
1
vote
1answer
27 views

Are SVD (Singular Value Decomposition) values always positive? Is there a relation between the maximum SVD value and the original data?

Assuming it's the standard SVD (no variation of it) with $A = USV^T$, would the $A$ matrix always have positive values (0 to $\infty$)? I noticed that the $U$ and $V^T$ matrices had some negative ...
0
votes
1answer
40 views

Different results for Singular Value Decomposition (SVD) using different tools

I am currently implementing Latent Semantic Analysis in Java using the EJML library for the preliminary Singular Value Decomposition (SVD). I am testing my code against the original term frequency ...
0
votes
0answers
14 views

Can LSA find correlations between multiple words?

I need to find correlations between multiple terms (say, 3 or 4) in a single-term search index. I'm trying to figure out if LSA fits to the problem. Am I right that LSA is no more than a term-to-term ...
0
votes
0answers
40 views

PCA eigenvalues meaning

When projecting the data set on the Eigen vectors of the co-variance matrix , the eigenvalues represent how much each example varies away from the mean of the data set in the projected direction , ...
0
votes
1answer
38 views

Question about PCA data recovery equation

In PCA , consider a 4 x 3 data matrix ( 4 examples each with 3 features ). After getting the 3 eigenvectors (a/b/c) and projecting data on the first 2 vectors, the equation looks like this : [ first ...
0
votes
0answers
76 views

SVD application for a Boolean sparse Matrix

Basically, I am trying to have a recommender system based on SVD for a Boolean utility matrix. ie If at all some entries are present in the utility matrix, they will be 1 (I made it pseudo-implicit ...
1
vote
0answers
42 views

Is there a way to perform SVD in a sequential manner?

My neurology experiment has a spike detector outputting 40 sample long spike waveforms. I'm using a dictionary method for sorting the spikes in real time. To ...
1
vote
1answer
71 views

PCA followed by LDA: dimension reduction strategy

I have a high dimensional dataset (n*p: 30 * 100) which I want to use as an testing dataset to build a two group classifier (LDA or QDA). I've read that you can do PCA to do an dimension reduction of ...
0
votes
0answers
46 views

SVD on a transposed matrix

I struggle with the interpretation of the singular values obtained by the SVD, also in relation to what the principal components of the PCA tell me. The mathematical background has been described ...
1
vote
0answers
40 views

PCA: When using eigendecompostion instead of single value decompostion

If you want to perform a PCA, I guess that using SVD will always work. Eigendecomposition on the covariance matrix only works when your data is not high dimensional(so n > p). But I'm wonder if there ...
0
votes
1answer
54 views

Singular Value Decomposition more columns than rows

I am confused by the singular value decomposition of a matrix. This may just be a misunderstanding of what singular value decomposition does so please be gentle with me. If I do a singular value ...
0
votes
0answers
84 views

What's wrong with my Kernel algorithm (Kernel SVD)?

I have a user-item matrix $A$ as data input, which is a sparse matrix containing a large number of missing values (as zeros). Each row is a user, and each column is an item. Generally, I am conducting ...
0
votes
0answers
33 views

Best way to classify a set through a single feature

I need to classify a single dataset through a numeric value. I added below a simple dataset to explain what I need. Restriction: Category has two values: 1 or ...
3
votes
1answer
166 views

What are the advantages of kernel PCA over standard PCA?

I want to implement an algorithm in a paper which uses kernel SVD to decompose a data matrix. So I have been reading materials about kernel methods and kernel PCA etc. But it still is very obscure to ...
0
votes
0answers
25 views

How to optimize a SVD recommender regarding number of factors?

I am using R to do recommendation based on pureSVD. It is basically to choose the number of factors and then SVD the user-item matrix and then restore the matrix and provide top-N recommendations for ...
0
votes
0answers
14 views

SVD Down to One Dimension - K=1

I ran an analysis on a very sparse 40K x 40K customer-item rating matrix for recommendations; I first ran SVD on this matrix using many different reduced rank sizes, k=20,30,40... I used the results ...
0
votes
1answer
67 views

Identify features corresponding to high singular/principal component values

In MATLAB, while SVD gives a diagonal matrix S of decreasing singular values, PCA gives a column vector LATENT of decreasing principal component values. How can S be used to obtain a subset of ...
2
votes
0answers
60 views

multiple linear regression with normalization - how to get non-scaled full covariance matrix

I am doing a quite complicated multiple regression modelling in physics and have a problem how to got back to covariance matrix for non-normalized parameters. I don't know how to calculate the error ...
2
votes
1answer
86 views

SVD & ICA — or why doesn't the other rotation matrix in SVD solve for independent components?

When data are a linear mixture of non-gaussian sources, it can be shown that with a rotation, an independent rescaling of each of the rotated axes, and a second rotation you can recover the original, ...
0
votes
0answers
91 views

The difference between SVD and SVD++

What if any is the connection between SVD (the one you learn about in your linear algebra course) and SVD++ (the one from the Netflix prize)? I know they both want to find latent factor spaces. But ...
0
votes
1answer
88 views

Compute the user and item features in SVD++

I have a sparse matrix. There is lots of missing data. Hence, I can't use SVD naively. I read Koren's SVD++ paper. I'm confused as to how the $q_i$ and $p_u$ vectors are determined. $q_i^Tp_u$ is ...
0
votes
1answer
78 views

Truncated SVD: how do I go from [Uk, Sk, Vk'] to low-dimension matrix?

I have a large word-frequency matrix (~6m unique words X ~4k documents) and I'm trying to use truncated singular value decomposition (SVD) to project it onto a matrix with fewer dimensions. I know how ...
1
vote
1answer
123 views

How to separate groups using PCA?

I have two groups: A) controls: $25\times2000$, B) patients: $12\times2000$. Group A has ...
3
votes
1answer
352 views

Eigenvalues of correlation matrices exhibit exponential decay

I have a data-set of $P$ samples of size $N$, and noticed that the eigenvalues of the correlation matrices $A^TA$, when presented in descending order, can in many cases be described as an exponential ...
0
votes
1answer
22 views

Building a Dictionary Using PCA / SVD and Regenerating the Signals

I have the following problem and I know I can solve it using PCA / SVD, I just can get to write it properly. I have many sensors, each produces M = 3500 samples of a signal. The signal of each ...
0
votes
1answer
206 views

PCA demeaning the data [duplicate]

What is the motivation for demeaning the data when doing PCA. I've been told to do it, but I've never heard a good and/or intuitive reason for it. Is this a case where doing it just makes the math ...
0
votes
0answers
37 views

intuitive understanding of max-min in optimization

I have problem in understanding max-min in optimization. I know what does it mean to maximize obejctive function or minimize it, I don't really understand what does it mean to have objective function ...
1
vote
1answer
39 views

Differing definitions of Matrix Condition Number

When describing the condition number of a correlation matrix I have seen is described as the ratio of the singular values (from singluar value decomposition). I have also seen it described as the ...
0
votes
0answers
40 views

How to re-construct a matrix from SVD

I have a Audio time-series, to which I'm trying to detect the most significant parts of the signal, i.e. the voiced parts and forget the unvoiced parts. $$ T = [0, 0, 1, 1, .....n] $$ I then ...
1
vote
0answers
40 views

Use of latent semantic analysis in topic modeling

I am learning Latent Semantic Analysis. After we have the SVD decomposition of term-document matrix, how to infer topics from that? For example, how to get the following result: Topic 1: apple, ...
0
votes
0answers
73 views

equivalent of PCA explained variance ratio for SVD ?

i am wondering if there is an equivalent of PCA explained variance ratio for SVD. What are the measures I can get to monitor the number of columns I keep after the SVD ? Are any of these metrics ...
3
votes
1answer
240 views

time series dimensionality reduction

I have a call center data (such as one below) that has call data collected every 15 minutes. For a day the periodicity is 96 and for a week the periodicity is (7 x 96 = 672). If I would like to ...
1
vote
1answer
122 views

Kernel PCA with an SVD algo

Suppose that I have a great algo for calculating the SVD and I want to do Kernel PCA. It is possible to first apply the Kernel function to my data and then run the SVD algo on the transformed data?
0
votes
0answers
65 views

Confusion related to Singular Value Decomposition (SVD)

I have this problem where I am given a matrix A of size nxp. I run svd on this matrix A. Then I am supposed to plot this matrix using imagesc in matlab where rows are sorted by the first row ...
2
votes
0answers
105 views

Relationship between L2 regularization and PCA?

I'm new in the forum. I remember having read somewhere in the web a connection between $\ell_2$ regularization and PCA: while using $\ell_2$-regularized regression with hyperparameter $\lambda$, if ...
8
votes
2answers
396 views

Why PCA of data by means of SVD of the data?

This question is about an efficient way to compute principal components. Many texts on linear PCA advocate using singular-value decomposition of the casewise data. That is, if we have data $\bf X$ ...
2
votes
0answers
72 views

Why do PCA and Factor Analysis return different results in this example?

The following question is about an example from "The Elements of Statistical Learning" by Hastie, Friedman and Tibshirani $X_1 = Z_1 $ $X_2 = X_1 + 0.001 * Z_2 $ $X_3 = 10 * Z_3 $ Where $ Z_1, ...
2
votes
0answers
60 views

Truncated singular value decomposition

Is it possible to get a "truncated SVD"-regularized solution for L1 norm min errors problem? $$min\|Ax-b\|_{1}$$ In L2 universe results are derived easily analytically. I want to formulate a problem ...
2
votes
1answer
158 views

What are the differences between two kinds of PCA?

The book "Elements of Statistical Learning" describes Principal Components Analysis through SVD as follows: $ X = UDV^T $ Then $ UD $ are the Principal Components and $ V $ are the directions. ...