SVD refers to the singular value decomposition of a matrix. Given an r x c matrix $\mathbf{A}$, the SVD is given by $\mathbf{A} = \mathbf{USV'}$ where $\mathbf{U}$ and $\mathbf{V}$ are orthogonal matrices and $\mathbf{S}$ is a diagonal matrix.

learn more… | top users | synonyms

0
votes
0answers
18 views

Best way to classify a set through a single feature

I need to classify a single dataset through a numeric value. I added below a simple dataset to explain what I need. Restriction: Category has two values: 1 or ...
2
votes
1answer
39 views

What are the advantages of kernel PCA over PCA?

I want to implement an algorithm in a paper which uses Kernel SVD to decompose a data matrix. So I have been reading materials about Kernel methods and kernel PCA etc. But it still is very obscure to ...
0
votes
0answers
20 views

Movie recommendation problem

I have a set of data with features of movies and features of users and a third matrix with ratings of user for each movie. I have to build a recommendation system for new users. Can you help me with ...
0
votes
0answers
5 views

How to optimize a SVD recommender regarding number of factors?

I am using R to do recommendation based on pureSVD. It is basically to choose the number of factors and then SVD the user-item matrix and then restore the matrix and provide top-N recommendations for ...
0
votes
0answers
10 views

SVD Down to One Dimension - K=1

I ran an analysis on a very sparse 40K x 40K customer-item rating matrix for recommendations; I first ran SVD on this matrix using many different reduced rank sizes, k=20,30,40... I used the results ...
0
votes
1answer
38 views

Identify features corresponding to high singular/principal component values

In MATLAB, while SVD gives a diagonal matrix S of decreasing singular values, PCA gives a column vector LATENT of decreasing principal component values. How can S be used to obtain a subset of ...
0
votes
0answers
8 views

Cannot run SVD with Armadillo [migrated]

I am trying to run an SVD on a 19016x19016 matrix on my Mac OSX Mavericks with Armadillo linked to Intel MKL. But I get the following error: ...
2
votes
0answers
22 views

multiple linear regression with normalization - how to get non-scaled full covariance matrix

I am doing a quite complicated multiple regression modelling in physics and have a problem how to got back to covariance matrix for non-normalized parameters. I don't know how to calculate the error ...
1
vote
1answer
35 views

SVD & ICA — or why doesn't the other rotation matrix in SVD solve for independent components?

When data are a linear mixture of non-gaussian sources, it can be shown that with a rotation, an independent rescaling of each of the rotated axes, and a second rotation you can recover the original, ...
0
votes
0answers
33 views

The difference between SVD and SVD++

What if any is the connection between SVD (the one you learn about in your linear algebra course) and SVD++ (the one from the Netflix prize)? I know they both want to find latent factor spaces. But ...
0
votes
1answer
30 views

Compute the user and item features in SVD++

I have a sparse matrix. There is lots of missing data. Hence, I can't use SVD naively. I read Koren's SVD++ paper. I'm confused as to how the $q_i$ and $p_u$ vectors are determined. $q_i^Tp_u$ is ...
0
votes
1answer
42 views

Truncated SVD: how do I go from [Uk, Sk, Vk'] to low-dimension matrix?

I have a large word-frequency matrix (~6m unique words X ~4k documents) and I'm trying to use truncated singular value decomposition (SVD) to project it onto a matrix with fewer dimensions. I know how ...
1
vote
1answer
91 views

How to separate groups using PCA?

I have two groups: A) controls: $25\times2000$, B) patients: $12\times2000$. Group A has ...
3
votes
1answer
148 views

Eigenvalues of correlation matrices exhibit exponential decay

I have a data-set of $P$ samples of size $N$, and noticed that the eigenvalues of the correlation matrices $A^TA$, when presented in descending order, can in many cases be described as an exponential ...
0
votes
1answer
20 views

Building a Dictionary Using PCA / SVD and Regenerating the Signals

I have the following problem and I know I can solve it using PCA / SVD, I just can get to write it properly. I have many sensors, each produces M = 3500 samples of a signal. The signal of each ...
0
votes
1answer
77 views

PCA demeaning the data [duplicate]

What is the motivation for demeaning the data when doing PCA. I've been told to do it, but I've never heard a good and/or intuitive reason for it. Is this a case where doing it just makes the math ...
0
votes
0answers
30 views

intuitive understanding of max-min in optimization

I have problem in understanding max-min in optimization. I know what does it mean to maximize obejctive function or minimize it, I don't really understand what does it mean to have objective function ...
1
vote
1answer
32 views

Differing definitions of Matrix Condition Number

When describing the condition number of a correlation matrix I have seen is described as the ratio of the singular values (from singluar value decomposition). I have also seen it described as the ...
0
votes
0answers
29 views

How to re-construct a matrix from SVD

I have a Audio time-series, to which I'm trying to detect the most significant parts of the signal, i.e. the voiced parts and forget the unvoiced parts. $$ T = [0, 0, 1, 1, .....n] $$ I then ...
1
vote
0answers
20 views

Use of latent semantic analysis in topic modeling

I am learning Latent Semantic Analysis. After we have the SVD decomposition of term-document matrix, how to infer topics from that? For example, how to get the following result: Topic 1: apple, ...
0
votes
0answers
40 views

equivalent of PCA explained variance ratio for SVD ?

i am wondering if there is an equivalent of PCA explained variance ratio for SVD. What are the measures I can get to monitor the number of columns I keep after the SVD ? Are any of these metrics ...
3
votes
1answer
199 views

time series dimensionality reduction

I have a call center data (such as one below) that has call data collected every 15 minutes. For a day the periodicity is 96 and for a week the periodicity is (7 x 96 = 672). If I would like to ...
1
vote
1answer
68 views

Kernel PCA with an SVD algo

Suppose that I have a great algo for calculating the SVD and I want to do Kernel PCA. It is possible to first apply the Kernel function to my data and then run the SVD algo on the transformed data?
0
votes
0answers
48 views

Confusion related to Singular Value Decomposition (SVD)

I have this problem where I am given a matrix A of size nxp. I run svd on this matrix A. Then I am supposed to plot this matrix using imagesc in matlab where rows are sorted by the first row ...
2
votes
0answers
76 views

Relationship between L2 regularization and PCA?

I'm new in the forum. I remember having read somewhere in the web a connection between $\ell_2$ regularization and PCA: while using $\ell_2$-regularized regression with hyperparameter $\lambda$, if ...
6
votes
2answers
222 views

Why PCA of data by means of SVD of the data?

This question is about an efficient way to compute principal components. Many texts on linear PCA advocate using singular-value decomposition of the casewise data. That is, if we have data $\bf X$ ...
2
votes
0answers
54 views

Why do PCA and Factor Analysis return different results in this example?

The following question is about an example from "The Elements of Statistical Learning" by Hastie, Friedman and Tibshirani $X_1 = Z_1 $ $X_2 = X_1 + 0.001 * Z_2 $ $X_3 = 10 * Z_3 $ Where $ Z_1, ...
2
votes
0answers
41 views

Truncated singular value decomposition

Is it possible to get a "truncated SVD"-regularized solution for L1 norm min errors problem? $$min\|Ax-b\|_{1}$$ In L2 universe results are derived easily analytically. I want to formulate a problem ...
2
votes
1answer
126 views

What are the differences between two kinds of PCA?

The book "Elements of Statistical Learning" describes Principal Components Analysis through SVD as follows: $ X = UDV^T $ Then $ UD $ are the Principal Components and $ V $ are the directions. ...
0
votes
0answers
31 views

R DFA eigenvalues

I'm doing some followup to MANOVA analysis and I was wondering how in R you get eigenvalues for DFA model? There is a attribute $svd in the return value of ...
0
votes
0answers
26 views

Creating a bound for response variable using softImpute?

I'm working on the Netflix challenge in R and was I'm curious if there is a way to create a bound for the possible responses that the softImpute algorithm can predict. It doesn't look like there is a ...
1
vote
0answers
108 views

Incremental SVD in Collaborative Filtering

In the so-called incremental SVD used for collaborative filtering: http://www.machinelearning.org/proceedings/icml2007/papers/407.pdf http://www2.research.att.com/~volinsky/papers/ieeecomputer.pdf ...
1
vote
0answers
97 views

Netflix Challenge - some help with SVD/SoftImpute

I'm currently working on the Netflix Challenge with the original huge dataset and have run into some problems. I don't have access to any servers or computing clusters so I've been running everything ...
2
votes
1answer
504 views

PCA and SVD matlab

I am having some problem in computing SVD and PCA in Matlab. I do not know if I am doing theoretical mistakes or programming mistakes. Starting with a data matrix $X$ PCA computes the eigenvalue ...
2
votes
0answers
143 views

Why would SVD be 'unstable' if you don't standardize your data first?

I'm reading an article about Direct Linear Transformation which processes data using SVD, and the data set is standardized so that it has zero mean and unit standard deviation (n.b., some people call ...
1
vote
2answers
257 views

SVD in linear regression

I was reading the book Elements of Statistical Learning and came across the section that tried to interpret ridge regression using singular value decomposition (SVD) of the design matrix, $X$. ...
0
votes
0answers
58 views

How to explain LSA results

I used an LSA (Latent Semantic Analysis) code in python to find the similarity of two documents. The results of the code are the following, but I cannot understand what is the score of the similarity. ...
0
votes
1answer
94 views

Truncated SVD matrix reconstruction: what is the meaning of the real values?

Im my algorithm, I am working with Singular Value Decomposition (SVD). I have an input matrix $A_{in} \in \{0,1\}^{(m * n)} $, made by $n$ rows and $m$ colums. All the entries are 0 or 1. I ...
5
votes
0answers
96 views

Using SVD to calculate PCA

The Wikipedia article on principal component analysis states that Efficient algorithms exist to calculate the SVD of $X$ without having to form the matrix $X^TX$, so computing the SVD is now the ...
1
vote
0answers
72 views

Computing similarity in LSA

I have a question regarding Latent Semantic Analysis (LSA) introduced in http://lsa.colorado.edu/papers/JASIS.lsi.90.pdf‎ . On page 14, schemas for calculating similarities between different types of ...
2
votes
1answer
1k views

LSA vs. PCA (document clustering)

I'm investigation various techniques used in document clustering and I would like to clear some doubts concerning PCA (principal component analysis) and LSA (latent semantic analysis). First thing - ...
0
votes
0answers
92 views

Making sense of the output of svd() in R

I created a matrix of dimensions 6X5 and applied the svd() function on it. ...
2
votes
6answers
190 views

Are there useful applications of SVD that use only the smallest singular values?

In a number of singular value decomposition (SVD) applications, for example Latent Semantic Indexing, only the biggest singular values are used to make searches and calculate distances. Are there ...
3
votes
2answers
202 views

Using the 'U' Matrix of SVD as Feature Reduction

This is a follow-up to the question asked regarding SVD and dimensionality reduction (question). In that question I asked how to use SVD for dimensionality reduction. Although not stated, the ...
1
vote
1answer
356 views

How to use SVD for dimensionality reduction [duplicate]

After reading several "tutorials" on SVD I am left still wondering how to use it for dimensionality reduction. Here is my confusion in an applied setting. If I limit svd to only considering the first ...
1
vote
0answers
49 views

What would be a reliable way to check if a square matrix is computationally singular?

To diagnose computational singularity, one could, for example, check rank deficiency or compute singular-value decomposition. I have the following questions: (1). Do the two methods listed above ...
3
votes
0answers
141 views

Prediction using SVD and Fisher's linear discriminant

Where can I get an explanation of the procedure used when making a prediction using SVD? Let me elaborate a bit more. Suppose you have data in a matrix $A$ containing two classes. In particular, you ...
5
votes
1answer
210 views

Interpreting matrices of SVD in practical applications

I have a question regarding the interpretation of the different matrices produced by singular value decomposition. Suppose a mxn matrix $A$ containing n images of m pixels. So each column of this ...
0
votes
0answers
87 views

Principal Component Analysis

I am a bit new to this. But I needed a clarification. I am creating a module in MATLAB that performs PCA on various data sets with varying number of explanatory and response variables. Depending on ...
0
votes
1answer
87 views

Noise in clustering of high dimensional sparse data

Questions: 1) How to detect noise variables in high dimensional data? 2) Does the method that is presented below make sense? 3) What clustering methods are most insensitive to random variables in ...