SVD refers to the singular value decomposition of a matrix. Given an r x c matrix A, the SVD is given by $\mathbf{A} = \mathbf{USV'}$ where $\mathbf{U}$ and $\mathbf{V} are orthogonal matrices and S is a diagonal matrix.
2
votes
0answers
21 views
Prediction using SVD and Fisher's linear discriminant
Where can I get an explanation of the procedure used when making a prediction using SVD?
Let me elaborate a bit more. Suppose you have data in a matrix $A$ containing two classes. In particular, you ...
4
votes
1answer
76 views
Interpreting matrices of SVD in practical applications
I have a question regarding the interpretation of the different matrices produced by singular value decomposition.
Suppose a mxn matrix $A$ containing n images of m pixels. So each column of this ...
0
votes
0answers
38 views
Principal Component Analysis
I am a bit new to this. But I needed a clarification. I am creating a module in MATLAB that performs PCA on various data sets with varying number of explanatory and response variables. Depending on ...
0
votes
1answer
29 views
Noise in clustering of high dimensional sparse data
Questions:
1) How to detect noise variables in high dimensional data?
2) Does the method that is presented below make sense?
3) What clustering methods are most insensitive to random variables in ...
-1
votes
1answer
60 views
Singular Value Decomposition and PCR
Can anyone guide me to understand the relation between Singular Value Decomposition (SVD) and Principle Component Regression (PCR)? I know that we can construct the principle components (PCs) using ...
0
votes
0answers
26 views
SVD in movie recommendation
So assuming that there are a 5 X 6 matrix that records a rating for six users for 5 movies. I have computed SVD. However, suppose I add another movie so now it becomes 6 X 6 matrix. But I only have a ...
0
votes
0answers
63 views
Ridge regression on subset of variables using SVD
I am trying to figure out an algorithm using singular value decomposition to run a modification of ridge regression in which only some of the variables are penalized.
I want the output to match the ...
0
votes
0answers
349 views
Finding the maximum contributer variables by applying the SVD - should the absolute values be used or just the maximum value?
My question: When doing SVD analysis, you can extract the maximum contributors to each component from the V matrix - does the sign of the the component matter?
Background
I'm currently enjoying the ...
0
votes
0answers
103 views
Singular value decompostion covariance matrix (numerical recipes)
I am trying to implement singular value decomposition in C. I am using routines svdfit and svdvar from Numerical Recipes.
The results from svdfit seem to be correct, but the results from svdvar are ...
3
votes
1answer
84 views
I have GBs of Event-Based Data. How do I figure out causation?
I have a lot of event-based data about users of our website. For example, data in the format (verb, timestamp). There's about 10 or so different verbs (call them A, B, C, etc).
I'm interested in ...
0
votes
1answer
80 views
What is a “thin” svd?
I've seen lots of mention of this in the implementation of SVDs in various programming environments. What does it actually mean?
8
votes
1answer
357 views
For a random matrix, shouldn't a SVD explain nothing at all? What am I doing wrong?
If I construct a 2-D matrix composed entirely of random data, I would expect the PCA and SVD components to essentially explain nothing.
Instead, it seems like the the first SVD column appears to ...
2
votes
1answer
206 views
Singular value decomposition procedure in R
I'm trying to follow Prof Strang exercise
but I have a problem with the signs of the resultant matrix.
He also had a problem with the signs during the exercise, so I have no way to find what is the ...
0
votes
1answer
182 views
Solving PCA with correlation matrix of a dataset and its singular value decomposition
Suppose I have a $d \times n$ matrix $\mathbf X$ (each entry point has $d$ dimensions) and after some manipulation of data (i.e. summarizing the data $\mathbf X$) I get its $d \times d$ symmetric, ...
2
votes
0answers
58 views
How would you preprocess data for SVD?
I am computing SVD on a matrix which is the empirical version of $E[XY^{\top}]$ for some $X \in \mathbb{R}^{m \times 1}$ and $Y \in \mathbb{R}^{n \times 1}$.
I am wondering if there are standard ways ...
2
votes
1answer
438 views
PCA and SVD, R prcomp
I am trying to replicate Shalizi's NY Times PCA example found in his Advanced Data Analysis with an Elementary Point of View book. I found sample code and data here
...
1
vote
1answer
59 views
Adversarial noise in PCA
PCA is known to be quite sensitive to outlier noise (and this is why several Robust PCA techniques exists.) However, I am looking for a concrete example of sensitivity of PCA to adversarial noise that ...
0
votes
0answers
87 views
PCA and using SVD [closed]
Suppose we have a data matrix $\textbf{X}$. The SVD of $\textbf{X}$ would be $\textbf{X} = \textbf{W} \Sigma \textbf{V}^{T}$ where $\textbf{W}$ is the matrix of eigenvectors of the covariance matrix ...
3
votes
1answer
184 views
How to explain the connection between SVD and clustering?
Is there an intuitive explanation for how SVD is related to co-clustering when performing SVD on a covariance matrix?
(i.e. the SVD is performed on the matrix $E[X Y^{\top}]$ where $X \in ...
1
vote
0answers
101 views
Rank-one nonnegative matrix factorization
For non-negative matrix factorization with Frobenius norm:
$\min\limits_{U\in\mathbb{R}_+^{m\times r}, V\in\mathbb{R}_+^{r\times n}}||A-UV||_2^2$, $A\in\mathbb{R}_+^{m\times n}$
$r=1$ is a very ...
4
votes
1answer
572 views
SVD on a 65 million by 3.4 million sparse matrix
What is the best way compute SVD on a very large positive matrix (65M x 3.4M) where data is extremely sparse?
sparse: less than 0.1% of the matrix is non zero.
best way:
will fit into memory (I ...
2
votes
1answer
204 views
A parellel between LSA and pLSA
In the original paper of pLSA the author, Thomas Hoffman, draw a parallel between pLSA and LSA data structures that I would like to discuss with you.
Background:
Taking inspiration the Information ...
2
votes
0answers
186 views
Clear description of PCA using SVD
After reading thousands of articles on PCA and SVD, using them in a number of programming frameworks and even implementing similar techniques (like Random Indexing) I found out that I still have ...
7
votes
1answer
961 views
Dimensionality reduction (SVD or PCA) on a large, sparse matrix
I have a large, sparse Matrix of features I would like to use in a machine learning algorithm:
...
0
votes
0answers
151 views
Confusion about singular value decomposition in LSA
I had a question about separate ideas suggested by two different sources. (you don't necessarily have to go through the sources for this question)
Source 1- ...
4
votes
0answers
667 views
What is the correct way to calculate the explained variance of each EOF as calculated from a gappy data set?
I am trying to determine the correct amount of variance explained by each mode of an Empirical Orthogonal Function (EOF) analysis (similar to "PCA") as applied to a gappy data set. (i.e., containing ...
9
votes
1answer
505 views
Why are eigen and svd decompositions of a covariance matrix based on sparse data yielding different results?
I am trying to decompose a covariance matrix based on a sparse / gappy data set. I'm noticing that the sum of lambda (explained variance), as calculated with svd, ...
4
votes
2answers
424 views
What happens when you apply SVD to a collaborative filtering problem? What is the difference between the two?
In Collaborative filtering, we have values that are not filled in. Suppose a user did not watch a movie then we have to put an 'na' in there.
If I am going to take an SVD of this matrix, then I have ...
6
votes
1answer
377 views
SVD of a matrix with missing values
Suppose I have a Netflix-style recommendation matrix, and I want to build a model that predicts potential future movie ratings for a given user. Using Simon Funk's approach, one would use stochastic ...
11
votes
0answers
305 views
SVD of correlated matrix should be additive but doesn't appear to be
I'm just trying to replicate a claim made in the following paper, Finding Correlated Biclusters from Gene Expression Data, which is:
Proposition 4. If $X_{IJ}=R_{I}C^{T}_{J}$. then we have:
...
5
votes
3answers
542 views
PCA of non-Gaussian data
I have a couple of quick questions about PCA:
Does the PCA assume that the dataset is Gaussian?
What happens when I apply a PCA to inherently non-linear data?
Given a dataset, the process is to ...
5
votes
2answers
1k views
How do I use the SVD in collaborative filtering?
I'm a bit confused with how the SVD is used in collaborative filtering. Suppose I have a social graph, and I build an adjacency matrix from the edges, then take an SVD (let's forget about ...
7
votes
3answers
546 views
PCA when the dimensionality is greater than the number of samples
I've come across a scenario where I have 10 signals/person for 10 people (so 100 samples) containing 14000 data points (dimensions) that I need to pass to a classifier. I would like to reduce the ...
1
vote
0answers
132 views
Projecting factors, forecast, using SVD and VAR
I'm curious whether something I tried makes sense statistically...
I took a pile of time series inputs and performed an SVD. I want to predict variable Y on the basis of its own time series, and the ...
4
votes
1answer
173 views
SVD of a data matrix after smoothing
Let's say I have a (n x m) centered data matrix $A$ with SVD $A = U \Sigma V^{T}$.
For example, m=50 columns (measurements) that are spectra with n=100 different frequencies. The matrix is centered ...
2
votes
3answers
235 views
How to cluster survey data?
I have designed a rather long (250 Qn) survey designed to uncover user clusters. The questions are such that the pattern of answering should elicit user clusters, but I am having trouble uncovering ...
3
votes
1answer
380 views
Sphering data with SVD components of covariance matrix
The Elements of Statistical Learning says on page 113:
Sphere the data with respect to the common covariance estimates $\hat{\Sigma}$:
$X^* \leftarrow D^{-1/2}U^TX$ where $\hat{\Sigma} = ...
3
votes
1answer
134 views
Biplots: Adding supplementry points to a biplot which only use a subset of the variables
I have a matrix X where the rows denote the cases and the columns the variables. I use a standard row-metric preserving biplot in order to represent the cases in a subspace. If I wanted to add a ...
0
votes
0answers
53 views
How to enlarge singular values of a matrix?
I have a matrix that gives singular values which are quite small. I would like to perturb it in a way that keeps it close to the original matrix, only it has larger singular values. For a square ...
1
vote
2answers
438 views
Java code for thin SVD
I am looking for a library/package/code to do thin SVD in Java. I found a good implementation of SVD in the COLT package for java, but that's not sufficiently robust with large sparse matrices when ...
0
votes
2answers
2k views
Finding matrix eigenvectors using QR decomposition
First, a general linear algebra question: Can a matrix have more than one set of (unit size) eigenvectors? From a different angle: Is it possible that different decomposition methods/algorithms (QR, ...
4
votes
1answer
245 views
Does the sign of the principal component become meaningless with centered variables?
Does the sign of the vectors returned by PCA have any significance after I center my variables?
4
votes
3answers
941 views
What is the point of singular value decomposition?
I don't understand why reduction in dimension is important. What is the benefit of taking some data and reducing their dimension?
5
votes
2answers
237 views
One component in PCA is always the mean vector in two-dimensions but not three
I've been testing PCA via SVD to decompose a simple time series data matrix, $X$. I have two signals $x_1(t)$ and $x_2(t)$ in a data matrix where $M$ rows represents each timepoint sample and each ...
1
vote
0answers
65 views
Comparing original variables with characteristic values of diagonalized variance-covariance matrix
If I have a reference data set comprising repeated measurements of 3 variables of a system in state $A$. Given new observations of these variables for a different system I would like to classify ...
5
votes
2answers
502 views
Simulated annealing and k-means
One of my problems http://stackoverflow.com/questions/7783933/clustering-data-outputs-irregular-plot-graph suffers from the curse of dimensionality, which also makes it infeasible for exhaustive ...
0
votes
0answers
111 views
princomp relevant information matrix
I want to get $ relative error = \frac{x-x_{0}}{x}$ between the results of R and mine with pca, to do so I am doing
...
12
votes
1answer
352 views
Eigenfunctions of an adjacency matrix of a time series?
Consider a simple time series:
> tp <- seq_len(10)
> tp
[1] 1 2 3 4 5 6 7 8 9 10
we can compute an adjacency matrix for this time series ...
7
votes
2answers
374 views
When do we combine dimensionality reduction with clustering?
I am trying to perform document-level clustering. I constructed the term-document frequency matrix and I am trying to cluster these high dimensional vectors using k-means. Instead of directly ...
2
votes
1answer
632 views
How to draw a scree plot in python?
I am using singular vector decomposition on a matrix and obtaining the U, S and Vt matrices. At this point, I am trying to choose a threshold for the number of dimensions to retain. I was suggested to ...
