Singular value decomposition (SVD) of a matrix $\mathbf{A}$ is given by $\mathbf{A} = \mathbf{USV}^\top$ where $\mathbf{U}$ and $\mathbf{V}$ are orthogonal matrices and $\mathbf{S}$ is a diagonal matrix.

learn more… | top users | synonyms

0
votes
0answers
14 views

Amount of data needed and hypothesis for SVD

I was looking into the definition of SVD and trying to understand which are the conditions needed to be met in order to be able to use it. Is there any hypothesis concerning the distribution of the ...
1
vote
0answers
36 views

Train & predict probabilities using LDA having multiple collinearities

I am trying to fit an LDA model and predict conditional probabilities of class membership with it. I believe I understand the basic method to do this using the covariance matrix and class means, but ...
0
votes
0answers
36 views

PCA/SVD with datetime fields

I have a dataset or flight data with several columns including delay time, and date of departure. There are several other parameters, and I would like to run some sort of PCA with SVD to see how ...
3
votes
1answer
81 views

How does centering make a difference in PCA (for SVD and eigen decomposition)?

What difference does centering (or de-meaning) your data make for PCA? I've heard that it makes the maths easier or that it prevents the first PC from being dominated by the variables' means, but I ...
2
votes
1answer
80 views

Standard library for Funk SVD or other gradient descent SVD/eigenvalue

I want to get the first few eigenvectors of real symmetric matrices with missing values. Since it has missing values, I won't be able to use the common linear programming techniques, but stochastic ...
0
votes
1answer
48 views

Moving window PCA vs local PCA vs kernel PCA vs rolling PCA

All all these terms mean the same thing? Are there other terms for MWPCA? Are there any decent online references to theory and applications? Which term is most popular (if they are equivalent)? I'm ...
2
votes
0answers
42 views

Confirming an understanding of SVD

I'm trying to sift through the concepts of low-rank approximations, matrix factorizations and SVD. There's a lot of info out there and the rabbit hole is deep so I just want to make sure my high level ...
2
votes
0answers
24 views

LSA projections of documents and terms

I am trying to understand how Latent Semantic Analysis works, reading demonstrations based on singular value decomposition. Let's denote $X$ a $D \times W$ document-term matrix. The $D$ rows of $X$ ...
14
votes
2answers
450 views

Theory behind partial least squares regression

Can anyone recommend a good exposition of the theory behind partial least squares regression (available online) for someone who understands SVD and PCA? I have looked at many sources online and have ...
1
vote
0answers
19 views

What are the pros and cons of applying pointwise mutual information on a word cooccurrence matrix before SVD?

One way to generate word embeddings is as follows: Get a corpora, e.g. "I enjoy flying. I like NLP. I like deep learning." Build the word cooccurrence matrix from it: Perform SVD on $X$, and ...
7
votes
1answer
343 views

What is the intuition behind SVD?

I have read about singular value decomposition (SVD). In almost all textbooks it is mentioned that it factorizes the matrix into three matrices with given specification. But what is the intuition ...
7
votes
1answer
197 views

Updating SVD decomposition after adding one new row to the matrix

Suppose that I have a dense matrix $ \textbf{A}$ of $m \times n$ size, with SVD decomposition $$\mathbf{A}=\mathbf{USV}^\top.$$ In R I can calculate the SVD as ...
0
votes
0answers
53 views

PCA with non-zero mean Gaussian error

Question Re-written: I am trying to apply PCA to a data matrix which has non-zero mean Gaussian error. My data has variables out of which only some are independent and rest are dependent (they are ...
1
vote
0answers
11 views

Interpretation of PARAFAC/TUCKER components for factor mode?

I am working on my thesis at the moment and I'm stuck... I would need to use a multimode-PCA method like Parafac/Tucker-N. My task is to find any deviating structure from the data which is in 3-mode. ...
0
votes
0answers
22 views

Propagation of uncertainty through a linear system of equations - rectangular matrix, pseudo-inverse

I refer you to this post as I have very similar problem: Propagation of uncertainty through a linear system of equations Can the same technique, proposed by Glen_b, be used to find uncertainty in ...
0
votes
0answers
11 views

Get the components from singular value decomposition (SVD) in PCA [duplicate]

I am performing a principal component analysis (PCA) in MATLAB. I have a data matrix $X$ ...
0
votes
0answers
41 views

SVD of gaussian noise

On page 3 of this paper, it is stated that for white gaussian noise signal with variance equal to sigma^2, the sum of the squared singular values is equal to the variance: Now, I suppose the author ...
2
votes
1answer
44 views

Singular value decomposition on RGB images

Is singular value decomposition (SVD) only done for grayscale images? All examples in the literature seem to focus on grayscale images. I was wondering if SVD also makes sense if applied to each of ...
3
votes
1answer
81 views

Low-rank SVD reconstruction and linear projection of the data

Short version: I wonder if the singular value decomposition (SVD) low-rank approximation $X = U_q\Sigma_q V^T_q$ is a projection onto $V_q$ or $V_q^T$? My understanding of the answer: The basic ...
1
vote
1answer
79 views

Percentage of variation in each column explained by each SVD mode

I performed singular value decomposition (SVD) on a data matrix. The mean of each column is zero. One of the scores one can measure is the percentage of the total variation that is explained by each ...
0
votes
0answers
14 views

How ksvd algorithm is considered generalized kmean?

I am trying to understand more details of this paper "KSVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation" proposed a new algorithm called Ksvd and claims it's a ...
0
votes
0answers
35 views

Meaning of SVD plot of $U$ and $V^T$

I am using SVD/PCA for text mining purposes. Having a $(|terms|,|documents|)$ normalized matrix $M$, by applying SVD, I should be able to reduce the dimensionality and just keep the most meaningful ...
0
votes
1answer
71 views

SVD from Matrix formulation to objective function

I'm writing the question to try to complete the circle after reading the 2 other questions on Cross Validated and the link on the third bullet point: What is the objective function of PCA? What ...
0
votes
0answers
44 views

Sequential Singular Value Decomposition

Is there any way (or any instructive paper) to implement singular value decomposition sequentially? i.e. compute $U_{n}$ using $U_{n-1}$ for a matrix $A_n$ such that $A_n = U_n\Sigma_nV_n^T$?
3
votes
1answer
76 views

Definition of orthogonal matrix

I am reading the book Elements of Statistical Learning and trying to understand singular value decomposition (SVD). In particular, what is an orthogonal matrix as it relates to SVD? According to ...
0
votes
0answers
27 views

R - SVD and PC full example [closed]

I would like to understand if I'm using the packages correctly: A complete example would be: I create a random matrix ...
1
vote
0answers
69 views

How to go from sparse matrix to linear regression model (using SVD)?

I am trying to replicate the Kosinski, Stillwell, & Graepel (2013) study about predicting private traits and attributes from Facebook like data for study purposes. First I have admit, however, ...
1
vote
2answers
59 views

Low rank approximation of binary valued matrix

How does one get the low rank approximation of binary matrix? Is the low rank approximation also a binary matrix? Note - Here binary matrix just means that any entry of the matrix can either be 0 or ...
2
votes
2answers
370 views

What fast algorithms exist for computing truncated SVD?

Possibly off topic here, but there exist several (one, two) related questions already. Poking around in the literature (or a google search for Truncated SVD Algorithms) turns up a lot of papers that ...
0
votes
1answer
38 views

Can PCA explained variance be computed from the components (or from SVD matrices)?

I'm looking to use Spark to calculate PCAs. However I need to get the explained variance for each component and the PCAModel class doesn't appear to provide that. Is there a way to calculated the ...
0
votes
0answers
27 views

Too small baseline predictors

I'm trying to implement a recommender system, based on SVD-algorithm. I have a matrix with binary rates, i.e. 0 and 1. This matrix is very sparse. I'm using a formula for learning process: ...
0
votes
1answer
294 views

SVD interpretation of Climate data

I was going through my homework. I have a question to interpret the plot which i obtain from first and second left singular vectors after computing SVD. What can i exactly interpret, is it specific to ...
1
vote
1answer
37 views

Projecting a new entry onto the two largest eigenvectors of a PCA model

I have a bit of problems understanding how PCA and SVD works, as most materials focus on calculating the factors rather than the classification of new entries. In order to provide some context of my ...
0
votes
0answers
41 views

Using SVD or PCA for reducing dimensionality [duplicate]

I have always heard that I can reduce dimensionality of a matrix using SVD. So, I'd like to ask something hypothetically. Suppose that the following matrix A has a high dimensionality and I want to ...
0
votes
0answers
76 views

Sparse features and dimension reduction

Let sparse feature be a feature which values are subsets of some set. For example, the set of countries from which user logged to server is a sparse feature, because for each user we've got the set of ...
2
votes
2answers
66 views

Singular value and eigen-decomposition of a square symmetric matrix should be identical, but differ in sign

As far as I know, singular value decomposition (SVD) and eigendecomposition give the same result for symmetric square matrices. But when I check the results in R, that's not what I see. Please see ...
3
votes
1answer
107 views

PCA on a rank-deficient matrix using SVD of the covariance matrix

I have a high-dimensional data matrix X where sample size is smaller than the variable size. I want to use PCA as a dimensionality reduction method, but I cannot ...
1
vote
1answer
119 views

Latent Semantic Indexing and Data Centering

In PCA it's common to center the data, i.e. preprocess the data matrix such that the columns have zero mean. PCA can be done via SVD, but in this case the data matrix also has to be mean-centered. If ...
0
votes
0answers
20 views

SVD of a large sparse matrix in R [duplicate]

I have a huge matrix with roughly around half a billion entries. However, the matrix is hugely sparse. If I store it as a matrix in R, it will be around 6 gb and if I store it in spare matrix format, ...
1
vote
0answers
208 views

Which SVD matrix do we use for cosine similarity

In Latent Semantic Analysis, we get 3 matrices from the singular value decomposition (SVD), but I am confused - which matrix do we use for cosine similarity?
1
vote
1answer
194 views

What is the relationship between SVD and UV decomposition?

I am aware that both are Matrix decomposition techniques. SVD decomposes the Matrix as $\mathbf C = \mathbf U_1 \mathbf L \mathbf V_1^\top$. UV decomposes the Matrix as $\mathbf C = \mathbf ...
1
vote
2answers
137 views

Treating missing data in voting pattern analysis

I'm trying to analyze voting patterns of Ukraine's parliament deputies. I scraped all the data on their voting during last session. Each data entry has following information: Deputy name, date, bill ...
0
votes
0answers
83 views

Single value decomposition applied to Lee Carter in R

I have a problem regarding the svd in R. It is applied to matrix m, 101 rows by 69 columns, which is the demeaned log mortality rates. When I apply svd(m) (I left nv and nu out), I get d which is a ...
0
votes
0answers
67 views

Getting Rotation Matrix from SVD

I'd like to use SVD to get the rotation of ellipsoid data. When I create ellipsoid data with a known rotation, I can see that there is some kind of relationship between the $V'$ and the original ...
1
vote
0answers
73 views

how to solve weighted low rank approximation problem with diagonal weight matrix

Without weights the low rank approximation problem can be solved in terms of svd of the original matrix.But is there any way to solve the problem following problem $$\min_C{\sum_{i=1}^n \sum_{j=1}^d ...
2
votes
0answers
102 views

Fastest algorithm for rank-one approximation of a matrix [closed]

I need to compute the best (in the sense of sum of squares) rank-one approximation to a matrix, i.e. a one-component SVD, $$ \mathbf{X} \approx \sigma\mathbf{u}\mathbf{v}^T $$ such that $$ ...
0
votes
0answers
76 views

minimization of weighted frobenius norm for pca

So my problem is i like to derive pca solution as the maximum likelihood estimate for the true data.So basically i am assuming that my measured data has two component one is low rank component and ...
0
votes
1answer
126 views

Singular Value decomposition positive components

I am using Singular Value Decomposition (SVD) applied to Singular Spectrum Analysis (SSA) of a timeseries. ...
1
vote
0answers
94 views

Rotation in PCA and Factor Analysis [duplicate]

I want to know what elements are (varimax-)rotated when I rotate after PCA and after Factor Analysis. Let’s assume a standardized data vector $X$ of dimension $N \times q$. In PCA, I have the ...
1
vote
0answers
26 views

PCA using prcomp and princomp in R [duplicate]

I have been trying to do Principal Component Analysis (PCA) via R. The data set is available at https://www.dropbox.com/s/s3jstl8pu1e1xcp/Cars.csv?dl=0 I tried to do PCA via 2 different methods - ...