Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [svd]

Singular value decomposition (SVD) of a matrix $\mathbf{A}$ is given by $\mathbf{A} = \mathbf{USV}^\top$ where $\mathbf{U}$ and $\mathbf{V}$ are orthogonal matrices and $\mathbf{S}$ is a diagonal matrix.

0
votes
0answers
18 views

Additional Property of Singular Value Decomposition

I am new to SVD so forgive me if the question is trivial. Following is my question. If I have two sets of linear equations, Y1 = T1.X Y2 = T2.X where T1 and T2 are mxn rectangular matrices. Now let'...
0
votes
0answers
27 views

Idea behind change of basis and how it relates to projecting your points onto principal components

I would like to clarify if my understanding is correct. In the traditional X-Y coordinate system, our choice of basis vectors are $\vec{i} = (1, 0)$ and $\vec{j} = (0, 1)$ and when you I have a point $...
0
votes
0answers
26 views

SVD for simultaneous row and column reduction of a squared matrix

I have an n x n similarity matrix which I'd like to reduce to a smaller square matrix. I am aware of this answer: How to use SVD for dimensionality reduction to reduce the number of columns (features)...
1
vote
0answers
19 views

Decomposing SVD of this dataset by hand (perspective)

I have the following data matrix $\left[\begin{array}{ccccc} 1 & 1 & 1 & 0 & 0\\ -3 & -3 & -3 & 0 & 0\\ 2 & 2 & 2 & 0 & 0\\ 0 & 0 & 0 & -1 &...
1
vote
0answers
32 views

Is low rank finite-iteration manifold identification possible?

In sparse optimization, I am trying to solve the problem $$ \min_{x\in \mathbb R^{n}} \quad f(x) + \|x\|_1 $$ and at optimality, $x^*$ may be sparse. If I define the sparse manifold as $\mathcal M = ...
0
votes
0answers
16 views

Eigenvalue decomposition/SVD and the filtering perspective

I have been studying the SVD algorithm recently and I can understand how it might be used for compression but I am trying to figure out if there is a perspective of SVD where it can be seen as a low ...
2
votes
0answers
32 views

Why truncated SVD can denoise images

There are a lot of empirical results about that truncated SVD (TSVD) can help denoise the noises of images, but I wonder what is the theoretical support behind that? We know that TSVD is the best low-...
0
votes
0answers
32 views

Does Column ordering matter in QR decomposition?

I am trying to understand if the ordering of columns matters in QR decompsoition. In general it seems that column ordering won't matter. I guess for SVD or any matrix factorization the way columns ...
0
votes
0answers
13 views

Signs in SPSS's PCA with rotations with the FACTOR algorithm

I am trying to reproduce the results of the PCA with rotations from SPSS in python. But there is some information I didn't find in their documentation. I am trying to do the PCA like in the FACTOR ...
4
votes
2answers
94 views

Eckart-Young-Mirsky theorem: rank $≤k$ or rank $=k$

The Eckart-Young-Mirsky theorem is sometimes stated with rank $\le k$ and sometimes with rank $= k$. Why? More specifically, given a matrix $X \in \mathbb{R}^{n \times d}$, and a natural number $k \...
1
vote
0answers
29 views

Extracting latent vectors from autoencoder similar to SVD

I have read that there is an equivalency between a linear autoencoder and performing SVD. SVD can be used in collaborative filtering, for example, factorization of a user-movies matrix $\mathbf{M}$ ...
0
votes
1answer
146 views

PCA in psych package with more columns than rows

Why is it impossible to do a PCA in R using principal from psych package without warnings with a matrix, which has more columns ...
1
vote
0answers
26 views

PCA for three-dimensional linear fit on time-resolved trajectory

I study the behavior of organisms that are able of self-locomotion and that show directed motion toward one another. This directed motion occurs through the detection of chemical trails released by ...
3
votes
1answer
51 views

Derivation of Procrustes transformation

I'm curious about the derivation for the Procrustes transformation. I'm following ESLII: see figure 14.25 and problem 14.8 (Procrustes distance with scaling). Given matrices $\mathbf{X}_1$ and $\...
1
vote
0answers
42 views

matrix factorization with non-negative constraint only on one of the factors

I have a 2D spectral data time series with a wavelength dimension and a time dimension, and I'd like to decompose it to the time evolution ($SV^T$ for SVD and $H$ for NNMF) of several spectral ...
2
votes
0answers
34 views

Is there any sort of higher-order SVD (quadratic and above) for dimensionality reduction?

X-Posted on math.stackexchange, apologies, though I thought this was equally relevant to both communities. I'm wondering if there exists any higher-order SVD for dimensionality reduction. Note that ...
0
votes
0answers
85 views

How SVD is used for dimensionality reduction? [duplicate]

Given a $M \times N$ data matrix $ D = (x_1, x_2, \cdots, x_M)^{T}$. Applying singular value decomposition to $D$ yields $$D = USV^{T} = (u_1, u_2, \cdots, u_M) \begin{pmatrix} s_1 &&0\\...
0
votes
0answers
85 views

Why is the size of fitted truncated svd model is so big?

I have a dataset with tfidf matrix of shape (200000, 565000). I am fitting truncated svd of 500 dimensions from sklearn onto it and pickling the resulting svd object for later use. The pickle file is ...
4
votes
2answers
119 views

The Curse of high Dimension And Distance

For extracting features from video frames (2 sample/sec) I use keras framework in python and load VGG16 that input size is (150,150,3) and output size is (4,4,512). After the feature extraction step I ...
5
votes
1answer
66 views

Evaluate approximation of PCA from randomized algorithms

I have been comparing different PCA implementations (some via explicit calculation of the covariance matrix, some with randomized/truncated SVD) in terms of speed, and now wanted to compare how good ...
2
votes
1answer
90 views

Latent variable and Factor analysis ICA

While I was going through the factor analysis for Independent component analysis, I got stuck in one statement. How does it come to co-variance of S* is I? Is A* =ART ? Following is what I was going ...
0
votes
0answers
51 views

How to choose reduced features using SVD from a dataset?

How svd reduces features from a matrix. suppose, Matrix A(m,n). if we apply svd to A then we will find matrix U(m,m), S(m,n), v(n,n) matrix. S is the strength/ diagonal matrix as I know. We can ignore ...
1
vote
1answer
110 views

Orthogonality as found by the Gram-Schmidt process vs. uncorrelated basis vectors

I have a data matrix $Y$ of size $n \times p$, a basis vector in $\mathbb{R}^p$ $v_1$, and a potential basis vector in $\mathbb{R}^p$ $v_2'$. Then, if I use the Gram-Schmidt process on $[v_1, v_2']$ ...
0
votes
0answers
15 views

How to return the variance size of a recommendation system using SVD

One problem that many people have when making a recommendation system is the reasonableness of the suggestion / prediction, so I wanted to know how we can calculate the variance size of a generic ...
2
votes
1answer
44 views

Has anyone seen Gibbs phenomenon in SVD?

I read the notes on online about Regularized matrix computation. It said The truncated SVD solution has “ringing,” e.g., Gibbs’s phenomenon in truncated Fourier series I haven't seen any work ...
4
votes
2answers
2k views

How does eigenvalues measure variance along the principal components in PCA? [duplicate]

I understand that eigenvalues measure variance along the principal components. Questions How are eigenvalues and variance same for PCA? What is the intuition behind this being the same? What is ...
0
votes
0answers
18 views

Confused about how to interpret principle components [duplicate]

I think I understand how PCA works. In summary... I have a set of mean-deviated observations. The covariance matrix $S$ for my observations is not diagonal, so for some reason it's hard to interpret. ...
0
votes
1answer
196 views

Relationship between Alternating Least Squares and SVD

I have been assuming that ALS is simply an alternative algorithm for doing matrix decomposition that is more efficient, but in the end produces the same $U$ & $V$ matrices that SVD does. Is this ...
0
votes
0answers
86 views

Methods for selecting the required n_components for TruncatedSVD?

Methods for selecting the required n_components for TruncatedSVD? I found this (https://chrisalbon.com/machine_learning/feature_engineering/...
2
votes
1answer
257 views

PCA - Reconstruction from a “clean” set of eigenvectors?

This is a question related to the explanation here on how to reconstruct data from PCs found here: How to reverse PCA and reconstruct original variables from several principal components? I have two ...
0
votes
0answers
56 views

Singular value decomposition

Can singular value decomposition used to impute missing values in highly nonlinear process under multiple input and multiple output behavior?
1
vote
2answers
542 views

Explaining dimensionality reduction using SVD (without reference to PCA)

I have seen Dimensionality reduction mentioned as one of the practical usages of SVD. However, the explanation for me has always been Let me find the directions in which the variance of the data is ...
0
votes
1answer
139 views

TruncatedSVD always reduces dataset to 1D

I know that I have a large sparse matrix which I'm using TruncatedSVD to condense into a smaller number of dimensions. Here is my code: ...
1
vote
0answers
47 views

Understanding Diagonal Matrix of SVD [closed]

I have a matrix A with dimensions 4x3. I performed SVD on the matrix using numpy (np.linalg.svd) on matrix A. The output dimensions of U, V, and S are (4x4),(3x3),(3,). So V comprise orthonormal ...
0
votes
0answers
84 views

Latent Semantic Indexing

I watched Videos on LSA and have a litte trouble to understand, which parts of the SVD are used. This is the Video I have trouble with. Starting with 20:15min the guy uses only the train.irlba$v. ...
0
votes
0answers
80 views

How do I assign meaning or give names to the latent variables in exploratory factor analysis?

I read the book Latent Variable Models, and in the chapter dealing with exploratory factor analysis the author shows a way to learn latent variables (factors) from ...
0
votes
0answers
187 views

What if my kNN model performs well on PCA data even if I only keep 30% of the variance?

I have a data set with 10 continuous and 1000 categorical (binary) features - as a result the data set is very sparse. Each observation belongs to one of two classes. I perform Principal Component ...
0
votes
1answer
237 views

What is the correct way to scale data, apply PCA and fit a Multivariate Normal Distribution for anomaly detection?

I want to train an anomaly detection model in python. I have a training data set with some 30,000 observations, 700 of which are anomalies, and I can distinguish between normal and anomalous cases (I ...
1
vote
0answers
357 views

Imputing missing values and SVD

Similar questions have been asked a lot of times but I have not found an answer that gives an intuitive explanation as to why this works. For reference I have read the answers here and here. As I ...
1
vote
1answer
71 views

For an eigenvector v, is it always true that v^tv =1?

I am reading this link PCA which is a very insightful tutorial however, in this tutorial, the author mentioned a constraint on PCA: $C^{T}C=1$ when we look at eigenvalue/eigenvector definition $...
5
votes
2answers
213 views

Find a matrix with orthonormal columns with minimal Frobenius distance to a given matrix

Let $\mathbf A$ be an arbitrary $n \times m$ matrix with $n \ge m$. I want to find $\mathbf X$ of the same size with orthonormal columns that minimizes the Frobenius norm of the difference between $\...
1
vote
0answers
197 views

SVD item similarity calculation

I am performing SVD on a rating matrix of Users and Items and I get 3 matrices out of which Vt provides latent feature for items. How do I compute similarities between a pair of items using these ...
1
vote
0answers
23 views

Theoretical form of the rank of the low rank approximation matrix

The low-rank matrix approximation problem involves finding of a rank k version of a m × n matrix A, labeled $A_k$, such that $A_k$ is as ”close” as possible to the best SVD approximation version of A ...
5
votes
1answer
463 views

For symmetric matrices, is the Cholesky decomposition better than the SVD? [closed]

I am inverting a sparse, symmetric, ill-conditioned matrix. I have used both SVD and the LDL decomposition. I find that my results are better with the latter. Why? I understand that LDL ...
3
votes
1answer
485 views

Whitening/Decorrelation - why does it work?

Given some whitening transform, we change some vectors $\textbf{x}$, where features are correlated, into some vector $\textbf{y}$, where components are uncorrelated. Then we run some learning ...
0
votes
1answer
34 views

show that minimising the distance to approximated vector is equivalent to selecting features with the most variance

First off, a little disclaimer: I am basing the question on my own interpretation of the problem. It's well possible that I am mistaken. Setup: We are given: feature vectors $x_1,\ldots, x_k$ with ...
1
vote
0answers
57 views

Interpreting Sign of Loadings in Truncated SVD

I'm performing Truncated SVD on a word co-occurrence matrix for topic extraction. I'm using the scikit-learn implementation. I have read conflicting ideas on how to interpret the sign of the ...
1
vote
1answer
586 views

SVD decomposition and original matrix are not equal [closed]

When l compute the SVD of my matrix x as defined in kernel_hist_to_SVD(). The resulted decomposition is not equal (approximatively) to the original matrix k. Here ...
1
vote
1answer
1k views

Which features to include for Truncated SVD?

I have a dataset of ~31000 8k-filings (ad-hoc announcements from companies listed on the stock exchange). Every document consists of a string (the actual filing, stemmed and stopwords removed) and a ...
2
votes
0answers
121 views

Why PMI + SVD works for similarities arithmetics?

Recently Julia Silge blogged here and here, quoting blog entry by Chris Moody, who suggested that the similarities arithmetic in word2vec can be approximated by using PMI indexes followed by SVD ...