Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [svd]

Singular value decomposition (SVD) of a matrix $\mathbf{A}$ is given by $\mathbf{A} = \mathbf{USV}^\top$ where $\mathbf{U}$ and $\mathbf{V}$ are orthogonal matrices and $\mathbf{S}$ is a diagonal matrix.

0
votes
1answer
40 views

SVD matrixes do not coincide with Eigen decomposition for covariance matrix [duplicate]

I am comparing the output from the singular value decomposition with the eigendecomposition of the covariance matrix (symmetric matrix). I am expecting that the Eigenvector and a non-diagonal matrix ...
0
votes
1answer
12 views

preparing free text column for regression

I have a column X which contains occupation/profession as an independent variable as free text, which is very much correlated with a continuous dependent variable. What techniques do you usually use ...
0
votes
2answers
51 views

What is the meaning of these principal components?

I have a matrix of data. I computed the principal components of my matrix using SVD (code shown below): subtract mean...then $$[U,S,V] = SVD({\rm matrix})$$ for $V$, which is the principal ...
1
vote
1answer
38 views

SVD PCA reconstruction of data [duplicate]

I have some data about the $\{noise,~ size,~ speed,~ length,~ width\}$ of cars. I have performed SVD, and I want to reconstruct my data using only the first 2 principal components. I subtracted mean ...
0
votes
0answers
25 views

Rank 1 SVD with constraint on U

I need to perform a particular rank 1 decomposition of a sparse matrix $\mathbf{A} \in \mathbb{R}^{n\times n}$. In particular I am looking for the positive vector $\mathbf{u} \in \mathbb{R}^{+n}$ ...
3
votes
1answer
245 views

Why is computing ridge regression with a Cholesky decomposition much quicker than using SVD?

By my understanding, for a matrix with n samples and p features: Ridge regression using Cholesky decomposition takes O(p^3) time Ridge regression using SVD takes O(p^3) time Computing SVD when only ...
2
votes
0answers
26 views

Why is it much quicker to compute ridge regression than regular linear regression?

By my understanding, for a matrix with n samples and p features: Ridge regression using cholesky takes O(p^3) time Ordinary linear regression takes O(p^3) time Singular value decomposition if u, v ...
0
votes
0answers
26 views

Using numpy SVD to calculate factor loadings [duplicate]

I'm doing PCA (Principal Component Analysis) in Python using the numpys Singular Value Decomposition. Effectively extracting the principal components like so: ...
0
votes
0answers
10 views

Meaning of matrix U in LSA?

LSA uses SVD algorithm i.e. the terms-documents matrix $=U \Sigma V^T$ What is the exact meaning of matrix U here: I know that it is a rotation but rotation of what? Is it a concepts space? If yes ...
0
votes
0answers
16 views

Can anyone help me with step by step procedure in MATLAB for missing data imputation using Singular Value Decomposition (SVD [duplicate]

I need step by step procedure for imputing data using SVD in Matlab. Please provide resources if any !
0
votes
0answers
6 views

Explanation of SVD in relation to specific research question (building vector on multiple 3D coordinates)

unfortunately I'm completely out of my comfort zone when it comes to the math involved in my question, appologies in advance. Background: I am evaluating the presicion of electrode placement in the ...
1
vote
1answer
28 views

Reference point in projection axis of SVD (singular value decomposition)

I am watching a YouTube video on SVD, and attempting to recreate some of its examples to better understand the internal machinery of the algorithm. In one of the slides, the instructor mentions that ...
0
votes
0answers
23 views

Additional Property of Singular Value Decomposition

I am new to SVD so forgive me if the question is trivial. Following is my question. If I have two sets of linear equations, Y1 = T1.X Y2 = T2.X where T1 and T2 are mxn rectangular matrices. Now let'...
0
votes
0answers
30 views

Idea behind change of basis and how it relates to projecting your points onto principal components

I would like to clarify if my understanding is correct. In the traditional X-Y coordinate system, our choice of basis vectors are $\vec{i} = (1, 0)$ and $\vec{j} = (0, 1)$ and when you I have a point $...
0
votes
0answers
30 views

SVD for simultaneous row and column reduction of a squared matrix

I have an n x n similarity matrix which I'd like to reduce to a smaller square matrix. I am aware of this answer: How to use SVD for dimensionality reduction to reduce the number of columns (features)...
1
vote
0answers
19 views

Decomposing SVD of this dataset by hand (perspective)

I have the following data matrix $\left[\begin{array}{ccccc} 1 & 1 & 1 & 0 & 0\\ -3 & -3 & -3 & 0 & 0\\ 2 & 2 & 2 & 0 & 0\\ 0 & 0 & 0 & -1 &...
1
vote
0answers
32 views

Is low rank finite-iteration manifold identification possible?

In sparse optimization, I am trying to solve the problem $$ \min_{x\in \mathbb R^{n}} \quad f(x) + \|x\|_1 $$ and at optimality, $x^*$ may be sparse. If I define the sparse manifold as $\mathcal M = ...
0
votes
0answers
23 views

Eigenvalue decomposition/SVD and the filtering perspective

I have been studying the SVD algorithm recently and I can understand how it might be used for compression but I am trying to figure out if there is a perspective of SVD where it can be seen as a low ...
2
votes
0answers
40 views

Why truncated SVD can denoise images

There are a lot of empirical results about that truncated SVD (TSVD) can help denoise the noises of images, but I wonder what is the theoretical support behind that? We know that TSVD is the best low-...
0
votes
0answers
50 views

Does Column ordering matter in QR decomposition?

I am trying to understand if the ordering of columns matters in QR decompsoition. In general it seems that column ordering won't matter. I guess for SVD or any matrix factorization the way columns ...
4
votes
2answers
106 views

Eckart-Young-Mirsky theorem: rank $≤k$ or rank $=k$

The Eckart-Young-Mirsky theorem is sometimes stated with rank $\le k$ and sometimes with rank $= k$. Why? More specifically, given a matrix $X \in \mathbb{R}^{n \times d}$, and a natural number $k \...
1
vote
0answers
40 views

Extracting latent vectors from autoencoder similar to SVD

I have read that there is an equivalency between a linear autoencoder and performing SVD. SVD can be used in collaborative filtering, for example, factorization of a user-movies matrix $\mathbf{M}$ ...
0
votes
1answer
206 views

PCA in psych package with more columns than rows

Why is it impossible to do a PCA in R using principal from psych package without warnings with a matrix, which has more columns ...
1
vote
0answers
29 views

PCA for three-dimensional linear fit on time-resolved trajectory

I study the behavior of organisms that are able of self-locomotion and that show directed motion toward one another. This directed motion occurs through the detection of chemical trails released by ...
3
votes
1answer
59 views

Derivation of Procrustes transformation

I'm curious about the derivation for the Procrustes transformation. I'm following ESLII: see figure 14.25 and problem 14.8 (Procrustes distance with scaling). Given matrices $\mathbf{X}_1$ and $\...
1
vote
0answers
51 views

matrix factorization with non-negative constraint only on one of the factors

I have a 2D spectral data time series with a wavelength dimension and a time dimension, and I'd like to decompose it to the time evolution ($SV^T$ for SVD and $H$ for NNMF) of several spectral ...
2
votes
0answers
38 views

Is there any sort of higher-order SVD (quadratic and above) for dimensionality reduction?

X-Posted on math.stackexchange, apologies, though I thought this was equally relevant to both communities. I'm wondering if there exists any higher-order SVD for dimensionality reduction. Note that ...
0
votes
0answers
86 views

How SVD is used for dimensionality reduction? [duplicate]

Given a $M \times N$ data matrix $ D = (x_1, x_2, \cdots, x_M)^{T}$. Applying singular value decomposition to $D$ yields $$D = USV^{T} = (u_1, u_2, \cdots, u_M) \begin{pmatrix} s_1 &&0\\...
0
votes
0answers
112 views

Why is the size of fitted truncated svd model is so big?

I have a dataset with tfidf matrix of shape (200000, 565000). I am fitting truncated svd of 500 dimensions from sklearn onto it and pickling the resulting svd object for later use. The pickle file is ...
4
votes
2answers
140 views

The Curse of high Dimension And Distance

For extracting features from video frames (2 sample/sec) I use keras framework in python and load VGG16 that input size is (150,150,3) and output size is (4,4,512). After the feature extraction step I ...
5
votes
1answer
72 views

Evaluate approximation of PCA from randomized algorithms

I have been comparing different PCA implementations (some via explicit calculation of the covariance matrix, some with randomized/truncated SVD) in terms of speed, and now wanted to compare how good ...
2
votes
1answer
97 views

Latent variable and Factor analysis ICA

While I was going through the factor analysis for Independent component analysis, I got stuck in one statement. How does it come to co-variance of S* is I? Is A* =ART ? Following is what I was going ...
0
votes
0answers
59 views

How to choose reduced features using SVD from a dataset?

How svd reduces features from a matrix. suppose, Matrix A(m,n). if we apply svd to A then we will find matrix U(m,m), S(m,n), v(n,n) matrix. S is the strength/ diagonal matrix as I know. We can ignore ...
1
vote
1answer
186 views

Orthogonality as found by the Gram-Schmidt process vs. uncorrelated basis vectors

I have a data matrix $Y$ of size $n \times p$, a basis vector in $\mathbb{R}^p$ $v_1$, and a potential basis vector in $\mathbb{R}^p$ $v_2'$. Then, if I use the Gram-Schmidt process on $[v_1, v_2']$ ...
0
votes
0answers
22 views

How to return the variance size of a recommendation system using SVD

One problem that many people have when making a recommendation system is the reasonableness of the suggestion / prediction, so I wanted to know how we can calculate the variance size of a generic ...
2
votes
1answer
46 views

Has anyone seen Gibbs phenomenon in SVD?

I read the notes on online about Regularized matrix computation. It said The truncated SVD solution has “ringing,” e.g., Gibbs’s phenomenon in truncated Fourier series I haven't seen any work ...
4
votes
2answers
3k views

How does eigenvalues measure variance along the principal components in PCA? [duplicate]

I understand that eigenvalues measure variance along the principal components. Questions How are eigenvalues and variance same for PCA? What is the intuition behind this being the same? What is ...
0
votes
0answers
20 views

Confused about how to interpret principle components [duplicate]

I think I understand how PCA works. In summary... I have a set of mean-deviated observations. The covariance matrix $S$ for my observations is not diagonal, so for some reason it's hard to interpret. ...
0
votes
1answer
261 views

Relationship between Alternating Least Squares and SVD

I have been assuming that ALS is simply an alternative algorithm for doing matrix decomposition that is more efficient, but in the end produces the same $U$ & $V$ matrices that SVD does. Is this ...
0
votes
0answers
114 views

Methods for selecting the required n_components for TruncatedSVD?

Methods for selecting the required n_components for TruncatedSVD? I found this (https://chrisalbon.com/machine_learning/feature_engineering/...
2
votes
1answer
372 views

PCA - Reconstruction from a “clean” set of eigenvectors?

This is a question related to the explanation here on how to reconstruct data from PCs found here: How to reverse PCA and reconstruct original variables from several principal components? I have two ...
0
votes
0answers
92 views

Singular value decomposition

Can singular value decomposition used to impute missing values in highly nonlinear process under multiple input and multiple output behavior?
1
vote
2answers
777 views

Explaining dimensionality reduction using SVD (without reference to PCA)

I have seen Dimensionality reduction mentioned as one of the practical usages of SVD. However, the explanation for me has always been Let me find the directions in which the variance of the data is ...
0
votes
1answer
165 views

TruncatedSVD always reduces dataset to 1D

I know that I have a large sparse matrix which I'm using TruncatedSVD to condense into a smaller number of dimensions. Here is my code: ...
1
vote
0answers
52 views

Understanding Diagonal Matrix of SVD [closed]

I have a matrix A with dimensions 4x3. I performed SVD on the matrix using numpy (np.linalg.svd) on matrix A. The output dimensions of U, V, and S are (4x4),(3x3),(3,). So V comprise orthonormal ...
-1
votes
0answers
95 views

Latent Semantic Indexing

I watched Videos on LSA and have a litte trouble to understand, which parts of the SVD are used. This is the Video I have trouble with. Starting with 20:15min the guy uses only the train.irlba$v. ...
0
votes
0answers
103 views

How do I assign meaning or give names to the latent variables in exploratory factor analysis?

I read the book Latent Variable Models, and in the chapter dealing with exploratory factor analysis the author shows a way to learn latent variables (factors) from ...
0
votes
0answers
228 views

What if my kNN model performs well on PCA data even if I only keep 30% of the variance?

I have a data set with 10 continuous and 1000 categorical (binary) features - as a result the data set is very sparse. Each observation belongs to one of two classes. I perform Principal Component ...
0
votes
1answer
274 views

What is the correct way to scale data, apply PCA and fit a Multivariate Normal Distribution for anomaly detection?

I want to train an anomaly detection model in python. I have a training data set with some 30,000 observations, 700 of which are anomalies, and I can distinguish between normal and anomalous cases (I ...
1
vote
0answers
403 views

Imputing missing values and SVD

Similar questions have been asked a lot of times but I have not found an answer that gives an intuitive explanation as to why this works. For reference I have read the answers here and here. As I ...