Questions tagged [svd]

Singular value decomposition (SVD) of a matrix $\mathbf{A}$ is given by $\mathbf{A} = \mathbf{USV}^\top$ where $\mathbf{U}$ and $\mathbf{V}$ are orthogonal matrices and $\mathbf{S}$ is a diagonal matrix.

Filter by
Sorted by
Tagged with
0
votes
0answers
19 views

Is there any relationship between SVD low rank approximation and DCT low frequency approximation?

Matrix can be approximated by low rank approximation using SVD by using major principle components. On the other hand, if we look at the frequency domain, matrix can also be approximated using DCT (...
1
vote
1answer
81 views

Does SVD provide the best low rank approximation for any matrix regardless of shape?

Wikipedia states (link below) that by the Eckart-Young-Mirsky theorem, the SVD provides the best low rank matrix approximation (on the basis of Frobenius norm of the error matrix) for any matrix A in ...
6
votes
2answers
304 views

What is finite precision arithmetic and how does it affect SVD when computed by computers?

Was reading the paper "DETECTING AND ASSESSING THE PROBLEMS CAUSED BY MULTICOLLINEARITY:A USE OF THE SINGULAR-VALUE DECOMPOSITION" by David Belsley and Virginia Klema. After performing SVD, while ...
1
vote
1answer
39 views

PCA with SVD exercice 23.5 understanding machine learning

In understanding machine learning Shai Sharev-Scwartz and Shai Ben-David exercice 23.5. I would like to use SVD to minimize : $$ \text{argmin}_{W \in \mathbb{R}^{n,d}, U \in \mathbb{R}^{d,n}}{\text{ }...
0
votes
0answers
13 views

Dimension reduction and get a dataset with the number of features that larger than the minimum of the numbers of samples and features

I'm trying to perform dimension reduction with SVD on an image dataset with the shape of roughly (3000,20000) as you see the number of each sample's features is way larger than the number of samples, ...
3
votes
1answer
313 views

Moore Penrose Pseudo-Inverse Fast Algorithm in R [closed]

I want to apply Moore Penrose Pseudo-Inverse on my matrix, which is a 20,000 * 20,000 symmetric matrix with rank 19,999. I found ginv() function from the ...
2
votes
1answer
70 views

Orthonormalization to use closed form Lasso solution

Given the Lasso problem $$ min_\beta (Y-X\beta)^\top(Y-X\beta) \quad s.t. \|\beta\|_1\leq\lambda, $$ and assuming that X is orthonormal such that $X^\top X=I$, we know that the closed form solution ...
3
votes
1answer
503 views

Singular Value Decomposition (SVD) for feature selection

I was reading a paper called "Production Optimization Using Machine Learning in Bakken Shale", and came across an approach I was a bit puzzled by. Unfortunately, the paper is behind a paywall, but I ...
2
votes
2answers
51 views

Least Square Fitted Vector Through SVD Equals to y

Elements of Statistical Learning, p 66 The SVD of the $N \times p$ matrix $X$ has the form $X = UDV^T$ Here $U$ and $V$ are $N \times p$ and $p \times p$ orthogonal matrices, with the ...
2
votes
1answer
46 views

SVD in R recomposition

...
0
votes
1answer
131 views

Projecting New Data onto Existing Principal Axes

Let $\mathbf{X}$ be a dataset of size $n \times d$, where $n$ is the number of samples (days) and $d$ is the number of variables (daily observations). All observations are taken at the same times each ...
13
votes
3answers
1k views

What does it mean when PCA does not produce a reduction in dimensionality?

I am a beginner in PCA, I am trying to apply it on a dataset I have. The features are different geometrical parameters with different units and variability, I do standardize the features matrix by ...
2
votes
0answers
16 views

How to approximate a Hermitian matrix with a transposed cross product of a single matrix?

I have a complex square matrix, and wish to learn latent factors (equally weighted latent factors, so not SVD) from this matrix. Given a Hermitian matrix A of ...
4
votes
1answer
355 views

Multivariate normal distribution transformation

Suppose that $X $ has a multivariate normal distribution $X\sim MVN (\mu, \Sigma) $, How can I transform $X$ into $Z$ so that $Z\sim MVN(\mu, I) $ where $I$ is the identity matrix? For instance, ...
0
votes
0answers
13 views

Intuitions behind Singular Value Decomposing generalizing eigen decomposition [duplicate]

Let M be mxn matrix then SVD of M will be UXW^* (sorry for X, assume summation). Then how does it generalizes eigen decomposition ? Since eigen decomposition is possible for nxn matrix and that are ...
2
votes
0answers
108 views

Row similarity in matrix vs in different factorizations

Suppose an arbitrary $m \times n$ matrix $M$ and the factorizations: Arbitrary: $M = U_a V_a^T$, where $U_a$ is $m \times k$, $V_a$ is $n \times k$ ($k < m,n$), and $rank(U_a)=rank(V_a)=k$. SVD: $...
0
votes
0answers
69 views

Can we always perform SVD on the data matrix before doing high-dimensional logistic regression?

So I'm using lasso logistic regression to classify my data. My data matrix $X$ has dimension $n\times p$ for $p >> n$. As $p$ is on the order of a billion, I expect to face some computational ...
1
vote
0answers
20 views

Principal Component Analysis on Numerical Predictors alone for Dimension Reduction

I'm trying to reduce the number of dimensions for this 'Network Anamoly Detection' dataset: https://www.kaggle.com/anushonkar/network-anamoly-detection The dataset has a total of 40 features out of ...
1
vote
0answers
35 views

Can I combine independent components from different models using PCA?

I have a set of independent components for each subject in my dataset (i.e. an ica model was generated for each subject). The samples used to generate each set of ICs are aligned across subjects, and ...
1
vote
1answer
480 views

What do the matrix (S, U, V) returned by singular value decomposition represent (in terms of variation)?

I believe SVD on a matrix A returns three matrices: U, S, and V. Let's imagine A is a data matrix with training examples/records/whatever you call them as its rows and attributes as its columns. I ...
2
votes
0answers
36 views

Signal Decomposition

I have two time dependent signal sources X & Y. Both can be modeled as having a linear combination of time dependent individual components and common components; so X(t)=a(t)+C(t)+noise, Y(t)=b(t)+...
2
votes
0answers
17 views

Can't Recreate Values for U, S, V from SVD in numpy [duplicate]

To better understand SVD, I'm trying to recreate the values for U, S, and V using straight ...
0
votes
1answer
69 views

Predictive model based on Principal Components when new data has different variables

I built a logistic regression model to classify a corpus of documents. The dependent variable is the type of document (eg A or B) while the dependent variables, because of dimensionality, are the ...
0
votes
0answers
62 views

Can I use the matrix $U$ instead of the matrix $V$ in Principal Component Analysis?

I'm taking Andrew NG's Machine Learning Course and got to the part of Principal Component Analysis. Andrew's implementation of PCA aroused 2 questions for me. 1. Let's say that we have the data ...
0
votes
0answers
339 views

SVD -> PCA -> t-SNE; Does it make sense?

I have a data set of size (4600, 10000). I did L2 normalization at first, then I did the following two steps to visualize it in a lower dimension: ...
1
vote
2answers
270 views

Why is there a reconstruction loss in PCA with orthonormal eigenvectors?

I've already read How to reverse PCA and reconstruct original variables from several principal components? and I understand conceptually and visually why there has to be a reconstruction loss. ...
0
votes
1answer
809 views

How to make a scree plot out of SVD data

After doing a singular value decomposition (SVD) of a data set, I'm left with three matrices: 1. An orthogonal Left Singular Vector (U) 2. diagonal matrix with elements in descending order (S) 3. ...
2
votes
1answer
68 views

Intuition About Principal Component Directions

I am trying to really get a deep understanding of PCA. From my understanding, a principal component is defined as $$\mathbf{z}_k = \phi_{1,k} \mathbf{x}_1 + \ldots + \phi_{p,k} \mathbf{x}_p = \mathbf{...
1
vote
0answers
59 views

SVD versus RSVD

In the so-called incremental SVD used for collaborative filtering: http://www.machinelearning.org/proceedings/icml2007/papers/407.pdf http://www2.research.att.com/~volinsky/papers/ieeecomputer.pdf ...
1
vote
0answers
258 views

Why does kmeans after SVD result in ideal clusters

I am clustering tweets which are related to eye fashion and they are extracted using keywords like mascara, eyeliner, eyeshadow, etc from twitter. I constructed a Tf-idf matrix (tweets x words) ...
0
votes
1answer
217 views

Interpretation of Eigenvalue vs. Singular Value plot

I'm doing some preliminary analysis on the feature matrix for a certain dataset (rows are observations, columns are feature dimensions). I have computed the SVD and PCA decompositions for this matrix ...
2
votes
0answers
24 views

Factorizing a matrix of distributions [closed]

Let's say we have a matrix $X \in \mathbb{R}^{m \times n}$, then the (R-truncated) SVD allows to approximate: $X_{i,j} \approx \sum\limits_{r=1}^{R} \sigma_r \times U_{r,i} \times V_{r, j}$ Now I ...
5
votes
1answer
625 views

Why do PCA and PCoA give the same components but different explained variances?

I'm quite familiar with Principal Component Analysisis, as I use it to study genetic structure. Lately, I was revisiting some of the functions I was using in R (...
1
vote
0answers
483 views

Calculating PCA coefficients using SVD, PCA (sklearn) and Covariance Matrix

I am trying to understand PCA implemented in different methods on python. I am failing to get equal PCA coefficients in each of the methods. By PCA coefficients I mean data projected in the principle ...
1
vote
0answers
37 views

How SVD factorisation -based recomendation algos deal with new user interaction

Classic SVD and SVD++ alogritms generate predictions based on a current known ratings only for known users and known items. But I need to make prediction for some new user on the old items. In the ...
0
votes
1answer
457 views

SVD matrixes do not coincide with Eigen decomposition for covariance matrix [duplicate]

I am comparing the output from the singular value decomposition with the eigendecomposition of the covariance matrix (symmetric matrix). I am expecting that the Eigenvector and a non-diagonal matrix ...
0
votes
1answer
27 views

preparing free text column for regression

I have a column X which contains occupation/profession as an independent variable as free text, which is very much correlated with a continuous dependent variable. What techniques do you usually use ...
0
votes
2answers
85 views

What is the meaning of these principal components?

I have a matrix of data. I computed the principal components of my matrix using SVD (code shown below): subtract mean...then $$[U,S,V] = SVD({\rm matrix})$$ for $V$, which is the principal ...
1
vote
1answer
952 views

SVD PCA reconstruction of data [duplicate]

I have some data about the $\{noise,~ size,~ speed,~ length,~ width\}$ of cars. I have performed SVD, and I want to reconstruct my data using only the first 2 principal components. I subtracted mean ...
0
votes
0answers
61 views

Rank 1 SVD with constraint on U

I need to perform a particular rank 1 decomposition of a sparse matrix $\mathbf{A} \in \mathbb{R}^{n\times n}$. In particular I am looking for the positive vector $\mathbf{u} \in \mathbb{R}^{+n}$ ...
3
votes
1answer
906 views

Why is computing ridge regression with a Cholesky decomposition much quicker than using SVD?

By my understanding, for a matrix with n samples and p features: Ridge regression using Cholesky decomposition takes O(p^3) time Ridge regression using SVD takes O(p^3) time Computing SVD when only ...
3
votes
0answers
212 views

Why is it much quicker to compute ridge regression than regular linear regression?

By my understanding, for a matrix with n samples and p features: Ridge regression using cholesky takes O(p^3) time Ordinary linear regression takes O(p^3) time Singular value decomposition if u, v ...
0
votes
0answers
101 views

Can anyone help me with step by step procedure in MATLAB for missing data imputation using Singular Value Decomposition (SVD [duplicate]

I need step by step procedure for imputing data using SVD in Matlab. Please provide resources if any !
3
votes
1answer
545 views

Reference point in projection axis of SVD (singular value decomposition)

I am watching a YouTube video on SVD, and attempting to recreate some of its examples to better understand the internal machinery of the algorithm. In one of the slides, the instructor mentions that ...
0
votes
0answers
41 views

Additional Property of Singular Value Decomposition

I am new to SVD so forgive me if the question is trivial. Following is my question. If I have two sets of linear equations, Y1 = T1.X Y2 = T2.X where T1 and T2 are mxn rectangular matrices. Now let'...
0
votes
0answers
132 views

Idea behind change of basis and how it relates to projecting your points onto principal components

I would like to clarify if my understanding is correct. In the traditional X-Y coordinate system, our choice of basis vectors are $\vec{i} = (1, 0)$ and $\vec{j} = (0, 1)$ and when you I have a point $...
0
votes
0answers
92 views

SVD for simultaneous row and column reduction of a squared matrix

I have an n x n similarity matrix which I'd like to reduce to a smaller square matrix. I am aware of this answer: How to use SVD for dimensionality reduction to reduce the number of columns (features)...
1
vote
0answers
22 views

Decomposing SVD of this dataset by hand (perspective)

I have the following data matrix $\left[\begin{array}{ccccc} 1 & 1 & 1 & 0 & 0\\ -3 & -3 & -3 & 0 & 0\\ 2 & 2 & 2 & 0 & 0\\ 0 & 0 & 0 & -1 &...
2
votes
0answers
34 views

Is low rank finite-iteration manifold identification possible?

In sparse optimization, I am trying to solve the problem $$ \min_{x\in \mathbb R^{n}} \quad f(x) + \|x\|_1 $$ and at optimality, $x^*$ may be sparse. If I define the sparse manifold as $\mathcal M = ...
2
votes
1answer
460 views

Eigenvalue decomposition/SVD and the filtering perspective

I have been studying the SVD algorithm recently and I can understand how it might be used for compression but I am trying to figure out if there is a perspective of SVD where it can be seen as a low ...

1
2
3 4 5
8