Questions tagged [svd]

Singular value decomposition (SVD) of a matrix $\mathbf{A}$ is given by $\mathbf{A} = \mathbf{USV}^\top$ where $\mathbf{U}$ and $\mathbf{V}$ are orthogonal matrices and $\mathbf{S}$ is a diagonal matrix.

Filter by
Sorted by
Tagged with
2
votes
0answers
9 views

How to approximate a Hermitian matrix with a transposed cross product of a single matrix?

I have a complex square matrix, and wish to learn latent factors (equally weighted latent factors, so not SVD) from this matrix. Given a Hermitian matrix A of ...
0
votes
0answers
16 views

Finding the optimal number of latent factors in Symmetric Singular Value Decomposition

Consider symmetric Singular Value Decomposition of a symmetric unitary matrix A into U, D, ...
0
votes
0answers
16 views

Subtracting PCA projection from original data

I'm trying to implement Anomaly Detection algorythm from this article https://iopscience.iop.org/article/10.1088/1742-6596/1069/1/012072/pdf at my work. So I have big questions for me: Did I ...
4
votes
1answer
160 views

Multivariate normal distribution transformation

Suppose that $X $ has a multivariate normal distribution $X\sim MVN (\mu, \Sigma) $, How can I transform $X$ into $Z$ so that $Z\sim MVN(\mu, I) $ where $I$ is the identity matrix? For instance, ...
0
votes
0answers
11 views

Intuitions behind Singular Value Decomposing generalizing eigen decomposition [duplicate]

Let M be mxn matrix then SVD of M will be UXW^* (sorry for X, assume summation). Then how does it generalizes eigen decomposition ? Since eigen decomposition is possible for nxn matrix and that are ...
0
votes
0answers
13 views

What information can I infer from the decay rate of the singular values of a matrix

Singular values of a matrix decay. What can someone understand from the shape of this decay? For instance, if the decrement is linear, or exponential? In practice, it happens that the plot of the ...
2
votes
0answers
83 views

Row similarity in matrix vs in different factorizations

Suppose an arbitrary $m \times n$ matrix $M$ and the factorizations: Arbitrary: $M = U_a V_a^T$, where $U_a$ is $m \times k$, $V_a$ is $n \times k$ ($k < m,n$), and $rank(U_a)=rank(V_a)=k$. SVD: $...
0
votes
0answers
33 views

Can we always perform SVD on the data matrix before doing high-dimensional logistic regression?

So I'm using lasso logistic regression to classify my data. My data matrix $X$ has dimension $n\times p$ for $p >> n$. As $p$ is on the order of a billion, I expect to face some computational ...
1
vote
0answers
19 views

Principal Component Analysis on Numerical Predictors alone for Dimension Reduction

I'm trying to reduce the number of dimensions for this 'Network Anamoly Detection' dataset: https://www.kaggle.com/anushonkar/network-anamoly-detection The dataset has a total of 40 features out of ...
0
votes
0answers
14 views

Binary Matrix Low Rank Matrix Factorization

Low Rank Matrix Factorization is a pretty popular problem in data mining. We need to find 2 matrices, $W, H$ such as $F = W \cdot H$. I know that this approximation is NPC problem, so we won't find ...
1
vote
0answers
22 views

Can I combine independent components from different models using PCA?

I have a set of independent components for each subject in my dataset (i.e. an ica model was generated for each subject). The samples used to generate each set of ICs are aligned across subjects, and ...
0
votes
0answers
7 views

What is the difference between SVD and Collabarative filtering precisely?

Both are sometimes used interchangebly but still there is a difference between them .
0
votes
1answer
21 views

What do the matrix (S, U, V) returned by singular value decomposition represent (in terms of variation)?

I believe SVD on a matrix A returns three matrices: U, S, and V. Let's imagine A is a data matrix with training examples/records/whatever you call them as its rows and attributes as its columns. I ...
1
vote
0answers
32 views

Signal Decomposition

I have two time dependent signal sources X & Y. Both can be modeled as having a linear combination of time dependent individual components and common components; so X(t)=a(t)+C(t)+noise, Y(t)=b(t)+...
2
votes
0answers
16 views

Can't Recreate Values for U, S, V from SVD in numpy [duplicate]

To better understand SVD, I'm trying to recreate the values for U, S, and V using straight ...
0
votes
1answer
59 views

Predictive model based on Principal Components when new data has different variables

I built a logistic regression model to classify a corpus of documents. The dependent variable is the type of document (eg A or B) while the dependent variables, because of dimensionality, are the ...
0
votes
0answers
29 views

prove for Eckart-young in Frobenius norm

On page 74, linear algebra and learning from data. P74 the prove for Eckart-young in the frobenius norm. I couldnot understand why G = 0 in the proof, anybody can help me? Thank you!
0
votes
0answers
19 views

Assessing the size of a cone by the singular values of $M$

Suppose I work with vectors from a high dimensional space with $100<N<1000$, e.g. word-embeddings. Say I have, already selected $R$ vectors, with $R\simeq10$, which form a matrix $M \in \mathbb{...
0
votes
0answers
21 views

Applying SVD on dataset with 4 columns

I have a dataset with following format and 200000 rows: X Y Z A 5608 142 740 1 4533 142 741 2 5620 143 740 0 4732 142 744 1 5500 143 742 1 5514 142 741 2 I am ...
0
votes
0answers
29 views

Can I use the matrix $U$ instead of the matrix $V$ in Principal Component Analysis?

I'm taking Andrew NG's Machine Learning Course and got to the part of Principal Component Analysis. Andrew's implementation of PCA aroused 2 questions for me. 1. Let's say that we have the data ...
0
votes
0answers
138 views

SVD -> PCA -> t-SNE; Does it make sense?

I have a data set of size (4600, 10000). I did L2 normalization at first, then I did the following two steps to visualize it in a lower dimension: ...
0
votes
0answers
28 views

Is each row of latent factors obtained from matrix decomposition (SVD) dependent on the other rows of the higher dimensional matrix?

I implemented a recommendation system using user-user interaction data, learning missing ratings through alternating least squares and matrix factorization, which as I understand it, adjusts and ...
1
vote
2answers
60 views

Why is there a reconstruction loss in PCA with orthonormal eigenvectors?

I've already read How to reverse PCA and reconstruct original variables from several principal components? and I understand conceptually and visually why there has to be a reconstruction loss. ...
0
votes
0answers
98 views

How to make a scree plot out of SVD data to validate PCA

After doing a singular value decomposition (SVD) of a data set, I'm left with three matrices: 1. An orthogonal Left Singular Vector (U) 2. diagonal matrix with elements in descending order (S) 3. ...
0
votes
0answers
8 views

How to measure changes in condition indices over time

I am trying to understand how adding data, one observation at a time, affects the condition indices of a model. A similar question is how adding individual observations affects the principal ...
2
votes
1answer
51 views

Intuition About Principal Component Directions

I am trying to really get a deep understanding of PCA. From my understanding, a principal component is defined as $$\mathbf{z}_k = \phi_{1,k} \mathbf{x}_1 + \ldots + \phi_{p,k} \mathbf{x}_p = \mathbf{...
0
votes
0answers
36 views

dimensionality reduction using SVD for forecasting with machine learning

I'm using a LSTM model to forecast time series data. My dataset has far too many variables and I would like to perform dimensionality reduction. My LSTM model works on a rolling window of 500. I ...
0
votes
0answers
31 views

Different number of Eigen/Singular values from PCA and SVD

My understanding is that a SVD done on a raw data matrix M and a PCA done on its covariance matrix C should return the same eigen/singular values. I have a 2736 x 356 data matrix and am using the ...
0
votes
0answers
12 views

Why is more Covariance Matrix different in these two computations? [duplicate]

I am trying to break down a matrix into its principal components step-by-step using SVD in order to better understand the process. What I don't understand is why I'm getting different forms of the ...
1
vote
0answers
36 views

SVD versus RSVD

In the so-called incremental SVD used for collaborative filtering: http://www.machinelearning.org/proceedings/icml2007/papers/407.pdf http://www2.research.att.com/~volinsky/papers/ieeecomputer.pdf ...
1
vote
0answers
114 views

Why does kmeans after SVD result in ideal clusters

I am clustering tweets which are related to eye fashion and they are extracted using keywords like mascara, eyeliner, eyeshadow, etc from twitter. I constructed a Tf-idf matrix (tweets x words) ...
0
votes
1answer
69 views

Interpretation of Eigenvalue vs. Singular Value plot

I'm doing some preliminary analysis on the feature matrix for a certain dataset (rows are observations, columns are feature dimensions). I have computed the SVD and PCA decompositions for this matrix ...
0
votes
0answers
16 views

Since SVDs are derived from existing variables, should you push the existing variables AND derived SVDs into a model?

wouldn't multicollinearity occur between the variables since SVDs are derived from the existing variables in the first place?
2
votes
0answers
22 views

Factorizing a matrix of distributions [closed]

Let's say we have a matrix $X \in \mathbb{R}^{m \times n}$, then the (R-truncated) SVD allows to approximate: $X_{i,j} \approx \sum\limits_{r=1}^{R} \sigma_r \times U_{r,i} \times V_{r, j}$ Now I ...
5
votes
0answers
311 views

Why do PCA and PCoA give the same components but different explained variances?

I'm quite familiar with Principal Component Analysisis, as I use it to study genetic structure. Lately, I was revisiting some of the functions I was using in R (...
1
vote
0answers
183 views

Calculating PCA coefficients using SVD, PCA (sklearn) and Covariance Matrix

I am trying to understand PCA implemented in different methods on python. I am failing to get equal PCA coefficients in each of the methods. By PCA coefficients I mean data projected in the principle ...
0
votes
0answers
27 views

Singular values for a latent-factor model

Suppose we build a latent-factor model using alternating least squares (ALS) or stochastic gradient descent (SGD). Can we calculate weights for each latent factor, in a similar way to how the singular-...
1
vote
0answers
22 views

How SVD factorisation -based recomendation algos deal with new user interaction

Classic SVD and SVD++ alogritms generate predictions based on a current known ratings only for known users and known items. But I need to make prediction for some new user on the old items. In the ...
0
votes
1answer
209 views

SVD matrixes do not coincide with Eigen decomposition for covariance matrix [duplicate]

I am comparing the output from the singular value decomposition with the eigendecomposition of the covariance matrix (symmetric matrix). I am expecting that the Eigenvector and a non-diagonal matrix ...
0
votes
1answer
16 views

preparing free text column for regression

I have a column X which contains occupation/profession as an independent variable as free text, which is very much correlated with a continuous dependent variable. What techniques do you usually use ...
0
votes
2answers
61 views

What is the meaning of these principal components?

I have a matrix of data. I computed the principal components of my matrix using SVD (code shown below): subtract mean...then $$[U,S,V] = SVD({\rm matrix})$$ for $V$, which is the principal ...
1
vote
1answer
332 views

SVD PCA reconstruction of data [duplicate]

I have some data about the $\{noise,~ size,~ speed,~ length,~ width\}$ of cars. I have performed SVD, and I want to reconstruct my data using only the first 2 principal components. I subtracted mean ...
0
votes
0answers
42 views

Rank 1 SVD with constraint on U

I need to perform a particular rank 1 decomposition of a sparse matrix $\mathbf{A} \in \mathbb{R}^{n\times n}$. In particular I am looking for the positive vector $\mathbf{u} \in \mathbb{R}^{+n}$ ...
3
votes
1answer
508 views

Why is computing ridge regression with a Cholesky decomposition much quicker than using SVD?

By my understanding, for a matrix with n samples and p features: Ridge regression using Cholesky decomposition takes O(p^3) time Ridge regression using SVD takes O(p^3) time Computing SVD when only ...
3
votes
0answers
91 views

Why is it much quicker to compute ridge regression than regular linear regression?

By my understanding, for a matrix with n samples and p features: Ridge regression using cholesky takes O(p^3) time Ordinary linear regression takes O(p^3) time Singular value decomposition if u, v ...
0
votes
0answers
26 views

Using numpy SVD to calculate factor loadings [duplicate]

I'm doing PCA (Principal Component Analysis) in Python using the numpys Singular Value Decomposition. Effectively extracting the principal components like so: ...
0
votes
0answers
26 views

Meaning of matrix U in LSA?

LSA uses SVD algorithm i.e. the terms-documents matrix $=U \Sigma V^T$ What is the exact meaning of matrix U here: I know that it is a rotation but rotation of what? Is it a concepts space? If yes ...
0
votes
0answers
83 views

Can anyone help me with step by step procedure in MATLAB for missing data imputation using Singular Value Decomposition (SVD [duplicate]

I need step by step procedure for imputing data using SVD in Matlab. Please provide resources if any !
2
votes
1answer
197 views

Reference point in projection axis of SVD (singular value decomposition)

I am watching a YouTube video on SVD, and attempting to recreate some of its examples to better understand the internal machinery of the algorithm. In one of the slides, the instructor mentions that ...
0
votes
0answers
39 views

Additional Property of Singular Value Decomposition

I am new to SVD so forgive me if the question is trivial. Following is my question. If I have two sets of linear equations, Y1 = T1.X Y2 = T2.X where T1 and T2 are mxn rectangular matrices. Now let'...

1 2 3 4 5 7