Questions tagged [svd]

Singular value decomposition (SVD) of a matrix $\mathbf{A}$ is given by $\mathbf{A} = \mathbf{USV}^\top$ where $\mathbf{U}$ and $\mathbf{V}$ are orthogonal matrices and $\mathbf{S}$ is a diagonal matrix.

Filter by
Sorted by
Tagged with
0
votes
0answers
5 views

How does partial least squares algorithm return more than one factor?

My understanding of PLS regression is that we find an eigen vector such that it maximises the covariance between X(matrix of independent variables) and Y(vector/matrix of dependent variable) i.e. find ...
1
vote
0answers
12 views

Use Matrix Factorization to predict probability of a recommendation system?

I have a dataset where I have a sparse utility matrix (user-product) with binary input: 1 if the user $i$ bought the product $j$, and 0 if it hasn't. However it has a different meaning on the test set,...
0
votes
0answers
15 views

In Probabilistic PCA, Where does the arbitrary orthogonal matrix(rotation matrix) come from?

I'm working on studying Probabilistic PCA based on the paper (Tipping & Bishop, 1999), I can follow the idea that the maximum likelihood function would reach the stationary point when the the ...
0
votes
1answer
22 views

Why is non-centered SVD accepted in LSA

In Latent Semantic Analysis (LSA) , we apply SVD to a term-document matrix $A$, then choose to ignore all but $k$ largest singular values. The term-document matrix is not centered, or normalised, ...
0
votes
0answers
21 views

Interpreting SVD on non-centered matrix

I have a very large, very sparse matrix $A \in \mathbb{N}^{n \times m}$ I'd like to perform SVD on. It is non-centered. When I center it to $A'$, I can't even fit it in memory (because $A'$ is in $\...
0
votes
0answers
13 views

Variability of K SVD components [duplicate]

Let's say I have a SVD of a matrix $A = U \Sigma V^T$, $A \in \mathbb{R}^{n \times m}$, and I'm using top-k components corresponding to $\sigma_1, ..\sigma_k$, the k largest values on the diagonal of $...
0
votes
0answers
10 views

Is there any relationship between SVD low rank approximation and DCT low frequency approximation?

Matrix can be approximated by low rank approximation using SVD by using major principle components. On the other hand, if we look at the frequency domain, matrix can also be approximated using DCT (...
1
vote
1answer
24 views

Does SVD provide the best low rank approximation for any matrix regardless of shape?

Wikipedia states (link below) that by the Eckart-Young-Mirsky theorem, the SVD provides the best low rank matrix approximation (on the basis of Frobenius norm of the error matrix) for any matrix A in ...
5
votes
2answers
219 views

What is finite precision arithmetic and how does it affect SVD when computed by computers?

Was reading the paper "DETECTING AND ASSESSING THE PROBLEMS CAUSED BY MULTICOLLINEARITY:A USE OF THE SINGULAR-VALUE DECOMPOSITION" by David Belsley and Virginia Klema. After performing SVD, while ...
1
vote
1answer
33 views

PCA with SVD exercice 23.5 understanding machine learning

In understanding machine learning Shai Sharev-Scwartz and Shai Ben-David exercice 23.5. I would like to use SVD to minimize : $$ \text{argmin}_{W \in \mathbb{R}^{n,d}, U \in \mathbb{R}^{d,n}}{\text{ }...
0
votes
0answers
11 views

Dimension reduction and get a dataset with the number of features that larger than the minimum of the numbers of samples and features

I'm trying to perform dimension reduction with SVD on an image dataset with the shape of roughly (3000,20000) as you see the number of each sample's features is way larger than the number of samples, ...
0
votes
0answers
16 views

How to project test data onto new space after using svd on trining data for dimension reduction [duplicate]

I'm working on a dataset of images and i was using the code below to apply dimension reduction to my data set and project the test data onto the new space: ...
0
votes
0answers
15 views

How to build a recommendation system for users outside of original model data?

I am creating a collaborative filtering model in Python to recommend possible movies a user may like. So, using typical methods in this field, I have a sparse matrix nxm with n users and m movies. ...
3
votes
1answer
71 views

Moore Penrose Pseudo-Inverse Fast Algorithm in R [closed]

I want to apply Moore Penrose Pseudo-Inverse on my matrix, which is a 20,000 * 20,000 symmetric matrix with rank 19,999. I found ginv() function from the ...
2
votes
1answer
49 views

Orthonormalization to use closed form Lasso solution

Given the Lasso problem $$ min_\beta (Y-X\beta)^\top(Y-X\beta) \quad s.t. \|\beta\|_1\leq\lambda, $$ and assuming that X is orthonormal such that $X^\top X=I$, we know that the closed form solution ...
2
votes
1answer
69 views

Singular Value Decomposition (SVD) for feature selection

I was reading a paper called "Production Optimization Using Machine Learning in Bakken Shale", and came across an approach I was a bit puzzled by. Unfortunately, the paper is behind a paywall, but I ...
2
votes
2answers
47 views

Least Square Fitted Vector Through SVD Equals to y

Elements of Statistical Learning, p 66 The SVD of the $N \times p$ matrix $X$ has the form $X = UDV^T$ Here $U$ and $V$ are $N \times p$ and $p \times p$ orthogonal matrices, with the ...
0
votes
0answers
15 views

Why does this pattern show up so often when doing svd on text data?

A lot of times, I'll get some text data and, as a quick fist step, I'll get a term-frequency matrix, take the first two dimensions of the truncated SVD, and plot it. I very often get a picture that ...
0
votes
0answers
7 views

Transformation of SVD for latent semantic analysis

General Idea: I'm working through a particular implementation of Latent Semantic Analysis via SVD. Here is some example code. fairly simple: ...
1
vote
1answer
21 views

SVD in R recomposition

...
0
votes
1answer
49 views

Projecting New Data onto Existing Principal Axes

Let $\mathbf{X}$ be a dataset of size $n \times d$, where $n$ is the number of samples (days) and $d$ is the number of variables (daily observations). All observations are taken at the same times each ...
13
votes
3answers
1k views

What does it mean when PCA does not produce a reduction in dimensionality?

I am a beginner in PCA, I am trying to apply it on a dataset I have. The features are different geometrical parameters with different units and variability, I do standardize the features matrix by ...
2
votes
0answers
12 views

How to approximate a Hermitian matrix with a transposed cross product of a single matrix?

I have a complex square matrix, and wish to learn latent factors (equally weighted latent factors, so not SVD) from this matrix. Given a Hermitian matrix A of ...
0
votes
0answers
35 views

Finding the optimal number of latent factors in Symmetric Singular Value Decomposition

Consider symmetric Singular Value Decomposition of a symmetric unitary matrix A into U, D, ...
0
votes
0answers
19 views

Subtracting PCA projection from original data

I'm trying to implement Anomaly Detection algorythm from this article https://iopscience.iop.org/article/10.1088/1742-6596/1069/1/012072/pdf at my work. So I have big questions for me: Did I ...
4
votes
1answer
197 views

Multivariate normal distribution transformation

Suppose that $X $ has a multivariate normal distribution $X\sim MVN (\mu, \Sigma) $, How can I transform $X$ into $Z$ so that $Z\sim MVN(\mu, I) $ where $I$ is the identity matrix? For instance, ...
0
votes
0answers
11 views

Intuitions behind Singular Value Decomposing generalizing eigen decomposition [duplicate]

Let M be mxn matrix then SVD of M will be UXW^* (sorry for X, assume summation). Then how does it generalizes eigen decomposition ? Since eigen decomposition is possible for nxn matrix and that are ...
0
votes
0answers
18 views

What information can I infer from the decay rate of the singular values of a matrix

Singular values of a matrix decay. What can someone understand from the shape of this decay? For instance, if the decrement is linear, or exponential? In practice, it happens that the plot of the ...
2
votes
0answers
91 views

Row similarity in matrix vs in different factorizations

Suppose an arbitrary $m \times n$ matrix $M$ and the factorizations: Arbitrary: $M = U_a V_a^T$, where $U_a$ is $m \times k$, $V_a$ is $n \times k$ ($k < m,n$), and $rank(U_a)=rank(V_a)=k$. SVD: $...
0
votes
0answers
38 views

Can we always perform SVD on the data matrix before doing high-dimensional logistic regression?

So I'm using lasso logistic regression to classify my data. My data matrix $X$ has dimension $n\times p$ for $p >> n$. As $p$ is on the order of a billion, I expect to face some computational ...
1
vote
0answers
19 views

Principal Component Analysis on Numerical Predictors alone for Dimension Reduction

I'm trying to reduce the number of dimensions for this 'Network Anamoly Detection' dataset: https://www.kaggle.com/anushonkar/network-anamoly-detection The dataset has a total of 40 features out of ...
0
votes
0answers
26 views

Binary Matrix Low Rank Matrix Factorization

Low Rank Matrix Factorization is a pretty popular problem in data mining. We need to find 2 matrices, $W, H$ such as $F = W \cdot H$. I know that this approximation is NPC problem, so we won't find ...
1
vote
0answers
25 views

Can I combine independent components from different models using PCA?

I have a set of independent components for each subject in my dataset (i.e. an ica model was generated for each subject). The samples used to generate each set of ICs are aligned across subjects, and ...
0
votes
0answers
7 views

What is the difference between SVD and Collabarative filtering precisely?

Both are sometimes used interchangebly but still there is a difference between them .
0
votes
1answer
55 views

What do the matrix (S, U, V) returned by singular value decomposition represent (in terms of variation)?

I believe SVD on a matrix A returns three matrices: U, S, and V. Let's imagine A is a data matrix with training examples/records/whatever you call them as its rows and attributes as its columns. I ...
2
votes
0answers
36 views

Signal Decomposition

I have two time dependent signal sources X & Y. Both can be modeled as having a linear combination of time dependent individual components and common components; so X(t)=a(t)+C(t)+noise, Y(t)=b(t)+...
2
votes
0answers
16 views

Can't Recreate Values for U, S, V from SVD in numpy [duplicate]

To better understand SVD, I'm trying to recreate the values for U, S, and V using straight ...
0
votes
1answer
60 views

Predictive model based on Principal Components when new data has different variables

I built a logistic regression model to classify a corpus of documents. The dependent variable is the type of document (eg A or B) while the dependent variables, because of dimensionality, are the ...
0
votes
0answers
46 views

prove for Eckart-young in Frobenius norm

On page 74, linear algebra and learning from data. P74 the prove for Eckart-young in the frobenius norm. I couldnot understand why G = 0 in the proof, anybody can help me? Thank you!
0
votes
0answers
19 views

Assessing the size of a cone by the singular values of $M$

Suppose I work with vectors from a high dimensional space with $100<N<1000$, e.g. word-embeddings. Say I have, already selected $R$ vectors, with $R\simeq10$, which form a matrix $M \in \mathbb{...
0
votes
0answers
21 views

Applying SVD on dataset with 4 columns

I have a dataset with following format and 200000 rows: X Y Z A 5608 142 740 1 4533 142 741 2 5620 143 740 0 4732 142 744 1 5500 143 742 1 5514 142 741 2 I am ...
0
votes
0answers
33 views

Can I use the matrix $U$ instead of the matrix $V$ in Principal Component Analysis?

I'm taking Andrew NG's Machine Learning Course and got to the part of Principal Component Analysis. Andrew's implementation of PCA aroused 2 questions for me. 1. Let's say that we have the data ...
0
votes
0answers
193 views

SVD -> PCA -> t-SNE; Does it make sense?

I have a data set of size (4600, 10000). I did L2 normalization at first, then I did the following two steps to visualize it in a lower dimension: ...
0
votes
0answers
55 views

Is each row of latent factors obtained from matrix decomposition (SVD) dependent on the other rows of the higher dimensional matrix?

I implemented a recommendation system using user-user interaction data, learning missing ratings through alternating least squares and matrix factorization, which as I understand it, adjusts and ...
1
vote
2answers
98 views

Why is there a reconstruction loss in PCA with orthonormal eigenvectors?

I've already read How to reverse PCA and reconstruct original variables from several principal components? and I understand conceptually and visually why there has to be a reconstruction loss. ...
0
votes
1answer
235 views

How to make a scree plot out of SVD data

After doing a singular value decomposition (SVD) of a data set, I'm left with three matrices: 1. An orthogonal Left Singular Vector (U) 2. diagonal matrix with elements in descending order (S) 3. ...
0
votes
0answers
10 views

How to measure changes in condition indices over time

I am trying to understand how adding data, one observation at a time, affects the condition indices of a model. A similar question is how adding individual observations affects the principal ...
2
votes
1answer
57 views

Intuition About Principal Component Directions

I am trying to really get a deep understanding of PCA. From my understanding, a principal component is defined as $$\mathbf{z}_k = \phi_{1,k} \mathbf{x}_1 + \ldots + \phi_{p,k} \mathbf{x}_p = \mathbf{...
0
votes
0answers
45 views

dimensionality reduction using SVD for forecasting with machine learning

I'm using a LSTM model to forecast time series data. My dataset has far too many variables and I would like to perform dimensionality reduction. My LSTM model works on a rolling window of 500. I ...
1
vote
0answers
46 views

SVD versus RSVD

In the so-called incremental SVD used for collaborative filtering: http://www.machinelearning.org/proceedings/icml2007/papers/407.pdf http://www2.research.att.com/~volinsky/papers/ieeecomputer.pdf ...

1
2 3 4 5
7