Questions tagged [svd]

Singular value decomposition (SVD) of a matrix $\mathbf{A}$ is given by $\mathbf{A} = \mathbf{USV}^\top$ where $\mathbf{U}$ and $\mathbf{V}$ are orthogonal matrices and $\mathbf{S}$ is a diagonal matrix.

Filter by
Sorted by
Tagged with
2
votes
1answer
25 views

Using SVD to write the least squares fitted vector

Elements of statistics p.66 Please I know the least squares solution for $\hat\beta = (X^TX)^{-1}X^Ty$ but I don't know how they were able to get $X\hat\beta= X(X^TX)^{-1}X^Ty = UU^Ty$ These are the ...
0
votes
1answer
15 views

How to compute the left singular eigenvector matrix (U) from the output of prcomp() for PCA in R?

I am examining the output of the prcomp() function in R for PCA in light of the singular value decomposition equation: $X = U \cdot \Sigma \cdot V^{T}$, where: $X$: ...
2
votes
0answers
17 views

Singular value decomposition used for dimensionality reduction in brain signal topographic data

I am trying to replicate the localizer method described in this paper (page 4). I am stuck on a step which I don't completely understand, and I would like your input and interpretation to progress. ...
4
votes
0answers
31 views

Physical interpretation of $U$ and $V$ matrices in SVD

I have a question about the physical interpretation of $U$ and $V$ matrices in SVD. I collect measurements at multiple devices across time are collected into an $m$ × $T$ matrix $M$, where m is the ...
1
vote
0answers
14 views

Kernel matrix decomposition

I had a look at the sklearn.kernel_approxiamtion.Nystroem implementation, which is also described in this post: Nystroem Method for Kernel Approximation Here, a ...
0
votes
0answers
13 views

Equal variance along left and right singular vectors?

Please confirm or reject my line of reasoning: Given SVD of $X$: $X_{NxP}=U_{NxP}D_{PxP}V_{PxP}'$, Variance along ith column vector of $U$ is given by $||X'u_i||^2=u'_iXX'u_i=u'_id_i^2u_i=d_i^2$, ...
0
votes
0answers
10 views

How to predict for test set when training a recommender by decomposing the utility matrix X=UV?

This probably sounds stupid but I don't get the workflow of building a recommending system by the utility matrix: X[i,j] = how much the ith user likes the jth object. For practical issues I refer to ...
0
votes
0answers
12 views

How does the Zeiger-McEwin & Kung algorithm work for fitting a sum of exponentials?

I am trying to understand this paper fit sum of exponentials but am having a bit of difficulty. Let me go through what I have understood so far. One has a certain time-series data set and want's to ...
1
vote
1answer
40 views

Question about the Proof of PCA in “Learning from Data” by Shwartz and Ben-David, P. 280-281

Does anyone know how to justify the red and blue line in the attached proof of PCA? Red line: $B \in \mathbb{R}^{ d \times n}$, arrange $B = [B_{j,1} | B_{j,2} | \cdots | B_{j,n}]$, then $B^\top B = \...
1
vote
0answers
18 views

What is so special about the least norm solution in case of an undetermined system of equations

In particular this is the go to approach in case of solving a least squares problem that lacks a unique solution, how does being the closest point to the origin among all the solutions make it any ...
0
votes
0answers
19 views

Principal components with and without centering [duplicate]

Suppose that I am given various samples of a vector random variable as the columns $v_1,v_2,\dots,v_n$ of a certain matrix $A$. Is there a relation between the SVD $A = USV^T$ the SVD $\hat{A} = \hat{...
1
vote
0answers
29 views

SVD for a complex data matrix — what is the meaning of the columns of $V$?

I've read this wonderful explanation of SVD, where the writer mentions that the columns of $V$ are the principal directions (Summary, #1). Is this also true when the data matrix $X$ is complex? If I'm ...
0
votes
0answers
27 views

Computing Pairwise Distances Through PCA or SVD

What should I do to reduce an mxn (m=17, n=650,000) matrix, where m are samples and n are features of these samples, into a matrix of pairwise distances (which I will then use to generate a dendrogram)...
0
votes
0answers
17 views

Which form of SVD is most commonly in stats/ML (e.g., PCA, least squares)?

SVM comes in several forms: e.g., the full form and the reduced forms https://en.wikipedia.org/wiki/Singular_value_decomposition#Reduced_SVDs. Is there one that is more commonly used in stats? If so, ...
2
votes
1answer
52 views

Human intuition behind SVD in case of recommendation system

This does not answer my question. I struggled very hard to understand the SVD from a linear-algebra point of view. But in some cases I failed to connect the dots. So, I started to see all the ...
0
votes
0answers
22 views

How vector projection works behind SVD?

I was reading a blog on mathematical intuition behind SVD. Here, author pointed out three information we get after vector decomposition. The directions of projection — the unit vectors (v₁ and v₂) ...
9
votes
3answers
556 views

SVD : Why right singular matrix is written as transpose

The SVD is always written as, A = U Σ V_Transpose The question is, Why is the right singular matrix written as V_Transpose? I mean lets say, W = V_Transpose and then write SVD as A = U Σ W SVD Image ...
1
vote
0answers
25 views

Relationship between the SVD and correlation matrices

I'm reading Data Driven Science and Engineering by Kutz and Brunton to understand more about the SVD. Consider $X = U\Sigma V$, $XX^*$, and $X^*X$ where $X \in \mathbb{R}^{m\times n} $ In particular, ...
0
votes
1answer
24 views

Penalized Canonical Correlation in R with PMA Module

I am trying to use sparse canonical correlation analysis as implemented in the R PMA package. I'm finding that the correlations output by the package seem slightly inconsistent with the ones you would ...
5
votes
1answer
101 views

How are eigenvalues/singular values related to variance (SVD/PCA)?

Let $X$ be a data matrix of size $n \times p$. Assume that $X$ is centered (column means subtracted). Then, the $p \times p$ covariance matrix is given by $$C = \frac{X^TX}{n-1}$$ Since $C$ is ...
1
vote
0answers
10 views

Symbolic Singular Value Decomposition? U,S,V as function of the elements of M [closed]

Suppose we want to compute the SVD of $\mathbf{M} = \begin{bmatrix} m_{11} & m_{12} & m_{13} \\ m_{21} & m_{22} & m_{23} \\ m_{31} & m_{32} & m_{33} \end{bmatrix}$ (...
0
votes
0answers
18 views

Variance of a principal component

Suppose that we have a SVD for our data matrix centered $X = UDV^T$. Then it is stated that the i-th principal component, $Xv_i$, has variance $\frac{d_i^2}{N}$. Consider these steps. $$ var(Xv_i) = ...
0
votes
0answers
15 views

Whitening on projection matrices

The projection matrix $P = I -xx^T\in \mathbf{R}^{d \times d}$ has a zero eigenvalue and eigenvalues equal to one with multiplicity $d-1$. Is it possible to apply whitening transform on $P$ taking ...
0
votes
0answers
9 views

How does partial least squares algorithm return more than one factor?

My understanding of PLS regression is that we find an eigen vector such that it maximises the covariance between X(matrix of independent variables) and Y(vector/matrix of dependent variable) i.e. find ...
1
vote
0answers
15 views

Use Matrix Factorization to predict probability of a recommendation system?

I have a dataset where I have a sparse utility matrix (user-product) with binary input: 1 if the user $i$ bought the product $j$, and 0 if it hasn't. However it has a different meaning on the test set,...
0
votes
0answers
22 views

In Probabilistic PCA, Where does the arbitrary orthogonal matrix(rotation matrix) come from?

I'm working on studying Probabilistic PCA based on the paper (Tipping & Bishop, 1999), I can follow the idea that the maximum likelihood function would reach the stationary point when the the ...
2
votes
1answer
60 views

Why is non-centered SVD accepted in LSA

In Latent Semantic Analysis (LSA) , we apply SVD to a term-document matrix $A$, then choose to ignore all but $k$ largest singular values. The term-document matrix is not centered, or normalised, ...
0
votes
0answers
23 views

Interpreting SVD on non-centered matrix

I have a very large, very sparse matrix $A \in \mathbb{N}^{n \times m}$ I'd like to perform SVD on. It is non-centered. When I center it to $A'$, I can't even fit it in memory (because $A'$ is in $\...
0
votes
0answers
13 views

Variability of K SVD components [duplicate]

Let's say I have a SVD of a matrix $A = U \Sigma V^T$, $A \in \mathbb{R}^{n \times m}$, and I'm using top-k components corresponding to $\sigma_1, ..\sigma_k$, the k largest values on the diagonal of $...
0
votes
0answers
10 views

Is there any relationship between SVD low rank approximation and DCT low frequency approximation?

Matrix can be approximated by low rank approximation using SVD by using major principle components. On the other hand, if we look at the frequency domain, matrix can also be approximated using DCT (...
1
vote
1answer
29 views

Does SVD provide the best low rank approximation for any matrix regardless of shape?

Wikipedia states (link below) that by the Eckart-Young-Mirsky theorem, the SVD provides the best low rank matrix approximation (on the basis of Frobenius norm of the error matrix) for any matrix A in ...
5
votes
2answers
231 views

What is finite precision arithmetic and how does it affect SVD when computed by computers?

Was reading the paper "DETECTING AND ASSESSING THE PROBLEMS CAUSED BY MULTICOLLINEARITY:A USE OF THE SINGULAR-VALUE DECOMPOSITION" by David Belsley and Virginia Klema. After performing SVD, while ...
1
vote
1answer
34 views

PCA with SVD exercice 23.5 understanding machine learning

In understanding machine learning Shai Sharev-Scwartz and Shai Ben-David exercice 23.5. I would like to use SVD to minimize : $$ \text{argmin}_{W \in \mathbb{R}^{n,d}, U \in \mathbb{R}^{d,n}}{\text{ }...
0
votes
0answers
12 views

Dimension reduction and get a dataset with the number of features that larger than the minimum of the numbers of samples and features

I'm trying to perform dimension reduction with SVD on an image dataset with the shape of roughly (3000,20000) as you see the number of each sample's features is way larger than the number of samples, ...
0
votes
0answers
16 views

How to project test data onto new space after using svd on trining data for dimension reduction [duplicate]

I'm working on a dataset of images and i was using the code below to apply dimension reduction to my data set and project the test data onto the new space: ...
0
votes
0answers
17 views

How to build a recommendation system for users outside of original model data?

I am creating a collaborative filtering model in Python to recommend possible movies a user may like. So, using typical methods in this field, I have a sparse matrix nxm with n users and m movies. ...
3
votes
1answer
122 views

Moore Penrose Pseudo-Inverse Fast Algorithm in R [closed]

I want to apply Moore Penrose Pseudo-Inverse on my matrix, which is a 20,000 * 20,000 symmetric matrix with rank 19,999. I found ginv() function from the ...
2
votes
1answer
52 views

Orthonormalization to use closed form Lasso solution

Given the Lasso problem $$ min_\beta (Y-X\beta)^\top(Y-X\beta) \quad s.t. \|\beta\|_1\leq\lambda, $$ and assuming that X is orthonormal such that $X^\top X=I$, we know that the closed form solution ...
2
votes
1answer
148 views

Singular Value Decomposition (SVD) for feature selection

I was reading a paper called "Production Optimization Using Machine Learning in Bakken Shale", and came across an approach I was a bit puzzled by. Unfortunately, the paper is behind a paywall, but I ...
2
votes
2answers
49 views

Least Square Fitted Vector Through SVD Equals to y

Elements of Statistical Learning, p 66 The SVD of the $N \times p$ matrix $X$ has the form $X = UDV^T$ Here $U$ and $V$ are $N \times p$ and $p \times p$ orthogonal matrices, with the ...
0
votes
0answers
15 views

Why does this pattern show up so often when doing svd on text data?

A lot of times, I'll get some text data and, as a quick fist step, I'll get a term-frequency matrix, take the first two dimensions of the truncated SVD, and plot it. I very often get a picture that ...
0
votes
0answers
8 views

Transformation of SVD for latent semantic analysis

General Idea: I'm working through a particular implementation of Latent Semantic Analysis via SVD. Here is some example code. fairly simple: ...
2
votes
1answer
30 views

SVD in R recomposition

...
0
votes
1answer
75 views

Projecting New Data onto Existing Principal Axes

Let $\mathbf{X}$ be a dataset of size $n \times d$, where $n$ is the number of samples (days) and $d$ is the number of variables (daily observations). All observations are taken at the same times each ...
13
votes
3answers
1k views

What does it mean when PCA does not produce a reduction in dimensionality?

I am a beginner in PCA, I am trying to apply it on a dataset I have. The features are different geometrical parameters with different units and variability, I do standardize the features matrix by ...
2
votes
0answers
15 views

How to approximate a Hermitian matrix with a transposed cross product of a single matrix?

I have a complex square matrix, and wish to learn latent factors (equally weighted latent factors, so not SVD) from this matrix. Given a Hermitian matrix A of ...
0
votes
0answers
42 views

Finding the optimal number of latent factors in Symmetric Singular Value Decomposition

Consider symmetric Singular Value Decomposition of a symmetric unitary matrix A into U, D, ...
0
votes
0answers
21 views

Subtracting PCA projection from original data

I'm trying to implement Anomaly Detection algorythm from this article https://iopscience.iop.org/article/10.1088/1742-6596/1069/1/012072/pdf at my work. So I have big questions for me: Did I ...
4
votes
1answer
226 views

Multivariate normal distribution transformation

Suppose that $X $ has a multivariate normal distribution $X\sim MVN (\mu, \Sigma) $, How can I transform $X$ into $Z$ so that $Z\sim MVN(\mu, I) $ where $I$ is the identity matrix? For instance, ...
0
votes
0answers
11 views

Intuitions behind Singular Value Decomposing generalizing eigen decomposition [duplicate]

Let M be mxn matrix then SVD of M will be UXW^* (sorry for X, assume summation). Then how does it generalizes eigen decomposition ? Since eigen decomposition is possible for nxn matrix and that are ...

1
2 3 4 5
8