Questions tagged [matrix-decomposition]

Matrix decomposition refers to the process of factorizing a matrix into a product of smaller matrices. By decomposing a large matrix, one can efficiently perform many matrix algorithms.

Filter by
Sorted by
Tagged with
0
votes
0answers
30 views

How can I construct a desired variance-covariance matrix for simulating multivariate Gaussian distribution samples using MATLAB?

I want to simulate multivariate normal distribution samples to help understand PCA, biplot, etc. For example, I want to see how the correlation structure affects the appearance of 2-D biplot. Two ...
0
votes
0answers
14 views

Compressed sensing and matrix recovery

I'm reading a paper estimation of (near) low-rank matrices with noise and high-dimensional scaling by Negahban and Wainwright where authors defied compressed sensing as a particular instance of the ...
0
votes
0answers
20 views

SVD on demeaned matrix

I'm trying to understand the effect of de-meaning with SVD. Suppose I have matrix $WW^T = \sum_{i=1}^n w_iw_i^T$ where $W$ is $n \times m$ and $w_i$ are its columns. Running SVD on this yields ...
0
votes
1answer
14 views

Latent factors are the same in both decomposed matrices?

This question is in the context of recommendation systems. We can use matrix factorization techniques to decompose a user-product explicit/implicit matrix(R) into two matrices(U, P). Let's say R is a ...
0
votes
0answers
48 views

Why absolute value of eigenvalues are used in PCA or LDA?

In PCA and LDA techniques, eigenvectors with the $k$ largest eigenvalues give principal components. However, when selecting these eigenvalues, are they to be sorted by the absolute value (regardless ...
0
votes
0answers
13 views

can we use a hybrid optimization schem for NMF

The NMF problem of the form $$X \simeq WH$$ is a constrained biconvex optimization problem, and is often solved by alternating updates schemes. For example, the multiplicative update rules use ...
0
votes
0answers
16 views

How to estimate variance in Bayesian matrix factorization using Gibbs samples?

I have implemented a Gibbs sampler for Bayesian Matrix Factorization /Completion of matrix $R = (r_{ij})$ which is $(N, M)$ dimensional and $p(r_{ij} | \textbf{u}_i, \textbf{v}_j) = N(r_{ij}|\textbf{u}...
0
votes
0answers
34 views

Drawing samples from matrix normal

I have to generate $n \times m$ sample ($A$) from a matrix normal distribution, given two covariance matrices: $n \times n$ row covariance matrix (matrix $B$) (defines the covariance between the rows ...
1
vote
0answers
12 views

Are principal component analysis (PCA) and empirical orthogonal function (EOF) methods the same?

As far as I've seen, EOF is just PCA but instead of thinking about the data matrix X as (number of samples, number of features), you consider it as (number of time points, number of different spatial ...
0
votes
1answer
50 views

Matrix Factorization and Linear Regression

Which matrix factorization algorithm is used in LinearRegression() function of scikit-learn?
1
vote
0answers
119 views

Adding explicit user info to matrix factorization

In the paper Matrix Factorization Techniques for Recommender Systems, it is claimed that we can incorporate extra user information into our recommender model by doing something like this: $$ \hat{r}_{...
1
vote
1answer
26 views

Understanding the decomposed matrices in Singular Value Decomposition [duplicate]

I am trying to get some intuition regarding the U and V matrices in SVD ($M=UEV^T$). I think these are orthonormal basis vectors, but I am struggling to get an intuition if they represent anything ...
1
vote
1answer
51 views

Why is there a discrepancy between the eigenvalues of the covariance matrix (PCA) and the eigenvalues of the kernel matrix (kernel PCA)?

I've done PCA on my data matrix $ \mathbf{X} $ which gives me i.a. the eigenvalues $ \lambda $ and eigenvectors $ v $ of the data covariance matrix $ C=\mathbf{X}^T \mathbf{X} $. I'm now extending my ...
0
votes
0answers
35 views

Why NNMF (non-negative matrix factorization) is a method for linear dimensionality reduction?

Some sources (for example this) say that NNMF is a method for linear dimensionality reduction. How to prove this statement? I see two different explanations of this and I want to know which of them (...
0
votes
0answers
14 views

SVD - vectors in matrix A

In SVD we have $A = U \Sigma V^T$. When applying it for ML, e.g. to calculate Moore-Penrose pseudoinverse for linear regression, I have seen that we take columns of $A$ as vectors. Typically in ML I ...
15
votes
1answer
1k views

Relationship between Cholesky decomposition and matrix inversion?

I've been reviewing Gaussian Processes and, from what I can tell, there's some debate whether the "covariance matrix" (returned by the kernel), which needs to be inverted, should be done so ...
1
vote
0answers
73 views

Alternating least squares --- what are the limitations?

I am taking an introductory course to Machine Learning and we learned alternating least squares for recommender systems. I learned that this method has some advantages --- easy to parallelize, and ...
0
votes
0answers
173 views

Offline Precision@k and Recall@k for recommender system

How can I evaluate offline Precision@k and Recall@k metrics for recommender system if I only have items-users matrix? I think I can't just compare recommendations and user data because it will be ...
0
votes
0answers
7 views

Physical significance of non-negative factors of a matrix?

I was trying to make a recommender system using matrix factorization techniques on rating data. I came across 2 algorithms - SVD and NMF. While the basic difference is very clear , I was wondering ...
2
votes
0answers
33 views

Convex Optimization Algorithm for Additive Matrix Decomposition

Given an m $\times$ n matrix $Y = S + L + E$, where $S$ is a sparse matrix, $L$ is a low rank matrix and $E$ is a noise matrix, I want to recover $S$, $L$. One of the techniques given in Agarwal et al ...
0
votes
0answers
11 views

Negative PCA component causing negative predictions in interval target variable with range 0 -1

By applying first a PCA analysis to a set of positive only variables (all binary), I get 5 components which are subsequently used in a non-negative factorized machine to predict a target variable (...
1
vote
0answers
86 views

Multivariate Normal Quadratic MGF: Eigendecomposition to Matrix form

If $X \sim \mathcal{N}(\mu, \Sigma)$ is a multivariate normal, then the quadratic $X^TAX$ has moment generating function $$M_{X^TAX}(t)= \frac{1}{\sqrt{\det(I - 2tA\Sigma)}}\exp\left(-\frac{1}{2}\mu'[...
0
votes
0answers
38 views

Optimisation of reconstruction error for PCA

I am trying to understand the math behind the minimal reconstruction error for PCA models in this book. Given a data matrix $Y$ which has dimensions $D\times n$ and assuming that it is centered. The ...
0
votes
0answers
34 views

What can I learn about the dimensions with highest variance of a matrix $M\approx L^TR$ from looking at $L$?

I have a high-dimensional, symmetric data matrix $M\in\mathbb{R}^{d\times d}$ , which is factorized by two matrices $L, R\in \mathbb{R}^{n\times d}$ : $L^TR\approx M$, where $n$ is much smaller than $...
0
votes
0answers
7 views

variance explained for CP factorisation

What is the best way to compute the variance explained for one or more terms in a CP tensor factorisation? Is it even defined? In PCA this is made easy by the fact that the eigenvalues are orthogonal. ...
0
votes
0answers
26 views

Recommending item category based on customer shopping behavior?

I'm trying to build some recommender system for online wine shop. What I'm thinking is recommending item category (ex: red_pinot, ...
1
vote
0answers
25 views

Fastest way to find Leading singular value and vector (power iteration, rsvd etc)

I want to know the fastest way to find out the leading singular value and vector of a large rectangular matrix. I have seen 2 suggestions and have questions on both of them : Power Method : For this ...
1
vote
0answers
73 views

How to explain the numerical discrepancy between FactoMineR::PCA() and the svd() in their output of the U matrix?

I am comparing the output of two functions in R to do Principal Component Analysis (PCA), the FactoMineR::PCA() and the ...
3
votes
0answers
107 views

Efficient computation of marginalized multivariate normal posterior distribution

In general,if we know that the marginal Gaussian distribution for some variable $\textbf{x}$ and a conditional Gaussian distribution for some $\textbf{y}|\textbf{x}$ of the forms: $$p(\textbf{x}) = \...
0
votes
0answers
118 views

Covariance Matrix Decomposition - Data Decorrelation

So I recently found out about Mahalanobis distance. Given a r.v $x$ in N-dimensional space, an associated metric is defined by $$M(x) = \sqrt{(x-\mu)^T S^{-1}(x-\mu)}$$ where $\mu$ and $S$ are mean ...
1
vote
1answer
34 views

How to compute the left singular eigenvector matrix (U) from the output of prcomp() for PCA in R?

I am examining the output of the prcomp() function in R for PCA in light of the singular value decomposition equation: $X = U \cdot \Sigma \cdot V^{T}$, where: $X$: ...
0
votes
0answers
18 views

Possible typo Logistic Matrix Factorisation?

In the paper Logistic Matrix Factorization for Implicit Feedback Data by CC. Johnson(see link). The author stated his maximum likelihood function to be (omit all the indices): $\prod p(l|X,Y,\beta)^{\...
0
votes
0answers
97 views

Implicit feedback ALS algorithm: the alpha parameter

I'm creating a recommender system for a video streaming service. My only knowledge about the user preference on a video is the watched percentage of that video. I'm using the implicit feedback ALS ...
1
vote
0answers
17 views

How to find upper and lower bound

Let $\Sigma \in S_{++}^n$ be a symmteric positive definte matrix with all diagonal entries one. Let $U \in R^{n \times k_1}$, $W \in R^{n \times k_2}$, $\Lambda \in R^{k_1 \times k_1}$ and $T \in R^{...
1
vote
0answers
31 views

Kernel matrix decomposition

I had a look at the sklearn.kernel_approxiamtion.Nystroem implementation, which is also described in this post: Nystroem Method for Kernel Approximation Here, a ...
0
votes
0answers
43 views

How to predict for test set when training a recommender by decomposing the utility matrix X=UV?

This probably sounds stupid but I don't get the workflow of building a recommending system by the utility matrix: X[i,j] = how much the ith user likes the jth object. For practical issues I refer to ...
1
vote
0answers
38 views

SVD for a complex data matrix — what is the meaning of the columns of $V$?

I've read this wonderful explanation of SVD, where the writer mentions that the columns of $V$ are the principal directions (Summary, #1). Is this also true when the data matrix $X$ is complex? If I'm ...
1
vote
1answer
42 views

Convert the following expression w.r.t to the whole dataset instead of element of the dataset?

I am in the process of expressing the w in LSSVM with data points and constants. After I resolve the KKT conditions for the LSSVM I got $$w = \sum ^N _{i=1} \alpha_i x_i$$ Is it possible to convert ...
4
votes
1answer
506 views

Prove that sample covariance matrix is positive definite [duplicate]

Consider the $p \times p$ sample covariance matrix: $$\mathbf{S} = \frac{1}{n-1} \cdot \mathbf{Y}_\mathbf{c}^\text{T} \mathbf{Y}_\mathbf{c} \quad \quad \quad \mathbf{Y}_\mathbf{c} = \mathbf{C} \mathbf{...
3
votes
1answer
238 views

Human intuition behind SVD in case of recommendation system

This does not answer my question. I struggled very hard to understand the SVD from a linear-algebra point of view. But in some cases I failed to connect the dots. So, I started to see all the ...
10
votes
3answers
683 views

SVD : Why right singular matrix is written as transpose

The SVD is always written as, A = U Σ V_Transpose The question is, Why is the right singular matrix written as V_Transpose? I mean lets say, W = V_Transpose and then write SVD as A = U Σ W SVD Image ...
0
votes
1answer
109 views

Penalized Canonical Correlation in R with PMA Module

I am trying to use sparse canonical correlation analysis as implemented in the R PMA package. I'm finding that the correlations output by the package seem slightly inconsistent with the ones you would ...
1
vote
0answers
31 views

Symbolic Singular Value Decomposition? U,S,V as function of the elements of M [closed]

Suppose we want to compute the SVD of $\mathbf{M} = \begin{bmatrix} m_{11} & m_{12} & m_{13} \\ m_{21} & m_{22} & m_{23} \\ m_{31} & m_{32} & m_{33} \end{bmatrix}$ (...
1
vote
1answer
32 views

Interpreting PCA results of first two components

I don't like the looks of my PCA graph here. PCA coordinates should be uncorrelated, yet the variance between the coordinates of the second component increases as the first component decreases. What's ...
1
vote
0answers
27 views

Use Matrix Factorization to predict probability of a recommendation system?

I have a dataset where I have a sparse utility matrix (user-product) with binary input: 1 if the user $i$ bought the product $j$, and 0 if it hasn't. However it has a different meaning on the test set,...
1
vote
0answers
188 views

Is there a version of NMF that normalizes the sum of scores of each sample?

I want to decompose a nonnegative data matrix $A \in \mathbb{R}^{n\times m}$ into nonnegative basis vectors $U \in \mathbb{R}^{n \times k}$ and a score matrix $V \in \mathbb{R}^{m \times k}$ such that ...
2
votes
1answer
80 views

ML model for Signal Decomposition [closed]

So recently I got a task which can be summarized as follows: Suppose we have 3 functions f1, f2, f3 and a certain combination of the functions gives us ...
2
votes
1answer
76 views

Decomposition of $x^T K x$ as $y^T y$

When $x$ is a vector of size Nx1, and $K$ is a very large symmetric sparse matrix of size NxN (say N=100K), is it possible to decompose $x^T K x$ as $y^T y$? As if I could get $y = K^{1/2} x$. Edit ...
3
votes
2answers
120 views

Cold starts in factorization - WALS projections

I read here (Google's crash course on recommendations) the following: Given a new item $i_0$ not seen in training, if the system has a few interactions with users, then the system can easily compute ...
1
vote
1answer
105 views

Does SVD provide the best low rank approximation for any matrix regardless of shape?

Wikipedia states (link below) that by the Eckart-Young-Mirsky theorem, the SVD provides the best low rank matrix approximation (on the basis of Frobenius norm of the error matrix) for any matrix A in ...

1
2 3 4 5 6