Questions tagged [matrix-decomposition]

Matrix decomposition refers to the process of factorizing a matrix into a product of smaller matrices. By decomposing a large matrix, one can efficiently perform many matrix algorithms.

Filter by
Sorted by
Tagged with
0
votes
1answer
25 views

Matrix Factorization and Linear Regression

Which matrix factorization algorithm is used in LinearRegression() function of scikit-learn?
1
vote
0answers
95 views

Adding explicit user info to matrix factorization

In the paper Matrix Factorization Techniques for Recommender Systems, it is claimed that we can incorporate extra user information into our recommender model by doing something like this: $$ \hat{r}_{...
1
vote
1answer
17 views

Understanding the decomposed matrices in Singular Value Decomposition [duplicate]

I am trying to get some intuition regarding the U and V matrices in SVD ($M=UEV^T$). I think these are orthonormal basis vectors, but I am struggling to get an intuition if they represent anything ...
1
vote
1answer
26 views

Why is there a discrepancy between the eigenvalues of the covariance matrix (PCA) and the eigenvalues of the kernel matrix (kernel PCA)?

I've done PCA on my data matrix $ \mathbf{X} $ which gives me i.a. the eigenvalues $ \lambda $ and eigenvectors $ v $ of the data covariance matrix $ C=\mathbf{X}^T \mathbf{X} $. I'm now extending my ...
0
votes
0answers
29 views

Why NNMF (non-negative matrix factorization) is a method for linear dimensionality reduction?

Some sources (for example this) say that NNMF is a method for linear dimensionality reduction. How to prove this statement? I see two different explanations of this and I want to know which of them (...
0
votes
0answers
13 views

SVD - vectors in matrix A

In SVD we have $A = U \Sigma V^T$. When applying it for ML, e.g. to calculate Moore-Penrose pseudoinverse for linear regression, I have seen that we take columns of $A$ as vectors. Typically in ML I ...
15
votes
1answer
759 views

Relationship between Cholesky decomposition and matrix inversion?

I've been reviewing Gaussian Processes and, from what I can tell, there's some debate whether the "covariance matrix" (returned by the kernel), which needs to be inverted, should be done so ...
1
vote
0answers
39 views

Alternating least squares — what are the limitations?

I am taking an introductory course to Machine Learning and we learned alternating least squares for recommender systems. I learned that this method has some advantages --- easy to parallelize, and ...
0
votes
0answers
131 views

Offline Precision@k and Recall@k for recommender system

How can I evaluate offline Precision@k and Recall@k metrics for recommender system if I only have items-users matrix? I think I can't just compare recommendations and user data because it will be ...
0
votes
0answers
7 views

Physical significance of non-negative factors of a matrix?

I was trying to make a recommender system using matrix factorization techniques on rating data. I came across 2 algorithms - SVD and NMF. While the basic difference is very clear , I was wondering ...
2
votes
0answers
33 views

Convex Optimization Algorithm for Additive Matrix Decomposition

Given an m $\times$ n matrix $Y = S + L + E$, where $S$ is a sparse matrix, $L$ is a low rank matrix and $E$ is a noise matrix, I want to recover $S$, $L$. One of the techniques given in Agarwal et al ...
0
votes
0answers
9 views

Negative PCA component causing negative predictions in interval target variable with range 0 -1

By applying first a PCA analysis to a set of positive only variables (all binary), I get 5 components which are subsequently used in a non-negative factorized machine to predict a target variable (...
1
vote
0answers
44 views

Multivariate Normal Quadratic MGF: Eigendecomposition to Matrix form

If $X \sim \mathcal{N}(\mu, \Sigma)$ is a multivariate normal, then the quadratic $X^TAX$ has moment generating function $$M_{X^TAX}(t)= \frac{1}{\sqrt{\det(I - 2tA\Sigma)}}\exp\left(-\frac{1}{2}\mu'[...
0
votes
0answers
35 views

Optimisation of reconstruction error for PCA

I am trying to understand the math behind the minimal reconstruction error for PCA models in this book. Given a data matrix $Y$ which has dimensions $D\times n$ and assuming that it is centered. The ...
0
votes
0answers
28 views

What can I learn about the dimensions with highest variance of a matrix $M\approx L^TR$ from looking at $L$?

I have a high-dimensional, symmetric data matrix $M\in\mathbb{R}^{d\times d}$ , which is factorized by two matrices $L, R\in \mathbb{R}^{n\times d}$ : $L^TR\approx M$, where $n$ is much smaller than $...
0
votes
0answers
6 views

variance explained for CP factorisation

What is the best way to compute the variance explained for one or more terms in a CP tensor factorisation? Is it even defined? In PCA this is made easy by the fact that the eigenvalues are orthogonal. ...
0
votes
0answers
20 views

Recommending item category based on customer shopping behavior?

I'm trying to build some recommender system for online wine shop. What I'm thinking is recommending item category (ex: red_pinot, ...
1
vote
0answers
23 views

Fastest way to find Leading singular value and vector (power iteration, rsvd etc)

I want to know the fastest way to find out the leading singular value and vector of a large rectangular matrix. I have seen 2 suggestions and have questions on both of them : Power Method : For this ...
1
vote
0answers
51 views

How to explain the numerical discrepancy between FactoMineR::PCA() and the svd() in their output of the U matrix?

I am comparing the output of two functions in R to do Principal Component Analysis (PCA), the FactoMineR::PCA() and the ...
3
votes
0answers
82 views

Efficient computation of marginalized multivariate normal posterior distribution

In general,if we know that the marginal Gaussian distribution for some variable $\textbf{x}$ and a conditional Gaussian distribution for some $\textbf{y}|\textbf{x}$ of the forms: $$p(\textbf{x}) = \...
0
votes
0answers
70 views

Covariance Matrix Decomposition - Data Decorrelation

So I recently found out about Mahalanobis distance. Given a r.v $x$ in N-dimensional space, an associated metric is defined by $$M(x) = \sqrt{(x-\mu)^T S^{-1}(x-\mu)}$$ where $\mu$ and $S$ are mean ...
1
vote
1answer
27 views

How to compute the left singular eigenvector matrix (U) from the output of prcomp() for PCA in R?

I am examining the output of the prcomp() function in R for PCA in light of the singular value decomposition equation: $X = U \cdot \Sigma \cdot V^{T}$, where: $X$: ...
0
votes
0answers
17 views

Possible typo Logistic Matrix Factorisation?

In the paper Logistic Matrix Factorization for Implicit Feedback Data by CC. Johnson(see link). The author stated his maximum likelihood function to be (omit all the indices): $\prod p(l|X,Y,\beta)^{\...
0
votes
0answers
53 views

Implicit feedback ALS algorithm: the alpha parameter

I'm creating a recommender system for a video streaming service. My only knowledge about the user preference on a video is the watched percentage of that video. I'm using the implicit feedback ALS ...
1
vote
0answers
14 views

How to find upper and lower bound

Let $\Sigma \in S_{++}^n$ be a symmteric positive definte matrix with all diagonal entries one. Let $U \in R^{n \times k_1}$, $W \in R^{n \times k_2}$, $\Lambda \in R^{k_1 \times k_1}$ and $T \in R^{...
1
vote
0answers
22 views

Kernel matrix decomposition

I had a look at the sklearn.kernel_approxiamtion.Nystroem implementation, which is also described in this post: Nystroem Method for Kernel Approximation Here, a ...
0
votes
0answers
29 views

How to predict for test set when training a recommender by decomposing the utility matrix X=UV?

This probably sounds stupid but I don't get the workflow of building a recommending system by the utility matrix: X[i,j] = how much the ith user likes the jth object. For practical issues I refer to ...
1
vote
0answers
36 views

SVD for a complex data matrix — what is the meaning of the columns of $V$?

I've read this wonderful explanation of SVD, where the writer mentions that the columns of $V$ are the principal directions (Summary, #1). Is this also true when the data matrix $X$ is complex? If I'm ...
1
vote
1answer
41 views

Convert the following expression w.r.t to the whole dataset instead of element of the dataset?

I am in the process of expressing the w in LSSVM with data points and constants. After I resolve the KKT conditions for the LSSVM I got $$w = \sum ^N _{i=1} \alpha_i x_i$$ Is it possible to convert ...
4
votes
1answer
293 views

Prove that sample covariance matrix is positive definite [duplicate]

Consider the $p \times p$ sample covariance matrix: $$\mathbf{S} = \frac{1}{n-1} \cdot \mathbf{Y}_\mathbf{c}^\text{T} \mathbf{Y}_\mathbf{c} \quad \quad \quad \mathbf{Y}_\mathbf{c} = \mathbf{C} \mathbf{...
0
votes
0answers
38 views

Confusion on vector, matrix notation on research papers and the need of row normalization

I attempt to understand the formulation of dictionary learning for this paper: Depression Detection via Harvesting Social Media: A Multimodal Dictionary Learning Solution Multimodal Task-Driven ...
3
votes
1answer
132 views

Human intuition behind SVD in case of recommendation system

This does not answer my question. I struggled very hard to understand the SVD from a linear-algebra point of view. But in some cases I failed to connect the dots. So, I started to see all the ...
10
votes
3answers
608 views

SVD : Why right singular matrix is written as transpose

The SVD is always written as, A = U Σ V_Transpose The question is, Why is the right singular matrix written as V_Transpose? I mean lets say, W = V_Transpose and then write SVD as A = U Σ W SVD Image ...
0
votes
1answer
72 views

Penalized Canonical Correlation in R with PMA Module

I am trying to use sparse canonical correlation analysis as implemented in the R PMA package. I'm finding that the correlations output by the package seem slightly inconsistent with the ones you would ...
0
votes
0answers
34 views

How to project a mxn matrix (m features, n samples) onto a space generated by a mxk matrix (m features, k factors)?

I got two matrices A and B, the dimension of A is m x n, where n represents the number of samples, and m represents the number of features. Then after dimensionality reduction, e.g., by NMF, I got B, ...
1
vote
0answers
17 views

Symbolic Singular Value Decomposition? U,S,V as function of the elements of M [closed]

Suppose we want to compute the SVD of $\mathbf{M} = \begin{bmatrix} m_{11} & m_{12} & m_{13} \\ m_{21} & m_{22} & m_{23} \\ m_{31} & m_{32} & m_{33} \end{bmatrix}$ (...
1
vote
1answer
30 views

Interpreting PCA results of first two components

I don't like the looks of my PCA graph here. PCA coordinates should be uncorrelated, yet the variance between the coordinates of the second component increases as the first component decreases. What's ...
0
votes
0answers
42 views

Is matrix factorization in collaborative filtering equivalent to a special type of 3-layer neural networks for multi-class classification?

In Andrew Ng's Coursera course Machine Learning, he introduced a collaborative filtering algorithm, which I later checked to be called matrix factorization, where the optimization objective is $$ \...
1
vote
0answers
21 views

Use Matrix Factorization to predict probability of a recommendation system?

I have a dataset where I have a sparse utility matrix (user-product) with binary input: 1 if the user $i$ bought the product $j$, and 0 if it hasn't. However it has a different meaning on the test set,...
1
vote
0answers
117 views

Is there a version of NMF that normalizes the sum of scores of each sample?

I want to decompose a nonnegative data matrix $A \in \mathbb{R}^{n\times m}$ into nonnegative basis vectors $U \in \mathbb{R}^{n \times k}$ and a score matrix $V \in \mathbb{R}^{m \times k}$ such that ...
2
votes
1answer
71 views

ML model for Signal Decomposition [closed]

So recently I got a task which can be summarized as follows: Suppose we have 3 functions f1, f2, f3 and a certain combination of the functions gives us ...
0
votes
0answers
16 views

What is gravity in the context of “folding” in recommender systems?

What is "gravity" in the context of recommender systems? More specifically, how is it supposed to help with the "folding" problem where irrelevant queries may be returned if we don't provide ...
2
votes
1answer
75 views

Decomposition of $x^T K x$ as $y^T y$

When $x$ is a vector of size Nx1, and $K$ is a very large symmetric sparse matrix of size NxN (say N=100K), is it possible to decompose $x^T K x$ as $y^T y$? As if I could get $y = K^{1/2} x$. Edit ...
2
votes
2answers
86 views

Cold starts in factorization - WALS projections

I read here (Google's crash course on recommendations) the following: Given a new item $i_0$ not seen in training, if the system has a few interactions with users, then the system can easily compute ...
0
votes
0answers
39 views

SGD for matrix factorization: Negative sampling and gravity

I read here the following downside of using SGD for Matrix factorization: Harder to handle the unobserved entries (need to use negative sampling or gravity). What do they mean by negative sampling ...
1
vote
1answer
74 views

Does SVD provide the best low rank approximation for any matrix regardless of shape?

Wikipedia states (link below) that by the Eckart-Young-Mirsky theorem, the SVD provides the best low rank matrix approximation (on the basis of Frobenius norm of the error matrix) for any matrix A in ...
2
votes
1answer
206 views

How does multicollinearity affect the eigenvalues of a matrix?

I have been looking into ridge regression as a method to address multicollinearity in data. I am aware that multicollinearity can cause high variance in coefficient estimates. I have seen equations ...
6
votes
2answers
293 views

What is finite precision arithmetic and how does it affect SVD when computed by computers?

Was reading the paper "DETECTING AND ASSESSING THE PROBLEMS CAUSED BY MULTICOLLINEARITY:A USE OF THE SINGULAR-VALUE DECOMPOSITION" by David Belsley and Virginia Klema. After performing SVD, while ...
0
votes
2answers
36 views

How does the dimensions of my regression matrix effect its eigenvalules?

I was reading through another question answer where it discussed the inversion of the XX' matrix in ridge regression. It stated that it is impossible to have positive eigenvalues if matrix $XX'$(=$VDV'...
1
vote
0answers
20 views

Generalization Error and Matrix Factorizations?

This is more of a discussion/conceptual idea but: Is the notion of generalization error well-defined for a problem such as matrix factorization? I'm working with tensors/matrices and performing ...

1
2 3 4 5 6