The tag has no usage guidance.

learn more… | top users | synonyms

2
votes
0answers
48 views

If I multiply two non-negative matrices together and then do NMF, will the result be close to the matrices I started with?

I've been playing around with non-negative matrix factorization (NMF), just doing some example computation, and arrived at the following question: Suppose I pick two non-negative matrices $\textbf{W}$...
0
votes
0answers
17 views

matrix signal deconvolution

I measured the dietary habits of thousands of animals throughout the world, giving me the following matrix: X = (M x N), where Xij = measurement of food j in animal i M = number of animals N = ...
1
vote
0answers
30 views

What is the most efficient algorithm for online Non- negative Matrix Factorization (NMF)?

What is the most efficient algorithm for online Non- negative Matrix Factorization (NMF) in recent study? Thanks.
0
votes
2answers
29 views

How you can you take the `min` of what looks like a single value calculation in SVD++?

I'm reading through this paper: http://www.cs.rochester.edu/twiki/pub/Main/HarpSeminar/Factorization_Meets_the_Neighborhood-_a_Multifaceted_Collaborative_Filtering_Model.pdf And I'm looking at the ...
1
vote
0answers
37 views

Matrix Factorization in Recommender Systems: Uniqueness of SVD?

I was studying the collaborative filtering approach about recommender system and I read about matrix factorization approach. In SVD version, I have not figured out how the non-uniqueness of the ...
0
votes
1answer
21 views

$LDL^T$ decomposition from Cholesky decomposition

Suppose we have a covariance matrix $\Sigma$. I know that the Cholesky decomposition $A^T A$ can be found from the LDL decomposition using $$ \Sigma = LDL^T = (LD^{\frac 1 2})(LD^{ \frac 1 2 })^T = A^...
2
votes
1answer
37 views

Can singular value decomposition be applied to a matrix of $n\times 1$ size?

Can singular value decomposition be applied to a matrix of $n \times 1$ size (a vector)? Usually I see that matrix is of size of $n \times m$. Any example?
1
vote
1answer
33 views

Choleski decomposition of the covariance matrix

I have a process described as $r_t = \mu + \Sigma_t^{1/2}z_t$ where $z_t$ is let's say a standard normal distribution residual and $\Sigma_t$ is the conditional covariance matrix. The $t$ stands ...
0
votes
1answer
42 views

How to obtain the inverse of a matrix while solving an equation?

Given a matrix $A$, let us assume there is a equation: $Ax = b$ To solve for $x$, we can write: $x = A^{-1} b$ One way to obtain the inverse of A is by single value decomposition: Decomposition ...
2
votes
1answer
19 views

Information decomposition in GLM MLE derivation

I am trying to understand the derivation of the MLE estimates of $\beta=(\beta_1,\dots,\beta_p)$ in Generalized Linear Models. The elements of the Information matrix are given by: $$ J_{jk}=\sum_{i=1}^...
0
votes
0answers
21 views

Glove-word embeddings: adding a new word to the vocabulary

The Glove model creates embeddings of words using SGD for matrix factorization. The basic idea is: Tabulate co-occurence counts ( $X_{ij})$ Factorize the matrix: approximate the counts using ...
1
vote
0answers
38 views

Application of Givens rotation to two matrices

I am reading this paper on Multiresolution Matrix Fatorization, http://arxiv.org/pdf/1507.04396v1.pdf, and have come across something that seems like an error to me. In Algorithm 2, the authors take $...
0
votes
1answer
27 views

Factor rotations in non-negative matrix factorization?

My understanding is that solutions from Non-Negative Matrix Factorization (NMF) are not necessarily unique, and rotations can be imposed during the optimization process or after the solutions have ...
1
vote
1answer
63 views

Evaluate performance of non-negative matrix factorization (NMF)

I have a complex pipeline for predictive modeling of text, where the non-negative matrix factorization (NMF) is one part. I would like to evaluate the performance of the NMF independently of the ...
0
votes
0answers
80 views

A technique like truncated SVD that uses non-orthogonal components?

Question: Is there a technique like truncated SVD that instead of orthogonal components relies on non-orthogonal components? Specifically, just like a truncated SVD computes simultaneously the ...
1
vote
0answers
19 views

How is this gradient derived?

I am reading this paper, http://math.ucla.edu/~dakuang/pub/sdm0125.pdf, and came across this function, $f(x) = ||A-H^TH||_F^2$. Later, the authors take the gradient of this function, $\nabla f$, but ...
0
votes
1answer
102 views

Optimization with both L1 and L2 regularization

After doing some research I suppose the hard part is that, L2 regularized problem is often solved by gradient descent, while L1 regularized problem is often solved by coordinate descent. But which ...
0
votes
0answers
21 views

The orthogonal complement of the range

I am reading about the pseudo inverse, and there is a concept that I am not quite able to understand. It would be nice if someone could give me a clue of how to work it out, or an easy example on how ...
0
votes
1answer
43 views

Prediction of unknown triples for relational learning tasks using RESCAL

Background: I am focusing on a relational learning task, where links between entities are predicted across several relations. An example of a relation in this task is if two entities have the same ...
0
votes
1answer
160 views

What is UV decomposition?

As I'm reading about different matrix decomposition methods, I see a reference to a decomposition method that is known as UV method where: U: has small number of columns V: has small number of rows ...
1
vote
1answer
86 views

Markov Cluster Algorithm transition matrix

I am reading the notes on Markov Cluster Algorithm by Kathy Macropol (http://www.cs.ucsb.edu/~xyan/classes/CS595D-2009winter/MCL_Presentation2.pdf) On slide 14/46 the author talks about inflation and ...
0
votes
0answers
8 views

How to determine if a lower rank approximation exists?

In all the literature I read on low-rank approximations, I have yet to run across situations in which people first check to see if a lower rank exists. My understanding is that a matrix whose ...
1
vote
1answer
85 views

Mahalanobis distance with LDL decomposition

I've got an extended Kalman filter with innovation covariance defined as $\mathbf{W}=\mathbf{H}\mathbf{P}\mathbf{H}^\textrm{T} + \mathbf{R}$. I want to know the squared Mahalanobis distance $\|z\|^2$ ...
2
votes
0answers
53 views

Confirming an understanding of SVD

I'm trying to sift through the concepts of low-rank approximations, matrix factorizations and SVD. There's a lot of info out there and the rabbit hole is deep so I just want to make sure my high level ...
0
votes
0answers
25 views

Comparison of two Products of Variance

Let $A$, $B$ and $B'$ be random variables, and consider $\text{Var}(AB) \text{ and} \text{ Var}(AB')$, such that $\text{Cov}(A,B)\ne 0$, $\text{Cov}(A,B')\ne 0$ and $\text{Var}(B)\geq \text{Var}(B'...
6
votes
1answer
124 views

Explain how `eigen` helps inverting a matrix

My question relates to a computation technique exploited in geoR:::.negloglik.GRF or geoR:::solve.geoR. In a linear mixed model ...
6
votes
0answers
168 views

State-of-the-art in Collaborative Filtering

I am working on a project for collaborative filtering (CF), i.e. completing a partially observed matrix or more generally tensor. I am a newbie to the field, and for this project eventually I have to ...
7
votes
1answer
624 views

Updating SVD decomposition after adding one new row to the matrix

Suppose that I have a dense matrix $ \textbf{A}$ of $m \times n$ size, with SVD decomposition $$\mathbf{A}=\mathbf{USV}^\top.$$ In R I can calculate the SVD as ...
4
votes
1answer
59 views

When do we use matrix norm?

When do we use matrix norm? matrix norm is one of the property of a matrix, but I am not sure when I will use it. Do we use it for calculating a upper bound of a matrix? https://en.wikipedia.org/wiki/...
2
votes
1answer
190 views

Overfitting in Matrix Factorization models used in Recommender Systems (Collaborative Filtering)

I'm wondering if I should check if a Matrix Factorization model I built for recommendation by Collaborative Filtering is overfitting. I trained a model using MLlib ALS (Alternating Least Squares) ...
0
votes
0answers
82 views

Calculation for variance of slope for linear model in R predict function

I'm looking through the code for the predict.lm function in R, and am confused about one of the calculations. I believe it is the calculation used to determine the ...
0
votes
1answer
46 views

Convergence of eigenvectors and eigenvalues of matrix that converges

For each random variable $X=x$, there is a symmetric positive definite matrix $M(x)$. Suppose there is a set of samples of random matrix $M_1,M_2,...,M_n$, where each $M_i$ is calculated based on the ...
2
votes
0answers
29 views

eigen problem of random matrix

Suppose I need to find the eigen vectors of a matrix $M(X)$ for any sample $X$ to be my dimension reduction projector. Then since I have samples $X_1,...,X_n$, so we propose to use the eigen vectors ...
2
votes
1answer
655 views

Scaling a Covariance Matrix by a factor?

I have a $n\times n$ covariance matrix representing the covariance between n random processes. I want to increase all of the covariances by a certain factor, say $1.5x$. Multiplying each covariance ...
0
votes
0answers
9 views

“Tucker decomposition often assumes Gaussian noises”. Are the noises here input or output?

I saw in some papers saying " Tucker decomposition methods have implicitly or explicitly assumed the noises are Gaussian" in http://www.bsp.brain.riken.jp/publications/2012/MBSS_BPAS-Zhou-Cichocki....
0
votes
2answers
60 views

optimizing over a set of symmetric matrices

I need to minimize a complicated loss function, $f\left(\Lambda\right)$ over a set of symmetric matrices, $S_{p}$ of dimension p, such that all the eigenvalues of $\Lambda \in \left[0,1\right]$. I ...
1
vote
1answer
155 views

Calculating Cosine Similarity with Matrix Decomposition (matrix multiplication with normalized columns)

To calculate the column cosine similarity of $\mathbf{R} \in \mathbb{R}^{m \times n}$, $\mathbf{R}$ is normalized by Norm2 of their columns, then the cosine similarity is calculated as $$\text{...
1
vote
0answers
23 views

Shrinkage of Schafer and Strimmer

As we all know that the sample covariance matrix $(S = (s_{ij}))$ is postive definite when the number of observations is smaller than the number of samples, that is n>p. But, the sample covariance ...
1
vote
0answers
71 views

Interpretation of matrix factorization results

Matrix factorization methods are known to give good results pertaining to problems like movie recommendation. The method reduces the feature space, which is then used for recommendations. For example ...
0
votes
1answer
117 views

How to find unknown correlation coefficients in a correlation matrix from known correlation coefficients? [duplicate]

I have a correlation matrix A given below. Here A should be a positive-definite matrix so that we can perform Cholesky decomposition of A. ...
0
votes
0answers
206 views

Non-negative matrix factorization in recommender systems

As i understand, in NMF we should have our three matrices elements non-negative. But i can't understand how to do it so far. Shouldn't we just initialize our factor matrices at the start with random ...
2
votes
2answers
72 views

Singular value and eigen-decomposition of a square symmetric matrix should be identical, but differ in sign

As far as I know, singular value decomposition (SVD) and eigendecomposition give the same result for symmetric square matrices. But when I check the results in R, that's not what I see. Please see ...
1
vote
1answer
355 views

What is the relationship between SVD and UV decomposition?

I am aware that both are Matrix decomposition techniques. SVD decomposes the Matrix as $\mathbf C = \mathbf U_1 \mathbf L \mathbf V_1^\top$. UV decomposes the Matrix as $\mathbf C = \mathbf ...
2
votes
0answers
163 views

orthogonalized impulse response's contradictory forms in a VAR(p) model

I have so far discovered three different ways of utilizing the Cholesky decomposition for calculating the OIRFs of a VAR(k). The different methods seem contradictory so I would like some input on ...
2
votes
1answer
139 views

Decomposition of inverse covariance matrix

Let $\Sigma$ be a covariance matrix and let $$x^T \Sigma^{-1} x = \|Ax\|_2^2.$$ What is the interpretation of matrix $A$? I tried solving for $A$ with an eigenvalue decomposition of $\Sigma$ as ...
3
votes
1answer
235 views

Derivative of $x^T A^Ty$ with respect to $\Sigma$ where $A$ is (an upper triangle matrix and ) Cholesky decomposition of $\Sigma$

I would like to evaluate: $$ \frac{ \partial x^T A^Ty}{\partial \Sigma} $$ where $A$ is a Cholesky decomposition of $\Sigma$ and an upper triangle matrix such that $\Sigma = A^T A$, $x$ and $y$ are a ...
0
votes
1answer
205 views

Relationship between Poisson generation and generalized Kullback-Leibler divergence

I have read that, in the context of matrix factorization, performing maximum likelihood estimation under the assumption that the entries are Poisson generated is equivalent to minimizing the ...
2
votes
0answers
114 views

What is the advantage of non-negativity in matrix factorization?

I am wondering why matrix factorization techniques in the machine learning domain almost always expect the provided matrix to be non-negative. What is the advantage of this constraint? Background: I ...
12
votes
1answer
1k views

What norm of the reconstruction error is minimized by the low-rank approximation matrix obtained with PCA?

Given a PCA (or SVD) approximation of matrix $X$ with a matrix $\hat X$, we know that $\hat X$ is the best low-rank approximation of $X$. Is this according to the induced $\parallel \cdot \...
1
vote
2answers
313 views

Maximally reducing the rank of a matrix by removing some rows or columns

I have a $N \times M$ matrix, and the rank of matrix, $r$, is near $\min(M,N)$. I want to minimize the rank by removing some of the rows or columns to get $r \ll \min(M,N)$. The goal is to achieve ...