The tag has no wiki summary.

learn more… | top users | synonyms

4
votes
1answer
39 views

Generalized inverse solution to system of linear equations proof

I'm going through a set of course notes for an introduction to the theory of linear models class at my university. Unfortunately, the professor who wrote this note set is no longer at this school, and ...
0
votes
2answers
47 views

Sample covariance matrix and its inverse

Assume we have the sample covariance matrix $S_1 = XX'/k$ which is not positive definite (in fact it is positive semi-definite) and not well conditioned in very large dimension (large $p$, small $k$). ...
0
votes
0answers
19 views

Interpretation of (diagonalized) inverse covariance matrix

There are several threads here about covariance matrix and inverse covariance matrix interpretation (here, here or here). However, I was wondering how to interpret the inverse covariance matrix (or ...
2
votes
0answers
40 views

Regularization parameter to generate inverse covariance matrix

My data consists of approx. 5 Million binary strings (n) and every string is 2788 characters long. My goal is to find out if position i is correlated with position j. I approximated a covariance ...
4
votes
1answer
32 views

How to find factor that is making matrix singular

I have a 300+ column data.frame, and no matter how I break it up I get this error every time: ...
1
vote
2answers
35 views

reasons for a non-invertible matrix

I am trying to factor analyze a matrix in R and I keep getting errors that lead me to believe my matrix is non-invertible. What are the reasons a matrix could be non-invertible? The only one I found ...
4
votes
1answer
59 views

Cholesky factorization and forward substitution less accurate than inversion?

I recently asked this question asking for an efficient way to compute the Mahalanobis distance (without calculating the inverse). The accepted solution was to use the Cholesky factorization and ...
0
votes
1answer
27 views

Marginal Likelihood of a Gaussian Process Model, Duplicate entries in kernel matrix

I am trying to fit a Gaussian process model using the toolbox and I got stuck in the following problem. Assuming that I have some duplicated data points in my training data, then those will map to ...
0
votes
1answer
29 views

I need a matrix in order to calculate g-inverse of it [closed]

I want to calculate g-inverse of a matrix, which has 4 rows and not a square matrix and has no inverse. Please help me find such a sound (good) matrix. I only need a matrix. You can suggest a book or ...
0
votes
1answer
73 views

Understanding the marginal distribution of multivariate normal distribution

I am trying to better understand the multivariate normal distribution. Here I try to refer to the conditional distribution part of wiki also the fifth page of this tutorial. I do not quite ...
0
votes
0answers
21 views

Evaluate the multivariate normal using variance matrices $\boldsymbol{\Lambda}+\alpha_{i}\mathbf{a}\mathbf{a}^{T}+\beta_{j}\mathbf{b}\mathbf{b}^{T}$

I need to calculate a huge amount of inverses and determinants to evaluate the pdf of the multivariate Gaussian. Specifically I need to compute the inverses and determinants of the following ...
0
votes
0answers
41 views

Propogation of error in a matrix inversion

I'm trying to find the deterministic error bounds for some parameters calculated through distance geometry. The equation can be simplified to the following form: $ \left[\begin{matrix} x_1 \\ x_2 \\ ...
3
votes
1answer
75 views

How to calculate the inverse of sum of a Kronecker product and a diagonal matrix

I want to calculate the inverse of a matrix of the form $S = (A\otimes B+C)$, where $A$ and $B$ are symetric and invertible, $C$ is a diagonal matrix with positive elements. Basically if the ...
9
votes
1answer
151 views

Why does inversion of a covariance matrix yield partial correlations between random variables?

I heard that partial correlations between random variables can be found by inverting the covariance matrix and taking appropriate cells from such resulting precision matrix (this fact is mentioned in ...
3
votes
1answer
66 views

Invertibility of $X^TX$ with severe multicollinearity in regression

I am studying about multicollinearity in regression and in the book it says, "if there is severe (but not perfect) multicollinearity, two or more predictor variables are highly correlated, so $X^TX$ ...
0
votes
1answer
67 views

Matrix Inversion Error

I a Multiple linear regression model, from published literature, I am implementing a spreadsheet to generate new predictions based on the published model. the literature stated Coefficients and the ...
1
vote
0answers
133 views

R - system is computationally singular - dealing with small numbers

I'm working with a ~200x200 Markovian transition matrix of non-zero probabilities. Forcibly, these probabilities are, for the large part, going to be very small. I am trying to find the inverse of my ...
0
votes
0answers
93 views

Condition Number in Single Variable Regression

I've calculated a single variable linear regression using OLS (in the python statsmodels library). The model has a large condition number, and suggests there may ...
0
votes
1answer
23 views

Is it posible to find a derivative of the mean function of Gaussian process regression?

The mean function $\hat{\mu}(x_*)$ of GPR is $k(x_*, X)(k(X, X) + \sigma^2_w I)^{-1}Y$ where $k(\cdot, \cdot)$ is a kernel matrix or vector of appropriate size and is parametrized by some ...
4
votes
1answer
177 views

Diagonal elements of the inverted correlation matrix

Is it true that the diagonal elements of the inverted correlation matrix will always be larger than 1? Why?
1
vote
0answers
134 views

The inverse of AR correlation matrix

I want to find the inverse of the following matrix: $$ R_{k-1}=\begin{pmatrix} 1 &\rho &\rho^2 &\cdots &\rho^{k-2} \\ \rho &1 &\rho &\cdots ...
0
votes
0answers
22 views

Convergence of sample concentration matrix

I'm interested in the Frobenius or $\infty$ norm convergence rate bound for the sample inverse convariance (concentration) matrix. That is, suppose: $$ Y \sim \mathcal{N}_p\left(0, \Sigma\right) $$ ...
2
votes
1answer
204 views

How to proof relationship between inverse covariance matrix and linear regression coefficients?

Edited: I would like to work out the above relationship, more precisely: Let $(Y_{1}, ..., Y_{m})$ be a zero-mean vector with covariance matrix $\Sigma$, and let $S \subset \{1, ..., m\}.$ The ...
0
votes
0answers
56 views

A non-negative definate matrix has a non-negative generalized inverse

I'm having trouble proving a N.N.D matrix has a N.N.D G-Inverse. So far I have: If we assume x = Az where x >= 0 and A is a nnd matrix. So if Y is a G-inverse than: x = Az = YAz = Yx >= 0 . Thus ...
0
votes
1answer
121 views

Orthogonalizing predictors for least squares estimation

I know that orthogonalization in LS is to avoid inverting X'X. The idea behind it is to find variables Z that are orthogonal to each other. Although the process to find those is clear to me, I don't ...
1
vote
1answer
123 views

Tolerance for pseudoinverse

In calculating the pseudoinverse of a matrix $A$, of size (m,n), I need to choose a tolerance threshold for the eigenvalues. I'm trying to understand how I should pick this. Matlab default is to use ...
3
votes
2answers
264 views

Inverting non positive definite covariance matrix

I have an expression for a covariance matrix $C$ in terms of the indices $i$ and $j$. In this way I can analytically calculate the elements of my covariance matrix, however when I try to invert $C$ ...
0
votes
2answers
121 views

Returning the inverse of a matrix in a quadratic program (SVM) in cvx optimization package

I am solving the dual QP of an SVM, and using the RBF kernel. As you know, the objective function is of the form $$f(\alpha) = \alpha^T Q \alpha $$ where $\alpha$ is the optimization variable and $Q$ ...
3
votes
1answer
100 views

Uniqueness of $x'\beta$ even when $\mathbb{E}(x^Tx)$ is not invertible

As discussed in user25658's answer to this question, when one wants to compute $$ \beta = \mathbb{E}(x^Tx)^{-1} \mathbb{E}(x^TY) $$ but $\mathbb{E}(x^Tx)$ is not invertible, $\beta$ is not uniquely ...
2
votes
0answers
104 views

Interpretation of regression coefficients obtained from applying left inverse of regressor matrix in an underdetermined system?

If $X^\dagger$ is the pseudo-inverse of $X$, $\beta = X^\dagger y$ is the least squares solution for $\beta$ when $y=X\beta$. In the overdetermined case, applying $X^{\dagger,L} = (X^TX)^{-1}X^T$ ...
1
vote
2answers
113 views

IV estimator: efficient implementation?

I would like to implement (in R) an instrumental variable (IV) estimator, that takes the most general form (here not 2SLS or GMM!): $$ \beta_{IV} = (Z'X)^{-1}Z'Y $$ I could code this in the naive ...
14
votes
2answers
5k views

What does the inverse of covariance matrix say about data? (Intuitively)

I'm curious about the nature of $\Sigma^{-1}$. Can anybody tell something intuitive about "What does $\Sigma^{-1}$ say about data?" Edit: Thanks for replies After taking some fantastic courses, I'd ...
1
vote
2answers
929 views

Linear regression and non-invertibility

In linear regression there are two approaches for minimizing the cost function: The first one is using gradient descent. The second one is setting the derivative of the cost function to zero and ...
2
votes
1answer
216 views

Inverse matrix for contrast coding

I'm trying to understand how "user defined contrast coding" works. My question refers to the example from http://www.ats.ucla.edu/stat/r/library/contrast_coding.htm#User: ...
6
votes
2answers
297 views

Variance-covariance matrix of the parameter estimates wrongly calculated?

I fitted an hyperbolic distribution to my data with the hyperbFit(mydata,hessian=TRUE) command (package HyperbolicDist). The hessian looks like: ...
4
votes
2answers
4k views

Numerical Instability of calculating inverse covariance matrix

I have a 65 samples of 21-dimensional data (pasted here) and I am constructing the covariance matrix from it. When computed in C++ I get the covariance matrix pasted here. And when computed in matlab ...
-1
votes
1answer
304 views

Singular matrix: eigenvalues perturbation vs Moore-Penrose generalized inverse

We often face singular matrices in practice: OLS with singular (X'X), GMM with singular weighting matrix, singular matrix in Wald statistics. I'm wondering how can we overcome this issue. I've seen ...
3
votes
2answers
1k views

Ways to measure distance from multivariate Gaussian (Mahalanobis distance)

I have a cluster of p-dimensional points and given a new p-dimensional point $x$ I want to determine whether or not it is likely to belong to this cluster. The cluster is made up of $n$ ...
6
votes
1answer
4k views

What to do when sample covariance matrix is not invertible?

I am working on some clustering techniques, where for a given cluster of d-dimension vectors I assume a multivariate normal distribution and calculate the sample d-dimensional mean vector and the ...
3
votes
1answer
328 views

How to interpret the sum of the elements of an inverse covariance matrix?

In the derivation of global minimum variance portfolio, we get The $(Σ^{-1}1) /(1'Σ^{-1}1)$. What's the meaning of $1'Σ^{-1}1$ and $Σ^{-1}1$. $Σ$ is a covariance matrix of assets returns.
3
votes
1answer
135 views

Question about inverse in a two-step estimator as a joint GMM-estimators approach

I'm reading Newey & McFadden - Large sample estimation and hypothesis testing (in the Handbook of Econometrics, Volume 4, 1994, page 2178). My model which I'm interested in has some former ...
2
votes
2answers
713 views

How to get conditional variance from Schur complement?

Suppose you have vectors X and Y with covariance matrix $V = \left( \begin{array}{cc} A & B \\ B^T & C \end{array} \right)$. This Wikipedia article says that $Var(X | Y) = A - BC^{-1}B^T$, ...
2
votes
1answer
228 views

Correlation and Hotelling test

To find a Hotelling $T^2$ score it is necessary to calculate the covariance matrix and then invert it. Now, when the test is a two-sample $T^2$ test, the covariance matrix is a pooled matrix. When ...
0
votes
1answer
289 views

Pseudo Inverse Instead of Inverse with LDA?

I have implemented the LDA algorithm. However when I had to get the inverse of a matrix Matlab threw an error and I had to use pinv (pseudo inverse) instead of inverse. Did I do something wrong or is ...
2
votes
0answers
975 views

Geometric intuition for why an outer product of two vectors makes a correlation matrix? [closed]

I understand that the outer product of two vectors, say representing two detrended time series, can represent a cross-correlation (well covariance) matrix. I also know that the inverse of a ...
1
vote
1answer
776 views

matlab gmdistribution.fit 'Regularize' - what regularization method?

I am wondering what is behind matlab 'Regularize' option for method gmdistribution.fit. If it is simply adding a 'little' value to diagonal elements of covariance matrix, so as to make covariance ...
1
vote
0answers
127 views

Estimation accuracy of precision matrix

I have a couple of questions related to estimation of high-dimensional precision matrix (inverse of the covariance matrix) in the case where p is close to 100 and n < p. As a measure of estimation ...
3
votes
1answer
315 views

Moore-Penrose generalized determinant

Is there a function in R to calculate the generalized determinant of a singular matrix? (similar to the ginv() used to compute the generalized inverse)
7
votes
3answers
1k views

To use Discrete Fourier Transform to invert a covariance matrix

I am working on a problem that its difficult part is to invert a covariance matrix (in R). I could not use usual approches like SVD and Chol. Then, I decided to use a Discrete Fourier Transform (DFT) ...
1
vote
1answer
3k views

Problem with singular covariance matrices when doing Gaussian process regression

I'm working with Gaussian process regression. Currently I start testing different covariance functions and compositions to see what type of data they could describe best. I made an own implementation ...