The tag has no usage guidance.

learn more… | top users | synonyms

4
votes
3answers
329 views

Matrix inverse not able to be calculated while determinant is non-zero

I was attempting to calculate an OLS regression in R when I saw some strange things. The inverse of a square matrix does not exist if and only if the determinants is 0. Also, the matrix must be of ...
2
votes
1answer
228 views

What is the purpose of inverse distance?

In many econometric models, they use the inverse distance, instead of just distance. For example, if they are looking at the impact of distance to CBD on land values, they might use the inverse ...
1
vote
1answer
42 views

Efficient, stable inverse of a patterned covariance matrix for gridded data

I have computed a stationary covariance matrix defined for data on a grid. The data y are regularly spaced in 3D, lexicographically ordered in the covariance matrix, and I'm using a using a square ...
6
votes
1answer
110 views

Explain how `eigen` helps inverting a matrix

My question relates to a computation technique exploited in geoR:::.negloglik.GRF or geoR:::solve.geoR. In a linear mixed model ...
2
votes
3answers
178 views

What does that statistically mean , if $(X'X)^{-1}$ does not exist?

It is not a real world case , but suppose that we have $n$ observations and $k$ variables , since $k= n - 1 $ , if $X$ is the design matrix,$(X'X)$ will be a square matrix , so What does that ...
2
votes
1answer
75 views

Notation for ordinary and ridge regression

I'm trying to obtain an estimator $f(x)=y$ where $x \in \mathbb{R}^{D_1}$ and $y \in \mathbb{R}^{D_2}$, both are column vectors. So my training set $X$ and $Y$ are data matrices of size $D_1 \times ...
1
vote
0answers
23 views

Deriving expectation involving Wishart distributions $E[\bf{A(A'WA)^-A'W}]=\bf{A(A'\Sigma A)^-A'\Sigma}$

I have a problem deriving two expectation involving Wishart distributions with mean zero. Let $\bf{W} \sim {W_p}({\bf{\Sigma }},$$n\bf{)}$ and $\bf{A}$$: p\times q$. Prove that ...
1
vote
0answers
51 views

System exactly singular with pgmm (package plm)

this is my first post, I'll do my best to be clear and complete. I am trying to run a pgmm regression (Arellano Bond estimator) following the example online with the EmplUK dataset. My dataset is ...
1
vote
1answer
26 views

Computing negative half power of a bandwidth matrix for multivariate KDE

I am currently working on a project on density-based clustering algorithms and I am trying to make sense of the formula for a multivariate kernel density estimator [see wiki]. $$ ...
4
votes
1answer
121 views

Generalized inverse solution to system of linear equations proof

I'm going through a set of course notes for an introduction to the theory of linear models class at my university. Unfortunately, the professor who wrote this note set is no longer at this school, and ...
0
votes
2answers
82 views

Sample covariance matrix and its inverse

Assume we have the sample covariance matrix $S_1 = XX'/k$ which is not positive definite (in fact it is positive semi-definite) and not well conditioned in very large dimension (large $p$, small $k$). ...
1
vote
1answer
64 views

Interpretation of (diagonalized) inverse covariance matrix

There are several threads here about covariance matrix and inverse covariance matrix interpretation (here, here or here). However, I was wondering how to interpret the inverse covariance matrix (or ...
2
votes
0answers
72 views

Regularization parameter to generate inverse covariance matrix

My data consists of approx. 5 Million binary strings (n) and every string is 2788 characters long. My goal is to find out if position i is correlated with position j. I approximated a covariance ...
4
votes
1answer
111 views

How to find factor that is making matrix singular

I have a 300+ column data.frame, and no matter how I break it up I get this error every time: ...
1
vote
2answers
133 views

reasons for a non-invertible matrix

I am trying to factor analyze a matrix in R and I keep getting errors that lead me to believe my matrix is non-invertible. What are the reasons a matrix could be non-invertible? The only one I found ...
4
votes
1answer
132 views

Cholesky factorization and forward substitution less accurate than inversion?

I recently asked this question asking for an efficient way to compute the Mahalanobis distance (without calculating the inverse). The accepted solution was to use the Cholesky factorization and ...
0
votes
1answer
36 views

Marginal Likelihood of a Gaussian Process Model, Duplicate entries in kernel matrix

I am trying to fit a Gaussian process model using the toolbox and I got stuck in the following problem. Assuming that I have some duplicated data points in my training data, then those will map to ...
0
votes
1answer
31 views

I need a matrix in order to calculate g-inverse of it [closed]

I want to calculate g-inverse of a matrix, which has 4 rows and not a square matrix and has no inverse. Please help me find such a sound (good) matrix. I only need a matrix. You can suggest a book or ...
2
votes
2answers
164 views

Understanding the marginal distribution of multivariate normal distribution

I am trying to better understand the multivariate normal distribution. Here I try to refer to the conditional distribution part of wiki also the fifth page of this tutorial. I do not quite ...
0
votes
0answers
29 views

Evaluate the multivariate normal using variance matrices $\boldsymbol{\Lambda}+\alpha_{i}\mathbf{a}\mathbf{a}^{T}+\beta_{j}\mathbf{b}\mathbf{b}^{T}$

I need to calculate a huge amount of inverses and determinants to evaluate the pdf of the multivariate Gaussian. Specifically I need to compute the inverses and determinants of the following ...
0
votes
0answers
99 views

Propogation of error in a matrix inversion

I'm trying to find the deterministic error bounds for some parameters calculated through distance geometry. The equation can be simplified to the following form: $ \left[\begin{matrix} x_1 \\ x_2 \\ ...
4
votes
1answer
107 views

How to calculate the inverse of sum of a Kronecker product and a diagonal matrix

I want to calculate the inverse of a matrix of the form $S = (A\otimes B+C)$, where $A$ and $B$ are symetric and invertible, $C$ is a diagonal matrix with positive elements. Basically if the ...
12
votes
1answer
673 views

Why does inversion of a covariance matrix yield partial correlations between random variables?

I heard that partial correlations between random variables can be found by inverting the covariance matrix and taking appropriate cells from such resulting precision matrix (this fact is mentioned in ...
3
votes
1answer
104 views

Invertibility of $X^TX$ with severe multicollinearity in regression

I am studying about multicollinearity in regression and in the book it says, "if there is severe (but not perfect) multicollinearity, two or more predictor variables are highly correlated, so $X^TX$ ...
0
votes
1answer
91 views

Matrix Inversion Error

I a Multiple linear regression model, from published literature, I am implementing a spreadsheet to generate new predictions based on the published model. the literature stated Coefficients and the ...
1
vote
0answers
270 views

R - system is computationally singular - dealing with small numbers

I'm working with a ~200x200 Markovian transition matrix of non-zero probabilities. Forcibly, these probabilities are, for the large part, going to be very small. I am trying to find the inverse of my ...
1
vote
0answers
170 views

Condition Number in Single Variable Regression

I've calculated a single variable linear regression using OLS (in the python statsmodels library). The model has a large condition number, and suggests there may ...
0
votes
1answer
28 views

Is it posible to find a derivative of the mean function of Gaussian process regression?

The mean function $\hat{\mu}(x_*)$ of GPR is $k(x_*, X)(k(X, X) + \sigma^2_w I)^{-1}Y$ where $k(\cdot, \cdot)$ is a kernel matrix or vector of appropriate size and is parametrized by some ...
6
votes
1answer
261 views

Diagonal elements of the inverted correlation matrix

Is it true that the diagonal elements of the inverted correlation matrix will always be larger than 1? Why?
1
vote
0answers
198 views

The inverse of AR correlation matrix

I want to find the inverse of the following matrix: $$ R_{k-1}=\begin{pmatrix} 1 &\rho &\rho^2 &\cdots &\rho^{k-2} \\ \rho &1 &\rho &\cdots ...
2
votes
1answer
377 views

How to proof relationship between inverse covariance matrix and linear regression coefficients?

Edited: I would like to work out the above relationship, more precisely: Let $(Y_{1}, ..., Y_{m})$ be a zero-mean vector with covariance matrix $\Sigma$, and let $S \subset \{1, ..., m\}.$ The ...
0
votes
0answers
75 views

A non-negative definate matrix has a non-negative generalized inverse

I'm having trouble proving a N.N.D matrix has a N.N.D G-Inverse. So far I have: If we assume x = Az where x >= 0 and A is a nnd matrix. So if Y is a G-inverse than: x = Az = YAz = Yx >= 0 . Thus ...
0
votes
1answer
134 views

Orthogonalizing predictors for least squares estimation

I know that orthogonalization in LS is to avoid inverting X'X. The idea behind it is to find variables Z that are orthogonal to each other. Although the process to find those is clear to me, I don't ...
1
vote
1answer
205 views

Tolerance for pseudoinverse

In calculating the pseudoinverse of a matrix $A$, of size (m,n), I need to choose a tolerance threshold for the eigenvalues. I'm trying to understand how I should pick this. Matlab default is to use ...
3
votes
2answers
360 views

Inverting non positive definite covariance matrix

I have an expression for a covariance matrix $C$ in terms of the indices $i$ and $j$. In this way I can analytically calculate the elements of my covariance matrix, however when I try to invert $C$ ...
0
votes
2answers
160 views

Returning the inverse of a matrix in a quadratic program (SVM) in cvx optimization package

I am solving the dual QP of an SVM, and using the RBF kernel. As you know, the objective function is of the form $$f(\alpha) = \alpha^T Q \alpha $$ where $\alpha$ is the optimization variable and $Q$ ...
3
votes
1answer
104 views

Uniqueness of $x'\beta$ even when $\mathbb{E}(x^Tx)$ is not invertible

As discussed in user25658's answer to this question, when one wants to compute $$ \beta = \mathbb{E}(x^Tx)^{-1} \mathbb{E}(x^TY) $$ but $\mathbb{E}(x^Tx)$ is not invertible, $\beta$ is not uniquely ...
2
votes
0answers
118 views

Interpretation of regression coefficients obtained from applying left inverse of regressor matrix in an underdetermined system?

If $X^\dagger$ is the pseudo-inverse of $X$, $\beta = X^\dagger y$ is the least squares solution for $\beta$ when $y=X\beta$. In the overdetermined case, applying $X^{\dagger,L} = (X^TX)^{-1}X^T$ ...
1
vote
2answers
120 views

IV estimator: efficient implementation?

I would like to implement (in R) an instrumental variable (IV) estimator, that takes the most general form (here not 2SLS or GMM!): $$ \beta_{IV} = (Z'X)^{-1}Z'Y $$ I could code this in the naive ...
23
votes
2answers
8k views

What does the inverse of covariance matrix say about data? (Intuitively)

I'm curious about the nature of $\Sigma^{-1}$. Can anybody tell something intuitive about "What does $\Sigma^{-1}$ say about data?" Edit: Thanks for replies After taking some fantastic courses, I'd ...
2
votes
2answers
1k views

Linear regression and non-invertibility

In linear regression there are two approaches for minimizing the cost function: The first one is using gradient descent. The second one is setting the derivative of the cost function to zero and ...
1
vote
1answer
255 views

Inverse matrix for contrast coding

I'm trying to understand how "user defined contrast coding" works. My question refers to the example from http://www.ats.ucla.edu/stat/r/library/contrast_coding.htm#User: ...
6
votes
2answers
419 views

Variance-covariance matrix of the parameter estimates wrongly calculated?

I fitted an hyperbolic distribution to my data with the hyperbFit(mydata,hessian=TRUE) command (package HyperbolicDist). The hessian looks like: ...
4
votes
2answers
5k views

Numerical Instability of calculating inverse covariance matrix

I have a 65 samples of 21-dimensional data (pasted here) and I am constructing the covariance matrix from it. When computed in C++ I get the covariance matrix pasted here. And when computed in matlab ...
-1
votes
1answer
333 views

Singular matrix: eigenvalues perturbation vs Moore-Penrose generalized inverse

We often face singular matrices in practice: OLS with singular (X'X), GMM with singular weighting matrix, singular matrix in Wald statistics. I'm wondering how can we overcome this issue. I've seen ...
3
votes
2answers
1k views

Ways to measure distance from multivariate Gaussian (Mahalanobis distance)

I have a cluster of p-dimensional points and given a new p-dimensional point $x$ I want to determine whether or not it is likely to belong to this cluster. The cluster is made up of $n$ ...
7
votes
1answer
6k views

What to do when sample covariance matrix is not invertible?

I am working on some clustering techniques, where for a given cluster of d-dimension vectors I assume a multivariate normal distribution and calculate the sample d-dimensional mean vector and the ...
3
votes
1answer
371 views

How to interpret the sum of the elements of an inverse covariance matrix?

In the derivation of global minimum variance portfolio, we get The $(Σ^{-1}1) /(1'Σ^{-1}1)$. What's the meaning of $1'Σ^{-1}1$ and $Σ^{-1}1$. $Σ$ is a covariance matrix of assets returns.
3
votes
1answer
145 views

Question about inverse in a two-step estimator as a joint GMM-estimators approach

I'm reading Newey & McFadden - Large sample estimation and hypothesis testing (in the Handbook of Econometrics, Volume 4, 1994, page 2178). My model which I'm interested in has some former ...
2
votes
2answers
907 views

How to get conditional variance from Schur complement?

Suppose you have vectors X and Y with covariance matrix $V = \left( \begin{array}{cc} A & B \\ B^T & C \end{array} \right)$. This Wikipedia article says that $Var(X | Y) = A - BC^{-1}B^T$, ...