The tag has no wiki summary.

learn more… | top users | synonyms

2
votes
0answers
31 views

How to proof relationship between inverse covariance matrix and linear regression coefficients?

Edited: I would like to work out the above relationship, more precisely: Let $(Y_{1}, ..., Y_{m})$ be a zero-mean vector with covariance matrix $\Sigma$, and let $S \subset \{1, ..., m\}.$ The ...
0
votes
0answers
30 views

A non-negative definate matrix has a non-negative generalized inverse

I'm having trouble proving a N.N.D matrix has a N.N.D G-Inverse. So far I have: If we assume x = Az where x >= 0 and A is a nnd matrix. So if Y is a G-inverse than: x = Az = YAz = Yx >= 0 . Thus ...
0
votes
1answer
72 views

Orthogonalizing predictors for least squares estimation

I know that orthogonalization in LS is to avoid inverting X'X. The idea behind it is to find variables Z that are orthogonal to each other. Although the process to find those is clear to me, I don't ...
1
vote
1answer
42 views

Tolerance for pseudoinverse

In calculating the pseudoinverse of a matrix $A$, of size (m,n), I need to choose a tolerance threshold for the eigenvalues. I'm trying to understand how I should pick this. Matlab default is to use ...
3
votes
2answers
110 views

Inverting non positive definite covariance matrix

I have an expression for a covariance matrix $C$ in terms of the indices $i$ and $j$. In this way I can analytically calculate the elements of my covariance matrix, however when I try to invert $C$ ...
0
votes
2answers
45 views

Returning the inverse of a matrix in a quadratic program (SVM) in cvx optimization package

I am solving the dual QP of an SVM, and using the RBF kernel. As you know, the objective function is of the form $$f(\alpha) = \alpha^T Q \alpha $$ where $\alpha$ is the optimization variable and $Q$ ...
3
votes
1answer
93 views

Uniqueness of $x'\beta$ even when $\mathbb{E}(x^Tx)$ is not invertible

As discussed in user25658's answer to this question, when one wants to compute $$ \beta = \mathbb{E}(x^Tx)^{-1} \mathbb{E}(x^TY) $$ but $\mathbb{E}(x^Tx)$ is not invertible, $\beta$ is not uniquely ...
2
votes
0answers
72 views

Interpretation of regression coefficients obtained from applying left inverse of regressor matrix in an underdetermined system?

If $X^\dagger$ is the pseudo-inverse of $X$, $\beta = X^\dagger y$ is the least squares solution for $\beta$ when $y=X\beta$. In the overdetermined case, applying $X^{\dagger,L} = (X^TX)^{-1}X^T$ ...
1
vote
2answers
94 views

IV estimator: efficient implementation?

I would like to implement (in R) an instrumental variable (IV) estimator, that takes the most general form (here not 2SLS or GMM!): $$ \beta_{IV} = (Z'X)^{-1}Z'Y $$ I could code this in the naive ...
5
votes
2answers
2k views

What does the inverse of covariance matrix says about data? (Intuitively)

I'm curious about the nature of $\Sigma^{-1}$. Can anybody tell something intuitive about "What does $\Sigma^{-1}$ say about data?" Edit: Thanks for replies After taking some fantastic courses, I'd ...
1
vote
2answers
421 views

Linear regression and non-invertibility

In linear regression there are two approaches for minimizing the cost function: The first one is using gradient descent. The second one is setting the derivative of the cost function to zero and ...
2
votes
1answer
148 views

Inverse matrix for contrast coding

I'm trying to understand how "user defined contrast coding" works. My question refers to the example from http://www.ats.ucla.edu/stat/r/library/contrast_coding.htm#User: ...
6
votes
2answers
192 views

Variance-covariance matrix of the parameter estimates wrongly calculated?

I fitted an hyperbolic distribution to my data with the hyperbFit(mydata,hessian=TRUE) command (package HyperbolicDist). The hessian looks like: ...
4
votes
2answers
2k views

Numerical Instability of calculating inverse covariance matrix

I have a 65 samples of 21-dimensional data (pasted here) and I am constructing the covariance matrix from it. When computed in C++ I get the covariance matrix pasted here. And when computed in matlab ...
-1
votes
1answer
245 views

Singular matrix: eigenvalues perturbation vs Moore-Penrose generalized inverse

We often face singular matrices in practice: OLS with singular (X'X), GMM with singular weighting matrix, singular matrix in Wald statistics. I'm wondering how can we overcome this issue. I've seen ...
3
votes
2answers
759 views

Ways to measure distance from multivariate Gaussian (Mahalanobis distance)

I have a cluster of p-dimensional points and given a new p-dimensional point $x$ I want to determine whether or not it is likely to belong to this cluster. The cluster is made up of $n$ ...
5
votes
1answer
2k views

What to do when sample covariance matrix is not invertible?

I am working on some clustering techniques, where for a given cluster of d-dimension vectors I assume a multivariate normal distribution and calculate the sample d-dimensional mean vector and the ...
3
votes
1answer
238 views

How to interpret the sum of the elements of an inverse covariance matrix?

In the derivation of global minimum variance portfolio, we get The $(Σ^{-1}1) /(1'Σ^{-1}1)$. What's the meaning of $1'Σ^{-1}1$ and $Σ^{-1}1$. $Σ$ is a covariance matrix of assets returns.
3
votes
1answer
108 views

Question about inverse in a two-step estimator as a joint GMM-estimators approach

I'm reading Newey & McFadden - Large sample estimation and hypothesis testing (in the Handbook of Econometrics, Volume 4, 1994, page 2178). My model which I'm interested in has some former ...
2
votes
2answers
485 views

How to get conditional variance from Schur complement?

Suppose you have vectors X and Y with covariance matrix $V = \left( \begin{array}{cc} A & B \\ B^T & C \end{array} \right)$. This Wikipedia article says that $Var(X | Y) = A - BC^{-1}B^T$, ...
2
votes
1answer
168 views

Correlation and Hotelling test

To find a Hotelling $T^2$ score it is necessary to calculate the covariance matrix and then invert it. Now, when the test is a two-sample $T^2$ test, the covariance matrix is a pooled matrix. When ...
0
votes
1answer
201 views

Pseudo Inverse Instead of Inverse with LDA?

I have implemented the LDA algorithm. However when I had to get the inverse of a matrix Matlab threw an error and I had to use pinv (pseudo inverse) instead of inverse. Did I do something wrong or is ...
2
votes
0answers
742 views

Geometric intuition for why an outer product of two vectors makes a correlation matrix? [closed]

I understand that the outer product of two vectors, say representing two detrended time series, can represent a cross-correlation (well covariance) matrix. I also know that the inverse of a ...
1
vote
1answer
540 views

matlab gmdistribution.fit 'Regularize' - what regularization method?

I am wondering what is behind matlab 'Regularize' option for method gmdistribution.fit. If it is simply adding a 'little' value to diagonal elements of covariance matrix, so as to make covariance ...
1
vote
0answers
115 views

Estimation accuracy of precision matrix

I have a couple of questions related to estimation of high-dimensional precision matrix (inverse of the covariance matrix) in the case where p is close to 100 and n < p. As a measure of estimation ...
3
votes
1answer
258 views

Moore-Penrose generalized determinant

Is there a function in R to calculate the generalized determinant of a singular matrix? (similar to the ginv() used to compute the generalized inverse)
6
votes
3answers
712 views

To use Discrete Fourier Transform to invert a covariance matrix

I am working on a problem that its difficult part is to invert a covariance matrix (in R). I could not use usual approches like SVD and Chol. Then, I decided to use a Discrete Fourier Transform (DFT) ...
1
vote
1answer
2k views

Problem with singular covariance matrices when doing Gaussian process regression

I'm working with Gaussian process regression. Currently I start testing different covariance functions and compositions to see what type of data they could describe best. I made an own implementation ...
1
vote
2answers
944 views

Gaussian Process covariance matrix gets zero determinant

I have a Gaussian process regression implementation and developed some example data to test the capabilities of those methods. In the posterior calculation one gets the covariance matrix $K$. For some ...
4
votes
4answers
3k views

Testing for linear dependence among the columns of a matrix

I have a correlation matrix of security returns whose determinant is zero. (This is a bit surprising since the sample correlation matrix and the corresponding covariance matrix should theoretically be ...
9
votes
1answer
6k views

Efficient calculation of matrix inverse in R

I need to calculate matrix inverse and have been using solve function. While it works well on small matrices, solve tends to be ...
9
votes
1answer
269 views

Fast computation/estimation of a low-rank linear system

Linear systems of equations are pervasive in computational statistics. One special system I have encountered (e.g., in factor analysis) is the system $$Ax=b$$ where $$A=D+ B \Omega B^T$$ Here $D$ is ...
1
vote
1answer
345 views

Solving a regression problem

I wanted to solve such a regression problem: $$Y = Xb + e$$ where $X$ is a $m$ by $n$ matrix, resulting in: b = (X'X)-1X'Y as a solution. Since $n$ is quite large (2400), I can't use the ...
6
votes
3answers
300 views

Computing $(X^TX)^{-1}X^Ty$ in OLS

Let $A\in\mathbb{R}^{n \times n}$ be a dense symmetric positive-definite matrix (the $X^TX$ from here) and $b$ a vector in $\mathbb{R}^n$. I need to compute $A^{-1}b$. Two questions: Could you ...