The tag has no wiki summary.

learn more… | top users | synonyms

4
votes
1answer
23 views

How to find factor that is making matrix singular

I have a 300+ column data.frame, and no matter how I break it up I get this error every time: ...
1
vote
2answers
21 views

reasons for a non-invertible matrix

I am trying to factor analyze a matrix in R and I keep getting errors that lead me to believe my matrix is non-invertible. What are the reasons a matrix could be non-invertible? The only one I found ...
4
votes
1answer
42 views

Cholesky factorization and forward substitution less accurate than inversion?

I recently asked this question asking for an efficient way to compute the Mahalanobis distance (without calculating the inverse). The accepted solution was to use the Cholesky factorization and ...
0
votes
1answer
18 views

Marginal Likelihood of a Gaussian Process Model, Duplicate entries in kernel matrix

I am trying to fit a Gaussian process model using the toolbox and I got stuck in the following problem. Assuming that I have some duplicated data points in my training data, then those will map to ...
0
votes
1answer
26 views

I need a matrix in order to calculate g-inverse of it [closed]

I want to calculate g-inverse of a matrix, which has 4 rows and not a square matrix and has no inverse. Please help me find such a sound (good) matrix. I only need a matrix. You can suggest a book or ...
0
votes
1answer
62 views

Understanding the marginal distribution of multivariate normal distribution

I am trying to better understand the multivariate normal distribution. Here I try to refer to the conditional distribution part of wiki also the fifth page of this tutorial. I do not quite ...
0
votes
0answers
18 views

Evaluate the multivariate normal using variance matrices $\boldsymbol{\Lambda}+\alpha_{i}\mathbf{a}\mathbf{a}^{T}+\beta_{j}\mathbf{b}\mathbf{b}^{T}$

I need to calculate a huge amount of inverses and determinants to evaluate the pdf of the multivariate Gaussian. Specifically I need to compute the inverses and determinants of the following ...
0
votes
0answers
31 views

Propogation of error in a matrix inversion

I'm trying to find the deterministic error bounds for some parameters calculated through distance geometry. The equation can be simplified to the following form: $ \left[\begin{matrix} x_1 \\ x_2 \\ ...
3
votes
1answer
61 views

How to calculate the inverse of sum of a Kronecker product and a diagonal matrix

I want to calculate the inverse of a matrix of the form $S = (A\otimes B+C)$, where $A$ and $B$ are symetric and invertible, $C$ is a diagonal matrix with positive elements. Basically if the ...
8
votes
0answers
89 views

Why does inversion of a covariance matrix yield partial correlations between random variables?

I heard that partial correlations between random variables can be found by inverting the covariance matrix and taking appropriate cells from such resulting precision matrix (this fact is mentioned in ...
0
votes
0answers
41 views

How to find the expectation of determinant of a matrix and inverse of a matrix

I have been stuck for a long with a problem on expectation. All help in this regard will be highly solicited. I want to know What is the mathematical expression for $\mathbb{E}(\mathbf{X}^{-1})$, ...
3
votes
1answer
63 views

Invertibility of $X^TX$ with severe multicollinearity in regression

I am studying about multicollinearity in regression and in the book it says, "if there is severe (but not perfect) multicollinearity, two or more predictor variables are highly correlated, so $X^TX$ ...
0
votes
1answer
61 views

Matrix Inversion Error

I a Multiple linear regression model, from published literature, I am implementing a spreadsheet to generate new predictions based on the published model. the literature stated Coefficients and the ...
1
vote
0answers
104 views

R - system is computationally singular - dealing with small numbers

I'm working with a ~200x200 Markovian transition matrix of non-zero probabilities. Forcibly, these probabilities are, for the large part, going to be very small. I am trying to find the inverse of my ...
0
votes
0answers
65 views

Condition Number in Single Variable Regression

I've calculated a single variable linear regression using OLS (in the python statsmodels library). The model has a large condition number, and suggests there may ...
0
votes
1answer
16 views

Is it posible to find a derivative of the mean function of Gaussian process regression?

The mean function $\hat{\mu}(x_*)$ of GPR is $k(x_*, X)(k(X, X) + \sigma^2_w I)^{-1}Y$ where $k(\cdot, \cdot)$ is a kernel matrix or vector of appropriate size and is parametrized by some ...
4
votes
1answer
152 views

Diagonal elements of the inverted correlation matrix

Is it true that the diagonal elements of the inverted correlation matrix will always be larger than 1? Why?
1
vote
0answers
111 views

The inverse of AR correlation matrix

I want to find the inverse of the following matrix: $$ R_{k-1}=\begin{pmatrix} 1 &\rho &\rho^2 &\cdots &\rho^{k-2} \\ \rho &1 &\rho &\cdots ...
0
votes
0answers
17 views

Convergence of sample concentration matrix

I'm interested in the Frobenius or $\infty$ norm convergence rate bound for the sample inverse convariance (concentration) matrix. That is, suppose: $$ Y \sim \mathcal{N}_p\left(0, \Sigma\right) $$ ...
2
votes
1answer
164 views

How to proof relationship between inverse covariance matrix and linear regression coefficients?

Edited: I would like to work out the above relationship, more precisely: Let $(Y_{1}, ..., Y_{m})$ be a zero-mean vector with covariance matrix $\Sigma$, and let $S \subset \{1, ..., m\}.$ The ...
0
votes
0answers
52 views

A non-negative definate matrix has a non-negative generalized inverse

I'm having trouble proving a N.N.D matrix has a N.N.D G-Inverse. So far I have: If we assume x = Az where x >= 0 and A is a nnd matrix. So if Y is a G-inverse than: x = Az = YAz = Yx >= 0 . Thus ...
0
votes
1answer
107 views

Orthogonalizing predictors for least squares estimation

I know that orthogonalization in LS is to avoid inverting X'X. The idea behind it is to find variables Z that are orthogonal to each other. Although the process to find those is clear to me, I don't ...
1
vote
1answer
101 views

Tolerance for pseudoinverse

In calculating the pseudoinverse of a matrix $A$, of size (m,n), I need to choose a tolerance threshold for the eigenvalues. I'm trying to understand how I should pick this. Matlab default is to use ...
3
votes
2answers
238 views

Inverting non positive definite covariance matrix

I have an expression for a covariance matrix $C$ in terms of the indices $i$ and $j$. In this way I can analytically calculate the elements of my covariance matrix, however when I try to invert $C$ ...
0
votes
2answers
102 views

Returning the inverse of a matrix in a quadratic program (SVM) in cvx optimization package

I am solving the dual QP of an SVM, and using the RBF kernel. As you know, the objective function is of the form $$f(\alpha) = \alpha^T Q \alpha $$ where $\alpha$ is the optimization variable and $Q$ ...
3
votes
1answer
100 views

Uniqueness of $x'\beta$ even when $\mathbb{E}(x^Tx)$ is not invertible

As discussed in user25658's answer to this question, when one wants to compute $$ \beta = \mathbb{E}(x^Tx)^{-1} \mathbb{E}(x^TY) $$ but $\mathbb{E}(x^Tx)$ is not invertible, $\beta$ is not uniquely ...
2
votes
0answers
94 views

Interpretation of regression coefficients obtained from applying left inverse of regressor matrix in an underdetermined system?

If $X^\dagger$ is the pseudo-inverse of $X$, $\beta = X^\dagger y$ is the least squares solution for $\beta$ when $y=X\beta$. In the overdetermined case, applying $X^{\dagger,L} = (X^TX)^{-1}X^T$ ...
1
vote
2answers
108 views

IV estimator: efficient implementation?

I would like to implement (in R) an instrumental variable (IV) estimator, that takes the most general form (here not 2SLS or GMM!): $$ \beta_{IV} = (Z'X)^{-1}Z'Y $$ I could code this in the naive ...
13
votes
2answers
5k views

What does the inverse of covariance matrix say about data? (Intuitively)

I'm curious about the nature of $\Sigma^{-1}$. Can anybody tell something intuitive about "What does $\Sigma^{-1}$ say about data?" Edit: Thanks for replies After taking some fantastic courses, I'd ...
1
vote
2answers
803 views

Linear regression and non-invertibility

In linear regression there are two approaches for minimizing the cost function: The first one is using gradient descent. The second one is setting the derivative of the cost function to zero and ...
2
votes
1answer
200 views

Inverse matrix for contrast coding

I'm trying to understand how "user defined contrast coding" works. My question refers to the example from http://www.ats.ucla.edu/stat/r/library/contrast_coding.htm#User: ...
6
votes
2answers
278 views

Variance-covariance matrix of the parameter estimates wrongly calculated?

I fitted an hyperbolic distribution to my data with the hyperbFit(mydata,hessian=TRUE) command (package HyperbolicDist). The hessian looks like: ...
4
votes
2answers
3k views

Numerical Instability of calculating inverse covariance matrix

I have a 65 samples of 21-dimensional data (pasted here) and I am constructing the covariance matrix from it. When computed in C++ I get the covariance matrix pasted here. And when computed in matlab ...
-1
votes
1answer
294 views

Singular matrix: eigenvalues perturbation vs Moore-Penrose generalized inverse

We often face singular matrices in practice: OLS with singular (X'X), GMM with singular weighting matrix, singular matrix in Wald statistics. I'm wondering how can we overcome this issue. I've seen ...
3
votes
2answers
1k views

Ways to measure distance from multivariate Gaussian (Mahalanobis distance)

I have a cluster of p-dimensional points and given a new p-dimensional point $x$ I want to determine whether or not it is likely to belong to this cluster. The cluster is made up of $n$ ...
5
votes
1answer
4k views

What to do when sample covariance matrix is not invertible?

I am working on some clustering techniques, where for a given cluster of d-dimension vectors I assume a multivariate normal distribution and calculate the sample d-dimensional mean vector and the ...
3
votes
1answer
307 views

How to interpret the sum of the elements of an inverse covariance matrix?

In the derivation of global minimum variance portfolio, we get The $(Σ^{-1}1) /(1'Σ^{-1}1)$. What's the meaning of $1'Σ^{-1}1$ and $Σ^{-1}1$. $Σ$ is a covariance matrix of assets returns.
3
votes
1answer
128 views

Question about inverse in a two-step estimator as a joint GMM-estimators approach

I'm reading Newey & McFadden - Large sample estimation and hypothesis testing (in the Handbook of Econometrics, Volume 4, 1994, page 2178). My model which I'm interested in has some former ...
2
votes
2answers
660 views

How to get conditional variance from Schur complement?

Suppose you have vectors X and Y with covariance matrix $V = \left( \begin{array}{cc} A & B \\ B^T & C \end{array} \right)$. This Wikipedia article says that $Var(X | Y) = A - BC^{-1}B^T$, ...
2
votes
1answer
217 views

Correlation and Hotelling test

To find a Hotelling $T^2$ score it is necessary to calculate the covariance matrix and then invert it. Now, when the test is a two-sample $T^2$ test, the covariance matrix is a pooled matrix. When ...
0
votes
1answer
279 views

Pseudo Inverse Instead of Inverse with LDA?

I have implemented the LDA algorithm. However when I had to get the inverse of a matrix Matlab threw an error and I had to use pinv (pseudo inverse) instead of inverse. Did I do something wrong or is ...
2
votes
0answers
931 views

Geometric intuition for why an outer product of two vectors makes a correlation matrix? [closed]

I understand that the outer product of two vectors, say representing two detrended time series, can represent a cross-correlation (well covariance) matrix. I also know that the inverse of a ...
1
vote
1answer
736 views

matlab gmdistribution.fit 'Regularize' - what regularization method?

I am wondering what is behind matlab 'Regularize' option for method gmdistribution.fit. If it is simply adding a 'little' value to diagonal elements of covariance matrix, so as to make covariance ...
1
vote
0answers
126 views

Estimation accuracy of precision matrix

I have a couple of questions related to estimation of high-dimensional precision matrix (inverse of the covariance matrix) in the case where p is close to 100 and n < p. As a measure of estimation ...
3
votes
1answer
299 views

Moore-Penrose generalized determinant

Is there a function in R to calculate the generalized determinant of a singular matrix? (similar to the ginv() used to compute the generalized inverse)
7
votes
3answers
977 views

To use Discrete Fourier Transform to invert a covariance matrix

I am working on a problem that its difficult part is to invert a covariance matrix (in R). I could not use usual approches like SVD and Chol. Then, I decided to use a Discrete Fourier Transform (DFT) ...
1
vote
1answer
2k views

Problem with singular covariance matrices when doing Gaussian process regression

I'm working with Gaussian process regression. Currently I start testing different covariance functions and compositions to see what type of data they could describe best. I made an own implementation ...
1
vote
2answers
1k views

Gaussian Process covariance matrix gets zero determinant

I have a Gaussian process regression implementation and developed some example data to test the capabilities of those methods. In the posterior calculation one gets the covariance matrix $K$. For some ...
9
votes
1answer
9k views

Efficient calculation of matrix inverse in R

I need to calculate matrix inverse and have been using solve function. While it works well on small matrices, solve tends to be ...
9
votes
1answer
314 views

Fast computation/estimation of a low-rank linear system

Linear systems of equations are pervasive in computational statistics. One special system I have encountered (e.g., in factor analysis) is the system $$Ax=b$$ where $$A=D+ B \Omega B^T$$ Here $D$ is ...