# Questions tagged [matrix-inverse]

The inverse of a given square matrix, $A$, is the matrix $A^{-1}$ such that $AA^{-1}$ is the identity matrix.

164 questions
Filter by
Sorted by
Tagged with
1 vote
28 views

### A formula involving generalized inverse of matrices [migrated]

Suppose (1) $A \,\ is \,\ n \times m$ therefore $A' \,\ is \,\ m \times n$ (2) $A^+$ is the pseudo-inverse of $A$ (3) $B$ is an $n \times n$ and invertible symmetric matrix. We could add ...
25 views

### Series expansion for a single coefficient of OLS multiple linear regression?

The Matrix/vector formulation of OLS is given by $$\vec \beta = \left( X^T X \right)^{-1} X^T \vec y$$ which is very nice as a matrix equation. But what is the equivalent scalar equation for a single ...
66 views

### Collapse of sampled Mahalanobis distance

Let $\{ x^{(1)}, \ldots, x^{(M)}\}$ be $M$ samples from a $n$-dimensional multivariate Gaussian distribution $\pi_{X} = \mathcal{N}(\mu, \Sigma)$. We recall the definition of the squared Mahalanobis ...
251 views

### Why does a valid Kernel only have to be positive semi-definite instead of positive definite?

I'm currently concerned with the topic of Gaussian Processes. To compute the covariance matrix of the conditional distribution, we have to invert $(K_{XX})^{-1}$, where $K_{XX}$ is a matrix of a ...
158 views

### Inverse of Matrix with normally distributed Elements?

Let's say we have some Matrix $X \in \mathbb{R}^{n_x \times n_x}$ whose elemets $x_{i,j} \sim \mathcal{N}(\mu, \sigma^2)$ are normally distributed. In other words: $vec(X) \sim \mathcal{N}(m, S)$, ...
88 views

### Running System GMM using pgmm on subset leads to system is singular error

I am running a System GMM regression using pgmm() in R. The parameters are set to ...
45 views

### The mean of Gaussian distribution subject to another Gaussian distribution, how to derive it？

I got a fomula $$N(y;cx,R)N(x;\bar{x},\Sigma)=N(y;c\bar{x},S)N(x;g,F)$$ where: \begin{align}S&=c\Sigma c^T+R\\g &=\bar{x}+\Sigma c^Ts^{-1}(y-c\bar{x})\\F &= \Sigma - \Sigma c^T s^{-1} c \...
342 views

### Parameter estimate in linear regression model with singular matrix $X^TX$

I started to study linear regression this autumn and got stuck at the estimation of regression parameters (in matrix form). In parameter estimation of the general linear model, the OLS method is used ...
565 views

### Why do large LMs use the transpose of the word embeddings matrix in the classification head?

All literature, guides and tutorials describing the construction of language models have used two separate matrices for the input and output projections: To project one-hot token IDs into hidden ...
1 vote
47 views

### Updating the Posterior Psi parameter for an Inverse Wishart Distribution

I am fitting a Mixed Multivariate Normal Distribution where the mixing occurs over the mean $\mu_j$ and the covariance matrix $H$ with mixing parameter $B_j$. The number of mixing elements is denotes ...
1 vote
110 views

### Why the columns of design matrix for linear models with additive Gaussian noise linear independent?

This question comes from page 142 of the book "Pattern Recognition and Machine Learning" by Christopher M. Bishop. Since it takes pages to arrive at the result, I excerpt the major settings ...
62 views

### compute Dirichlet distribution parameter from known mean distribution

For a particular Bayesian study I am going to apply Dirichlet distribution as my proposal random number generator. I am going to update the distribution parameter every trial based on a given ...
196 views

### Natural gradients with Moore–Penrose inverse of the Fisher information matrix

I'd like to show you my rough sketch for scaling up natural gradients to deep neural networks that appears to be easy to automate just like automatic differentiation. I think there must be a flaw ...
113 views

### What do you do in regression if $(X^\text{T} X)^{-1}$ does not exist?

Assume we want to solve $X z=b$ for $z$ where $X$ is a non-square matrix and $z$ and $b$ are column vectors. In case the system is overdetermined, we have no solution and can look for a solution ...
126 views

1 vote
13 views

### How to simplify fallowing martrices' Expected value elements in the equation

I have a matrix equation with four variables inside, $x^1_{00}$, $x^1_{tt}$ and $x^2_{tt}$, $x^2_{tt}$. $x^1_{00}$, $x^1_{tt}$ come from the same distribution , they are only shifted by timelag $t$. ...
117 views

### Distribution of solution to linear system

I have a random symmetric matrix $A \in \mathbb{R}^{M \times M}$, and random vector $b \in \mathbb{R}^M$. I also have access to expressions for the mean and variance of each element of $A$ and $b$ (...
250 views

### When is the Optimal weighting matrix in GMM singular?

currently I am trying to estimate a simple linear regression: \begin{equation} y_t = X \beta + \varepsilon_t, \end{equation} where I try to find 4 coefficients and one specific predictor is an ...
22 views

467 views

### Squared Multiple Correlation (SMC) of my correlation matrix tend towards 1. How to interpret this?

In order to be able to conduct exploratory factor analysis, I want to carry out parallel analysis to determine the number of factors to be extracted. To do so, I want to extract the eignevalues of the ...
1 vote
446 views

### Upper bound trace of inverse of covariance matrix

Let C be the covariance matrix from any normal distribution. If the trace of C is upper-bounded by a constant k (i.e., tr(C)<=k), can I find an upper bound for the trace of the inverse of C (i.e., ...