Questions tagged [matrix-inverse]
The inverse of a given square matrix, $A$, is the matrix $A^{-1}$ such that $AA^{-1}$ is the identity matrix.
153
questions
2
votes
1
answer
73
views
Invertibility of the Gram matrix of a convex combination
Let's assume two real valued matrices $A,B\in R^{w\times d}$ for which $d>w$ and they both have full (column) rank.
I am interested in the invertibility of the Gram matrix $$H(t):=(A+t(B-A))(A+t(B-...
- 21
0
votes
0
answers
7
views
inverse of canonical correlation
The canonical correlation obtains the most correlative pairs of components across two matrices. However, the inverse is not commonly known but could be useful for obtaining the most different ...
- 73
1
vote
2
answers
88
views
How to obtain least squares when $X^TX$ cannot be inverted
This work is all theoretical and for school, so we were only provided this information to work with, no actual y values. I have a simple linear model I have been asked to translate into a matrix, ...
- 65
0
votes
0
answers
18
views
how can an inverse correlation matrix have <1 in diagonals. the correlations x1, x2 are about 0.5 [duplicate]
The inverse correlation matrix was calculated with the minverse function. The results show that in the diagonal the values are less than 1. how is this possible? the x1,x2 independent variables are ...
- 1
2
votes
1
answer
36
views
Inverse of the outer product of some vectors with their transpose
Assume i have $n$ 3D unit vectors $v_s$, with different values.
Then i define a matrix $T$ as:
$$
T = \frac{1}{n} \sum_{s=1}^{n} v_s \times v_s'
$$
where $v_s$ are $3\times 1$ vectors and therefore $...
- 43
0
votes
0
answers
15
views
Positive definiteness of integral of matrix
I was reading a paper, and did not understand a statement that the author made without further explanation. The author derives the limiting distribution of a non-linear least-squares estimator and ...
- 1
1
vote
1
answer
56
views
Condition for covariance matrix to be non-invertible
Context: I'm working on a machine learning problem where I'm using multivariate normal likelihood which requires calculating determinant and inverting the covariance matrix. I'm trying to generate ...
- 749
1
vote
1
answer
124
views
How to derive an inverse of Gaussian Kernel
As an example, say I have a function (Gaussian process kernel):
$$K(x_i, x_j) \equiv \exp(-\alpha |x_i-x_j|^2)+\beta \delta_{ij}$$
Is there a way to analytically express $K^{-1}(x_i,x_j)$, s.t. ...
- 261
2
votes
1
answer
56
views
How to prove $(P^{-1} + B^T R^{-1} B)^{-1} B^T R^{-1} = PB^T(BPB^T + R)^{-1}$
It is Equation C.5 from https://www.seas.upenn.edu/~cis520/papers/bishop_appendix_C.pdf
I tried right multiply both sides with $(BPB^T + R)$, but not sure how to continue from there.
- 877
1
vote
1
answer
140
views
How can nuisance parameters in Fisher matrix can deteriorate the useful constraints?
I have a Fisher matrix $F$ which has the matrix blocks form like this :
$$
F=\begin{bmatrix}
A & B\\
C & D
\end{bmatrix}
$$
The block $A$ is the most important block, in the sense the ...
- 29
0
votes
1
answer
29
views
Problem dealing with OLS estimator
I'm an econometrics student and I'm having a little trouble with lineal algebra.
I have seen that the OLS estimator, given the following regression in matrix form:
$$
y=X \beta+u
$$
Is:
$$
\hat{\beta}=...
- 3
1
vote
0
answers
12
views
How to simplify fallowing martrices' Expected value elements in the equation
I have a matrix equation with four variables inside, $x^1_{00}$, $x^1_{tt}$ and $x^2_{tt}$, $x^2_{tt}$. $x^1_{00}$, $x^1_{tt}$ come from the same distribution , they are only shifted by timelag $t$. ...
- 527
0
votes
0
answers
50
views
Distribution of solution to linear system
I have a random symmetric matrix $ A \in \mathbb{R}^{M \times M}$, and random vector $b \in \mathbb{R}^M$. I also have access to expressions for the mean and variance of each element of $A$ and $b$ (...
0
votes
0
answers
26
views
Distribution of matrix products
Let's say I have two random vectors $A, B \in \mathbb{R}^N$ that are distributed approximately normally, with distribution $\mathcal{N}(\mu, \Sigma)$. I then define $\bar{A}$ as being a diagonal ...
0
votes
0
answers
19
views
Correlation matrix with 0 determinant [duplicate]
I am looking at crypto coin price data. I compute the correlation matrix but I am unable to invert due to a zero determinant. I'm not quite sure why this is happening as none of the columns are that ...
- 1,404
0
votes
0
answers
72
views
When is the Optimal weighting matrix in GMM singular?
currently I am trying to estimate a simple linear regression:
\begin{equation}
y_t = X \beta + \varepsilon_t,
\end{equation}
where I try to find 4 coefficients and one specific predictor is an ...
0
votes
0
answers
18
views
Singularity of a modelmatrix that consists of one predictor raised to multiple powers
I was playing around a little bit with linear regression models in R and wanted to try out whether I can get a perfect fit for a linear model where I have a response $$Y =
\left(
\begin{array}{c}
y_1\...
- 87
1
vote
1
answer
186
views
Mercer's theorem and eigenfunctions
Consider a kernel $K$ which satisfies the conditions of Mercer's theorem. We know that $K(x,y) = \sum\limits_{i=1}^{+\infty}\lambda_ie_i(x)e_i(y)$ where $\lambda_i$ and $e_i$ are the eigenvalues/...
- 161
0
votes
0
answers
64
views
Finding inverse matrix $(X'X)^{-1}$ with $X$ as design matrix [duplicate]
I'm relatively new to all this and I am trying to figure out how I can derive the matrix $(X'X)^{-1}$ when I have given $x_1, x_2, x_3$ and $y$. $X$ is the design matrix in that case but not sure how ...
3
votes
1
answer
108
views
linear regression and obfuscation matrix - just not clicking for me?
Here is the basis of my question.
We have an exercise to take a data set of made up insurance claims and train a linear regression model to predict future claims; however, because it is sensitive data,...
- 33
3
votes
1
answer
107
views
completing a square
If I have a density function of the form $p(x) \propto \exp(−q(x)/2)$ where $q(x)$ has
the following quadratic function
$$q(x)=x^Tx+y^Ty-[x^TA+y^TB][A^TA+B^TB+\beta\mathbb{I}]^{-1}[A^Tx+B^Ty]$$
where $...
- 205
1
vote
1
answer
13
views
What is an example of a systems problem that can be solved by either 'matrix inversion' or an 'iterative procedure' to arrive at the same result?
I recently heard over a radio program (in French) that a given problem - can't recall it exactly, but it involved solving large systems - could be solved "through matrix inversion" or "...
- 1,048
17
votes
1
answer
3k
views
Relationship between Cholesky decomposition and matrix inversion?
I've been reviewing Gaussian Processes and, from what I can tell, there's some debate whether the "covariance matrix" (returned by the kernel), which needs to be inverted, should be done so ...
- 1,650
3
votes
1
answer
114
views
What dimensionality reduction methods allow a lower dimensional reconstruction of the original data besides PCA via invertible transformations?
In eigenfaces, one used the inverse transformation PCA is capable of doing to reconstruct the low dimensional face image.
In tsne one may not reconstruct the original dataset to produce something akin ...
user294054
-2
votes
1
answer
170
views
Why doesn't $(e^{A})^{-1} = e^{-A}$ hold for a symmetric matrix in Python?
$e^A$ is just the $A$ matrix with all of its elements exponentiated, called a matrix exponential.
It follows that the inverse $(e^{A})^{-1} = e^{-A}$ for square matrices, although I could find nothing ...
- 3,047
0
votes
1
answer
244
views
Proving that the hat matrix is unchanged even when the predictors are multiplied by constants
I know that the hat matrix $H = X(X^T X)^{-1} X^T$, and that $\hat{Y} = HY$. When we have some non-zero constants that we multiply each respective predictor by, which just multiplies every column in ...
- 3
2
votes
0
answers
32
views
Why is it not possible to simplify $b=(X'X)^{-1}X'y$ [duplicate]
Why is the following not possible:
$$
b=(X'X)^{-1}X'y = X^{-1}(X')^{-1}X'y=X^{-1}y
$$
While this term $(AB)^{-1}=B^{-1}A^{-1}$ applies to any two matrices as long as both are of full rank and are $nxn$...
- 31
3
votes
1
answer
663
views
Moore Penrose Pseudo-Inverse Fast Algorithm in R [closed]
I want to apply Moore Penrose Pseudo-Inverse on my matrix, which is a 20,000 * 20,000 symmetric matrix with rank 19,999. I found ginv() function from the ...
- 87
0
votes
1
answer
433
views
How to do multiple linear regressions with overlapping predictors efficiently
I'm using covariance matrices to do a linear regression. I have the same number of predictors and dependent variables (3000 of each). For each $i$, $X_{i}$ and $Y_{i}$ have a kind of symmetry.
I ...
0
votes
0
answers
73
views
NN type/architecture needed for inverse covariance matrix approximation
The idea is to construct a neural network (NN) that takes N series of financial returns as input and returns the (approximation of the) inverse of the sample covariance matrix (an N times N matrix). I ...
- 11
1
vote
0
answers
449
views
Normalization for solving linear equations
Suppose I want to solve a linear equation system in the form of
$$A x = b$$
to get $x$, where $A$ are $n$ by $n$ matrix and $b$ is $n$ by 1 vector. Is there any normalization procedure necessary ...
- 1,633
4
votes
1
answer
593
views
Updating the covariance matrix after deleting the i-th column and row
Suppose I have a covariance matrix $A_{n\times n}$ and $A^{-1}_{n\times n}$ is its inverse. Then I randomly exclude the $i$-th row and column, where $1\le i\le n \in \mathbb{N}$, obtaining the new ...
- 236
6
votes
2
answers
589
views
Least Squares removing first $k$ observations Woodbury formula?
Given the matrix $X_{n,p}$ from the least squares problem $$ \mathbf{X} \cdot \mathbf{\beta} = z $$
Where the normal equation is:
$$ \mathbf{\hat{\beta}} = \left(\mathbf{X}^T \mathbf{X}\right)^{-1} ...
- 195
0
votes
0
answers
302
views
Squared Multiple Correlation (SMC) of my correlation matrix tend towards 1. How to interpret this?
In order to be able to conduct exploratory factor analysis, I want to carry out parallel analysis to determine the number of factors to be extracted. To do so, I want to extract the eignevalues of the ...
- 469
1
vote
1
answer
342
views
Upper bound trace of inverse of covariance matrix
Let C be the covariance matrix from any normal distribution. If the trace of C is upper-bounded by a constant k (i.e., tr(C)<=k), can I find an upper bound for the trace of the inverse of C (i.e., ...
- 73
0
votes
0
answers
46
views
Fast inverse of positive definite matrix subtracted by its Nystrom approximation
Assume a positive definite symmetric covariance matrix $$C_{n,n}$$ and let its Nystrom approximation be $$\hat{C}_{n,n}=C_{n,q} C^{-1}_{q,q} C_{q,n}$$ for some $q<n$
Inverting $C_{n,n}$ is of $\...
- 2,074
5
votes
1
answer
1k
views
Generalized Least Squares using Moore Penrose pseudo inverse
I'm using GLS to fit a model where some independent variables are strongly correlated. Therefore my covariance matrix is singular. I have found that Moore-Penrose pseudo inverse can be used to find an ...
- 51
1
vote
1
answer
495
views
Bypassing inverse matrix calculation and the comparison of Gradient Descent and Newton Descent
I am currently reading Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.
In the 5th chapter the Gradient Descent algorithm is introduced with the following notations :
$...
- 111
2
votes
0
answers
276
views
Distribution of the Sum of an AR(1) Model Time Series
I have the following model for my model
$\Delta X_{t} = \mu \Delta t + \rho \Delta X_{t-1} + \sigma \sqrt{\Delta t} Z_t$
with the following initial conditions -
$\Delta X_{1} = \mu \Delta t + \...
- 153
3
votes
1
answer
484
views
numerically stable sparse Gaussian process regression (matrix inversion)
In sparse approximations of GP for large data set $(X,\mathbf{y})$ with $n$ samples, usually $m$ inducing points are chosen such that the true covariance matrix is approximated by $K_{nn}\to K_{nm}K_{...
6
votes
1
answer
475
views
Incremental solution for matrix inverse using Shermann-Morrison in $O(n^2)$
I have been reading a presentation on Value Function Approximation by David Silver (Introduction to Reinforcement Learning Course). On page 43 he finds a solution for linear least squares for an ...
- 1,349
6
votes
1
answer
2k
views
Which is more numerically stable for OLS: pinv vs QR
If I am doing standard OLS and want to calculate beta values (OLS estimators), which of the following is the more numerically stable method? And why?
Assuming that the columns of $X$ are already mean-...
- 576
3
votes
1
answer
324
views
Use $X^{-1}Y$ instead of $(𝑋^T𝑋)^{−1}𝑋^T Y$ to calculate $\beta$ when $X$ is already a square matrix in the least square problem
In the least squares problem $X\beta = Y$, the solution is $\hat{\beta} = (𝑋^T𝑋)^{−1}𝑋^TY$. I learned that two facts:
$𝑋^T𝑋$ is square matrix so that the definition of matrix inversion is ...
- 419
3
votes
1
answer
156
views
Proof of Pearson-Aitken selection formula
I am trying to understand the proof of the Pearson-Aitken selection formula, widely used in statistical genetics. A proof that the formula is general is given by Aitken (1936). However, I failed to ...
- 1,132
3
votes
1
answer
119
views
Why the representation in the form of $Z'X(X'X)^{-1}X'Z$ can not be simplified into $Z'Z$
Representation similar to $Z'X(X'X)^{-1}X'Z$ frequently appear to e.g. 2SLS.
I think that $Z'X(X'X)^{-1}X'Z = Z'XX^{-1}X'^{-1}X'Z = Z'(XX^{-1})(X'^{-1}X')Z = Z'Z$. So why it seems that in the context ...
- 539
1
vote
1
answer
35
views
matrix inequality related to finance
I'm trying to show that, for certain investment strategies, it pays to have more precise estimates of the covariance matrix of your returns. I have always took this for granted, but I've been having ...
- 18.6k
1
vote
1
answer
688
views
How are biases updated when 'batch size' > 1?
This is my network represented in matrices: (a dot represents an arbitrary number)
Feed-forwarding: (I omitted nesting it all in an activation function for the sake of brevity)
Backpropagation
The ...
3
votes
1
answer
4k
views
Matrices: system that is "computationally singular" versus "exactly singular" [closed]
I would like to know the mathematical concepts behind singular matrices. Matrices that do not have inverses in R throw one of two errors. I have provided some examples of both errors below:
Error in ...
- 31
3
votes
1
answer
666
views
Explanation of generalization of Newton's Method for multiple dimensions
I've been following the CS 229 lecture videos for machine learning, and in lecture 4 (~14:00), Ng explains Newton's Method for optimization to maximize an objective function ($f$), but doesn't clearly ...
- 131
1
vote
0
answers
25
views
Recursively expressing matrix inverse
Let $X$ be an $D \times N$ matrix. Let $I$ be a $D \times D$ identity matrix. Also let $y$ be a $N \times 1$ column vector. Suppose we are trying to solve $(X X ^T + k I) w = Xy$ for a $D$ dimensional ...
- 11