Questions tagged [singular-matrix]
A matrix is singular when its determinant is 0; for such matrices, the inverse is not defined. Also, related topics like singular fits
92
questions
99
votes
1
answer
121k
views
What correlation makes a matrix singular and what are implications of singularity or near-singularity?
I am doing some calculations on different matrices (mainly in logistic regression) and I commonly get the error "Matrix is singular", where I have to go back and remove the correlated variables. My ...
50
votes
2
answers
123k
views
Dealing with singular fit in mixed models
Let's say we have a model
...
11
votes
1
answer
9k
views
Understanding degenerate multivariate normal distribution
MVN is degenerate when the covariance matrix $\Sigma$ is singular.
I am trying to understand mainly conceptual (but also theoretical) implications of this. The Wikipedia article is quite terse. It ...
9
votes
3
answers
1k
views
Is the sum of two singular covariance matrices also singular?
I have two sample covariance matrices, computed from $n$ samples, less than $p$ variables: they are singular then.
I know that the sum of two covariance matrices is also a covariance matrix.
My ...
9
votes
2
answers
4k
views
What is the relation between singular correlation matrix and PCA?
Can anyone kindly give me some information about the statement (last sentence) at the end of below definition. What does it mean by "It can be used when a correlation matrix is singular"?
This quote ...
8
votes
1
answer
6k
views
Uniqueness for OLS linear regression
I'm implementing OLS linear regression without using the built-in functions in Matlab with normal equation:
I know this is probably very basic, but I want to double check, the input X yields a unique ...
8
votes
1
answer
485
views
Bayesian regression with singular $(X'X)$ - Is the posterior well-defined?
SE community, I hope to get some insights into the following problem.
Given a simple linear regression model $$Y=X\beta+\epsilon\text{ , where } Y\in\mathbb{R}^T,X\in\mathbb{R}^{T \times N}.$$
Under a ...
8
votes
0
answers
11k
views
Singular fit with simplest random structure in lmer (lme4), is a Bayesian approach the only option?
I'm running a mixed model with the lmer function from the lme4 package in R and ran into some issues with singular fits. I get the warning message 'singular fit', ...
6
votes
2
answers
2k
views
Sampling from matrix-variate normal distribution with singular covariances? [duplicate]
The matrix-variate normal distribution can be sampled indirectly by utilizing the Cholesky decomposition of two positive definite covariance matrices. However, if one or both of the covariance ...
5
votes
2
answers
23k
views
How do I avoid computationally singular matrices in R?
I'm fitting a logistic regression model (with R's caret package) to data here. I aim to predict whether Hillary or Trump will ...
5
votes
1
answer
273
views
Limit of Bernoulli R.V.s is a singular distribution
Working through an exercise in Probability (the question can be found in Lamperti).
Let $X_1,\dots$ be independent Bernoulli random variables with $\mathbb{P}(X_i=1) = p$ and $\mathbb{P}(X_i=0)=1-p$. ...
5
votes
1
answer
4k
views
Singular Fisher Information matrix, why is it a problem
I need to perform some research on the consequence of a singular Fisher Information matrix in statistical inference.
I am confused about what kind of problems a singular Fisher Information matrix ...
5
votes
2
answers
3k
views
R gls() vs. SAS proc mixed with interaction: Why does R complain about a singular matrix when SAS does not?
I like to keep analyses all in SAS or all in R when I can help it and lately have been using R more and more, but there's one analysis that I do somewhat routinely that has given me trouble in R.
I ...
4
votes
2
answers
2k
views
How to solve the error of singular fit in glmm in R
I am trying to fit a GLMM for binary data of whether colonies of bees perform mass flight or not. I have time when the mass flight was performed, temperature, location of the hive and species of the ...
4
votes
1
answer
5k
views
Singular Matrix and Linear Dependency
Singular matrix is defined as a square matrix with determinant of zero. I am aware that linear dependency among columns or rows leads to determinant being equal to zero (e.g. one column is a linear ...
4
votes
1
answer
221
views
Linear Mixed Model equation (as of lme4 package)
I am trying to derive the equations of a linear mixed model as specified in the documentation of the lme4 package:
"Fitting Linear Mixed-Effects Models using lme4" jstatsoft.org/article/view/v067i01
...
4
votes
1
answer
4k
views
Why is within class scatter matrix in LDA singular? [closed]
I read that when number of data points are much less than the dimension of data, the within class scatter matrix in singular? Can someone explain why this is the case? For example, while using LDA for ...
4
votes
1
answer
2k
views
Does it make sense to use PCA when the determinant of the correlation matrix is (almost) zero?
I'm running a PCA over a data set of $N \times p$ size ($N\approx 1000$ being the number of measurements and $p\approx 200$ being the number of dimensions/predictors).
I expect many of the predictors ...
4
votes
1
answer
227
views
Warnings for confint() in lme4
I am fitting the following model (random intercepts and slopes) on my data:
lmer(MuscleActivity ~ Period+ (1 + Period|ppnr), data = df)
My goal is to test whether ...
4
votes
1
answer
805
views
Are Kalman Filter recursions valid when the state noise has a singular covariance matrix?
Consider a Linear Gaussian State-Space Model where the states are denoted by $X_t$ and observations are denoted by $Y_t$:
\begin{align}
X_t &= A X_{t-1} + \epsilon_t, &&\epsilon_t \sim \...
4
votes
0
answers
531
views
Cramer-Rao bound in case of non-invertible Fisher Information matrix
I am learning about the Cramer-Rao lower bound (CRLB) and Fisher Information matrix (FIM), and started trying to apply it to some simple toy models from physics. However, even for a simple example I ...
3
votes
3
answers
12k
views
Logistic Regression Failed in statsmodel but works in sklearn; Breast Cancer dataset
I am learning about both the statsmodel library and sklearn. I am trying to construct a logistic model for both libraries trained on the same dataset.
In sklearn, the following works:
...
3
votes
2
answers
226
views
R glmer poisson model and random effect singularity
I am running a Poisson model using glmer to look at the effect of a treatment on fat scores (scale from 1-5, hence the Poisson) of an animal. There are multiple timepoints in which fat scores were ...
3
votes
1
answer
5k
views
Matrices: system that is "computationally singular" versus "exactly singular" [closed]
I would like to know the mathematical concepts behind singular matrices. Matrices that do not have inverses in R throw one of two errors. I have provided some examples of both errors below:
Error in ...
3
votes
2
answers
124
views
Mixed Model - Random Variable implies Fixed Variable
I am fitting a linear mixed model for the first time. I have a dataset that looks something like this:
Surgeon
Handed
Illness
Speed
1
Left
A
8
2
Right
B
15
3
Right
C
12
4
Left
A
10
1
Left
B
10
...
3
votes
2
answers
758
views
Choosing Random Effects to Include in a Linear Mixed Model
I'm trying to run a linear mixed model (in R) but my model either never seems to finish running or (with a simpler random effects structure) there is a warning about singular effects. My full model is ...
3
votes
3
answers
265
views
OLS: covariance matrix invertibility problem when rows < columns
I have read that in OLS when a number of rows (i.e. observations) is smaller than a number of columns (i.e. variables), the covariance matrix $X^{T}X$ cannot be inverted when parameters are being ...
3
votes
1
answer
57
views
Solutions to a 'singular fit' in generalized linear mixed-effects models
What are common causes of a 'singular fit' in generalized linear mixed-effects models (GLMMs), especially when including random intercepts for grouping variables?
When using the ...
3
votes
1
answer
734
views
What is causing the singularity in a glmm with simple random effect?
First time poster but have been very grateful over the past couple months for this forum. First and foremost, I apologize in advance if I am not following the right procedures in asking a question.
I ...
3
votes
1
answer
1k
views
Mixed Model Repeated Measures for Before & After Comparison
I'm trying to assess the effectiveness of a program by comparing employee performance before the program vs. after the program. I have 4 years (2 years before vs. 2 years after) of individual-level ...
3
votes
1
answer
2k
views
Multivariate normal with singular covariance
I'm an undergraduate student. I read about multivariate normal distribution in hogg and craig. And i wonder why the covariance is allowed to be positive SEMI-definite. I read this
Normal distribution ...
3
votes
2
answers
2k
views
Formal errors from non-negative least-squares?
I am computing a standard linear regression subject to a positivity constraint using non-negative least squares (lsqnonneg in Matlab, actually). Is it possible to ...
3
votes
1
answer
890
views
Boundaries on correlation coefficient given five other correlations
Is there a general formula for the boundaries of a correlation coefficient given a set of other correlation coefficients? I have seen the formula for three random variables where two correlations are ...
3
votes
1
answer
5k
views
singular fit in lmer, despite no high correlations of random effects
I ran a mixed effects model a few weeks ago, it all went fine, no errors. Here is the model:
...
3
votes
0
answers
212
views
GLMM and BLUPs: high correlation between random effects in a logistic GLMM
Background:
In an experiment, subjects had to choose whether they wanted an immediate reward or to wait for a larger reward (dichotomous dependent variable: yes/no). This choice was made multiple ...
3
votes
0
answers
146
views
Mixed effect model covariance prior
How should I choose the covariance prior for my bglmer model?
This is a model which has the singularity problem.
...
2
votes
1
answer
5k
views
lme4 lmer() multilevel model: why do I have singular fit and a -1 correlation between random effects slope and intercept?
I'm running a varying intercepts varying slopes multilevel model with the lme4::lmer() function with no group level predictors and only one predictor: FilingFee to predict evictionfilingrate.
I ...
2
votes
1
answer
130
views
How to prove that this joint distribution is Gaussian without using probability densities?
Question: I am wondering if there was a way to prove this result without using probability densities:
If $\bf x \sim \mathcal N (m, P)$ and $\bf y \;|\; x \sim \mathcal N (Hx, R)$, then $$\begin{...
2
votes
1
answer
1k
views
Covariance Matrix and Correlation Matrix - Singularity
If a covariance matrix is non-singular, does this implies that correlation matrix is also non-singular.
My guess is it depends on mean vector in
$K_{X} = R_{X} - m_X.{m_X}^H$
Not sure though.
2
votes
1
answer
1k
views
Identical observations in linear regression
I want to do a linear regression $Y = X\beta + e$, but some of the observations (rows in $X$) are identical (about 30 000 out of 50 000 remain after deleting all duplicates), so when I try to ...
2
votes
1
answer
126
views
Largest singular values
Given the positive semi-definite, symmetric matrix $A = bb^T + \sigma^2I$ where b is a column vector is it possible to find the singular values and singular vectors of the matrix analytically? I know ...
2
votes
1
answer
151
views
Does $E(XX^{\top})$ being full rank imply $E(XX^{\top}\mathbf{1}(Y\in A))$ being full rank?
suppose $X=\begin{bmatrix}X_{1}\\X_{2}\end{bmatrix}$ is a discrete random vector with finite support, and $Y$ is a continuous random variable with finite support $[a,b]$, and $A$ is a subset of $[a,b]$...
2
votes
1
answer
727
views
Getting singular fit error on lmer model after standardizing the response variable
I'm running a mixed model with the lmerfunction in R, and am running into an issue with singular fits. My dataset is comprised of 48,538 observations of sleep ...
2
votes
1
answer
1k
views
Prior for covariance matrices in Gaussian Mixtures Model
I am looking to choose a prior that helps me avoid singularities (as mentioned in this answer) in the covariance matrices of a GMM model. The Jeffrey prior (or a simple improper prior) would be very ...
2
votes
1
answer
261
views
Under-constrained models and invertibility of covariance matrix
In Goodfellow et al.'s Deep Learning, the authors write on page 232:
[$\mathbf{X^\top X}$] can be singular whenever the data-generating distribution truly has no variance in some direction, or when ...
2
votes
1
answer
238
views
estimators with singular covariance matrix
Suppose I have 2 vectors of random variables $\boldsymbol\theta_1 \in \mathbb{R^n}$ and $\boldsymbol\theta_2 \in \mathbb{R^m}$ with asymptotic covariance $\Sigma_1$ and $\Sigma_2$ respectively. I want ...
2
votes
1
answer
265
views
Linear dependency among columns and rows
Singular matrix is defined as square matrix with the determinant of zero. The determinant of zero occurs when matrix columns are linearly dependent (i.e. one of the columns can be defined as a linear ...
2
votes
0
answers
78
views
Why financial time series have perfect multicollinearity?
I have daily financial time series of stock returns (35 stocks) which I took the natural logarithm and subtracted the risk-free rate. However, I get the issue non-invertibility of the covariance ...
2
votes
0
answers
328
views
Where does problems analytically arise in logistic regression on singular data matrix?
If you do linear regression without regularization on close-to-singular data matrix $X$ (or it does not have enough data), the problem arises even in closed-form solution $w = (X^TX)^{-1}X^Ty$ when ...
2
votes
0
answers
355
views
Loss function for rank deficient covariance matrices?
I'm trying to compare the efficiency of different estimators of the covariance matrix of a particular type of multivariate normally distributed data. This comparison, as well as the estimation process ...