Questions tagged [singular-matrix]

A matrix is singular when its determinant is 0; for such matrices, the inverse is not defined. Also, related topics like singular fits

Filter by
Sorted by
Tagged with
99 votes
1 answer
121k views

What correlation makes a matrix singular and what are implications of singularity or near-singularity?

I am doing some calculations on different matrices (mainly in logistic regression) and I commonly get the error "Matrix is singular", where I have to go back and remove the correlated variables. My ...
Error404's user avatar
  • 1,411
50 votes
2 answers
123k views

Dealing with singular fit in mixed models

Let's say we have a model ...
User33268's user avatar
  • 1,722
11 votes
1 answer
9k views

Understanding degenerate multivariate normal distribution

MVN is degenerate when the covariance matrix $\Sigma$ is singular. I am trying to understand mainly conceptual (but also theoretical) implications of this. The Wikipedia article is quite terse. It ...
Davor Josipovic's user avatar
9 votes
3 answers
1k views

Is the sum of two singular covariance matrices also singular?

I have two sample covariance matrices, computed from $n$ samples, less than $p$ variables: they are singular then. I know that the sum of two covariance matrices is also a covariance matrix. My ...
Larel5000's user avatar
9 votes
2 answers
4k views

What is the relation between singular correlation matrix and PCA?

Can anyone kindly give me some information about the statement (last sentence) at the end of below definition. What does it mean by "It can be used when a correlation matrix is singular"? This quote ...
kzmlbyrk's user avatar
  • 193
8 votes
1 answer
6k views

Uniqueness for OLS linear regression

I'm implementing OLS linear regression without using the built-in functions in Matlab with normal equation: I know this is probably very basic, but I want to double check, the input X yields a unique ...
Austin's user avatar
  • 753
8 votes
1 answer
485 views

Bayesian regression with singular $(X'X)$ - Is the posterior well-defined?

SE community, I hope to get some insights into the following problem. Given a simple linear regression model $$Y=X\beta+\epsilon\text{ , where } Y\in\mathbb{R}^T,X\in\mathbb{R}^{T \times N}.$$ Under a ...
Stefan Voigt's user avatar
  • 1,330
8 votes
0 answers
11k views

Singular fit with simplest random structure in lmer (lme4), is a Bayesian approach the only option?

I'm running a mixed model with the lmer function from the lme4 package in R and ran into some issues with singular fits. I get the warning message 'singular fit', ...
Urs's user avatar
  • 81
6 votes
2 answers
2k views

Sampling from matrix-variate normal distribution with singular covariances? [duplicate]

The matrix-variate normal distribution can be sampled indirectly by utilizing the Cholesky decomposition of two positive definite covariance matrices. However, if one or both of the covariance ...
baf84b4c's user avatar
  • 163
5 votes
2 answers
23k views

How do I avoid computationally singular matrices in R?

I'm fitting a logistic regression model (with R's caret package) to data here. I aim to predict whether Hillary or Trump will ...
Noah Walton's user avatar
5 votes
1 answer
273 views

Limit of Bernoulli R.V.s is a singular distribution

Working through an exercise in Probability (the question can be found in Lamperti). Let $X_1,\dots$ be independent Bernoulli random variables with $\mathbb{P}(X_i=1) = p$ and $\mathbb{P}(X_i=0)=1-p$. ...
user89635's user avatar
5 votes
1 answer
4k views

Singular Fisher Information matrix, why is it a problem

I need to perform some research on the consequence of a singular Fisher Information matrix in statistical inference. I am confused about what kind of problems a singular Fisher Information matrix ...
dietervdf's user avatar
  • 1,212
5 votes
2 answers
3k views

R gls() vs. SAS proc mixed with interaction: Why does R complain about a singular matrix when SAS does not?

I like to keep analyses all in SAS or all in R when I can help it and lately have been using R more and more, but there's one analysis that I do somewhat routinely that has given me trouble in R. I ...
Sam Dickson's user avatar
4 votes
2 answers
2k views

How to solve the error of singular fit in glmm in R

I am trying to fit a GLMM for binary data of whether colonies of bees perform mass flight or not. I have time when the mass flight was performed, temperature, location of the hive and species of the ...
Awanti's user avatar
  • 41
4 votes
1 answer
5k views

Singular Matrix and Linear Dependency

Singular matrix is defined as a square matrix with determinant of zero. I am aware that linear dependency among columns or rows leads to determinant being equal to zero (e.g. one column is a linear ...
PsychometStats's user avatar
4 votes
1 answer
221 views

Linear Mixed Model equation (as of lme4 package)

I am trying to derive the equations of a linear mixed model as specified in the documentation of the lme4 package: "Fitting Linear Mixed-Effects Models using lme4" jstatsoft.org/article/view/v067i01 ...
Seiji's user avatar
  • 65
4 votes
1 answer
4k views

Why is within class scatter matrix in LDA singular? [closed]

I read that when number of data points are much less than the dimension of data, the within class scatter matrix in singular? Can someone explain why this is the case? For example, while using LDA for ...
Oliver Blue's user avatar
4 votes
1 answer
2k views

Does it make sense to use PCA when the determinant of the correlation matrix is (almost) zero?

I'm running a PCA over a data set of $N \times p$ size ($N\approx 1000$ being the number of measurements and $p\approx 200$ being the number of dimensions/predictors). I expect many of the predictors ...
Marco Mene's user avatar
4 votes
1 answer
227 views

Warnings for confint() in lme4

I am fitting the following model (random intercepts and slopes) on my data: lmer(MuscleActivity ~ Period+ (1 + Period|ppnr), data = df) My goal is to test whether ...
Leon164's user avatar
  • 41
4 votes
1 answer
805 views

Are Kalman Filter recursions valid when the state noise has a singular covariance matrix?

Consider a Linear Gaussian State-Space Model where the states are denoted by $X_t$ and observations are denoted by $Y_t$: \begin{align} X_t &= A X_{t-1} + \epsilon_t, &&\epsilon_t \sim \...
user avatar
4 votes
0 answers
531 views

Cramer-Rao bound in case of non-invertible Fisher Information matrix

I am learning about the Cramer-Rao lower bound (CRLB) and Fisher Information matrix (FIM), and started trying to apply it to some simple toy models from physics. However, even for a simple example I ...
PianoEntropy's user avatar
3 votes
3 answers
12k views

Logistic Regression Failed in statsmodel but works in sklearn; Breast Cancer dataset

I am learning about both the statsmodel library and sklearn. I am trying to construct a logistic model for both libraries trained on the same dataset. In sklearn, the following works: ...
finite_diffidence's user avatar
3 votes
2 answers
226 views

R glmer poisson model and random effect singularity

I am running a Poisson model using glmer to look at the effect of a treatment on fat scores (scale from 1-5, hence the Poisson) of an animal. There are multiple timepoints in which fat scores were ...
bluebird8's user avatar
3 votes
1 answer
5k views

Matrices: system that is "computationally singular" versus "exactly singular" [closed]

I would like to know the mathematical concepts behind singular matrices. Matrices that do not have inverses in R throw one of two errors. I have provided some examples of both errors below: Error in ...
sebelly's user avatar
  • 31
3 votes
2 answers
124 views

Mixed Model - Random Variable implies Fixed Variable

I am fitting a linear mixed model for the first time. I have a dataset that looks something like this: Surgeon Handed Illness Speed 1 Left A 8 2 Right B 15 3 Right C 12 4 Left A 10 1 Left B 10 ...
Carol Eisen's user avatar
3 votes
2 answers
758 views

Choosing Random Effects to Include in a Linear Mixed Model

I'm trying to run a linear mixed model (in R) but my model either never seems to finish running or (with a simpler random effects structure) there is a warning about singular effects. My full model is ...
SilvaC's user avatar
  • 512
3 votes
3 answers
265 views

OLS: covariance matrix invertibility problem when rows < columns

I have read that in OLS when a number of rows (i.e. observations) is smaller than a number of columns (i.e. variables), the covariance matrix $X^{T}X$ cannot be inverted when parameters are being ...
PsychometStats's user avatar
3 votes
1 answer
57 views

Solutions to a 'singular fit' in generalized linear mixed-effects models

What are common causes of a 'singular fit' in generalized linear mixed-effects models (GLMMs), especially when including random intercepts for grouping variables? When using the ...
Wagathu's user avatar
  • 179
3 votes
1 answer
734 views

What is causing the singularity in a glmm with simple random effect?

First time poster but have been very grateful over the past couple months for this forum. First and foremost, I apologize in advance if I am not following the right procedures in asking a question. I ...
user326575's user avatar
3 votes
1 answer
1k views

Mixed Model Repeated Measures for Before & After Comparison

I'm trying to assess the effectiveness of a program by comparing employee performance before the program vs. after the program. I have 4 years (2 years before vs. 2 years after) of individual-level ...
LY1's user avatar
  • 33
3 votes
1 answer
2k views

Multivariate normal with singular covariance

I'm an undergraduate student. I read about multivariate normal distribution in hogg and craig. And i wonder why the covariance is allowed to be positive SEMI-definite. I read this Normal distribution ...
Aeroplane's user avatar
  • 463
3 votes
2 answers
2k views

Formal errors from non-negative least-squares?

I am computing a standard linear regression subject to a positivity constraint using non-negative least squares (lsqnonneg in Matlab, actually). Is it possible to ...
EMB's user avatar
  • 41
3 votes
1 answer
890 views

Boundaries on correlation coefficient given five other correlations

Is there a general formula for the boundaries of a correlation coefficient given a set of other correlation coefficients? I have seen the formula for three random variables where two correlations are ...
Cristian's user avatar
3 votes
1 answer
5k views

singular fit in lmer, despite no high correlations of random effects

I ran a mixed effects model a few weeks ago, it all went fine, no errors. Here is the model: ...
Galit's user avatar
  • 177
3 votes
0 answers
212 views

GLMM and BLUPs: high correlation between random effects in a logistic GLMM

Background: In an experiment, subjects had to choose whether they wanted an immediate reward or to wait for a larger reward (dichotomous dependent variable: yes/no). This choice was made multiple ...
バシル's user avatar
  • 103
3 votes
0 answers
146 views

Mixed effect model covariance prior

How should I choose the covariance prior for my bglmer model? This is a model which has the singularity problem. ...
User33268's user avatar
  • 1,722
2 votes
1 answer
5k views

lme4 lmer() multilevel model: why do I have singular fit and a -1 correlation between random effects slope and intercept?

I'm running a varying intercepts varying slopes multilevel model with the lme4::lmer() function with no group level predictors and only one predictor: FilingFee to predict evictionfilingrate. I ...
chase171's user avatar
2 votes
1 answer
130 views

How to prove that this joint distribution is Gaussian without using probability densities?

Question: I am wondering if there was a way to prove this result without using probability densities: If $\bf x \sim \mathcal N (m, P)$ and $\bf y \;|\; x \sim \mathcal N (Hx, R)$, then $$\begin{...
user avatar
2 votes
1 answer
1k views

Covariance Matrix and Correlation Matrix - Singularity

If a covariance matrix is non-singular, does this implies that correlation matrix is also non-singular. My guess is it depends on mean vector in $K_{X} = R_{X} - m_X.{m_X}^H$ Not sure though.
urvah shabbir's user avatar
2 votes
1 answer
1k views

Identical observations in linear regression

I want to do a linear regression $Y = X\beta + e$, but some of the observations (rows in $X$) are identical (about 30 000 out of 50 000 remain after deleting all duplicates), so when I try to ...
H.v.M.'s user avatar
  • 123
2 votes
1 answer
126 views

Largest singular values

Given the positive semi-definite, symmetric matrix $A = bb^T + \sigma^2I$ where b is a column vector is it possible to find the singular values and singular vectors of the matrix analytically? I know ...
user avatar
2 votes
1 answer
151 views

Does $E(XX^{\top})$ being full rank imply $E(XX^{\top}\mathbf{1}(Y\in A))$ being full rank?

suppose $X=\begin{bmatrix}X_{1}\\X_{2}\end{bmatrix}$ is a discrete random vector with finite support, and $Y$ is a continuous random variable with finite support $[a,b]$, and $A$ is a subset of $[a,b]$...
ExcitedSnail's user avatar
  • 2,748
2 votes
1 answer
727 views

Getting singular fit error on lmer model after standardizing the response variable

I'm running a mixed model with the lmerfunction in R, and am running into an issue with singular fits. My dataset is comprised of 48,538 observations of sleep ...
T.Grover's user avatar
2 votes
1 answer
1k views

Prior for covariance matrices in Gaussian Mixtures Model

I am looking to choose a prior that helps me avoid singularities (as mentioned in this answer) in the covariance matrices of a GMM model. The Jeffrey prior (or a simple improper prior) would be very ...
nestor556's user avatar
  • 270
2 votes
1 answer
261 views

Under-constrained models and invertibility of covariance matrix

In Goodfellow et al.'s Deep Learning, the authors write on page 232: [$\mathbf{X^\top X}$] can be singular whenever the data-generating distribution truly has no variance in some direction, or when ...
Vivek Subramanian's user avatar
2 votes
1 answer
238 views

estimators with singular covariance matrix

Suppose I have 2 vectors of random variables $\boldsymbol\theta_1 \in \mathbb{R^n}$ and $\boldsymbol\theta_2 \in \mathbb{R^m}$ with asymptotic covariance $\Sigma_1$ and $\Sigma_2$ respectively. I want ...
Kumar's user avatar
  • 709
2 votes
1 answer
265 views

Linear dependency among columns and rows

Singular matrix is defined as square matrix with the determinant of zero. The determinant of zero occurs when matrix columns are linearly dependent (i.e. one of the columns can be defined as a linear ...
PsychometStats's user avatar
2 votes
0 answers
78 views

Why financial time series have perfect multicollinearity?

I have daily financial time series of stock returns (35 stocks) which I took the natural logarithm and subtracted the risk-free rate. However, I get the issue non-invertibility of the covariance ...
Mataunited17's user avatar
2 votes
0 answers
328 views

Where does problems analytically arise in logistic regression on singular data matrix?

If you do linear regression without regularization on close-to-singular data matrix $X$ (or it does not have enough data), the problem arises even in closed-form solution $w = (X^TX)^{-1}X^Ty$ when ...
MInner's user avatar
  • 283
2 votes
0 answers
355 views

Loss function for rank deficient covariance matrices?

I'm trying to compare the efficiency of different estimators of the covariance matrix of a particular type of multivariate normally distributed data. This comparison, as well as the estimation process ...
Ruben van Bergen's user avatar