Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

A matrix is singular when its determinant is 0; for such matrices, the inverse is not defined.

1
vote
1answer
46 views

Matrices: system that is “computationally singular” versus “exactly singular”

I would like to know the mathematical concepts behind singular matrices. Matrices that do not have inverses in R throw one of two errors. I have provided some examples of both errors below: Error in ...
1
vote
2answers
37 views

Multiple regression - singularity issues [duplicate]

I am trying to fit multivariate regression models to my data; however, I get singularity warnings. Please find a part of my data below: ...
0
votes
0answers
25 views

RBF Interpolation Singular Matrix Error

I am attempting to RBF interpolate a set of data, however the interpolation fails when numpy reaches a singular matrix it cannot handle. I saw this post and thought ...
2
votes
0answers
32 views

Why financial time series have perfect multicollinearity?

I have daily financial time series of stock returns (35 stocks) which I took the natural logarithm and subtracted the risk-free rate. However, I get the issue non-invertibility of the covariance ...
0
votes
1answer
58 views

Why may a matrix be singular or ill-conditioned with standard learning algorithm for linear classification?

In the learning algorithm for linear classification by least square method, which find a weight vector $\hat w\in R^d$ and bias $\hat b\in R$ for a linear scoring function $f(x) = \hat w ^T x +\hat b$ ...
1
vote
1answer
90 views

Prior for covariance matrices in Gaussian Mixtures Model

I am looking to choose a prior that helps me avoid singularities (as mentioned in this answer) in the covariance matrices of a GMM model. The Jeffrey prior (or a simple improper prior) would be very ...
1
vote
1answer
174 views

Prove $A(A+B)^{-1}B=B(A+B)^{-1}A$

I have this equality, $A(A+B)^{-1}B=B(A+B)^{-1}A$ and the question specifically only states that $A+B$ is nonsingular. I have looked at this many ways but the only I can see it working is if $A+B$ ...
0
votes
1answer
24 views

differences in forecast and reconstruction in SSA, in R

I was playing around with the Rssa when I discovered this: Firstly: I created to sequences: library Rssa x<-1:100 x1<-1:80 then the corresponding function:...
2
votes
1answer
159 views

Boundaries on correlation coefficient given five other correlations

Is there a general formula for the boundaries of a correlation coefficient given a set of other correlation coefficients? I have seen the formula for three random variables where two correlations are ...
1
vote
1answer
248 views

Stepwise quantile regression: What's the reason behind these strange results?

So I am attempting to build a model using quantile regression & am using stepwise regression for initial data exploration. I'm well aware that stepwise methods are widely frowned upon & am ...
1
vote
0answers
36 views

Consequence of singular weight matrices in neural networks

I want to know the effects of weight matrices being singular when using neural networks. Is there any supported literature? For non-square weights matrices, I am concerned about lower rank matrices. ...
0
votes
0answers
12 views

p-values close to eps [duplicate]

After fitting a linear mixed effect model with lme, I run a posthoc analysis with glht and get the following results: ...
1
vote
1answer
36 views

Under-constrained models and invertibility of covariance matrix

In Goodfellow et al.'s Deep Learning, the authors write on page 232: [$\mathbf{X^\top X}$] can be singular whenever the data-generating distribution truly has no variance in some direction, or when ...
1
vote
0answers
98 views

How can I figure out the singularity problem in estimating the indicator variable coefficient in time series model ?

I want to capture the role of the sign of the previous return in my time series model(AR(1) and QAR(1)) in R both with quantile and OLS regression , but I face error. Here is the code: ...
2
votes
1answer
610 views

Singular Fisher Information matrix, why is it a problem

I need to perform some research on the consequence of a singular Fisher Information matrix in statistical inference. I am confused what kind of problems a singular Fisher Information matrix creates. ...
3
votes
1answer
536 views

Uniqueness for OLS linear regression

I'm implementing OLS linear regression without using the built-in functions in Matlab with normal equation: I know this is probably very basic, but I want to double check, the input X yields a unique ...
6
votes
1answer
2k views

Understanding degenerate multivariate normal distribution

MVN is degenerate when the covariance matrix $\Sigma$ is singular. I am trying to understand mainly conceptual (but also theoretical) implications of this. The Wikipedia article is quite terse. It ...
1
vote
0answers
272 views

Singular matrix but it's full rank

I'm using matlab to fit a logit GLM to a data (detection problem). I have a Nx5 matrix of independent variables and a binary (i.e 0-1) column vector of responses. When I try to fit the GLM model with <...
0
votes
0answers
137 views

Choice of window length in Singular Spectrum Analysis

Thanks in advance for your answers, I'm trying to decompose a stock price using the Rssa package provided in R, after millions of trials, I tried to change the window Length and the eigentriples, the ...
3
votes
0answers
133 views

Where does problems analytically arise in logistic regression on singular data matrix?

If you do linear regression without regularization on close-to-singular data matrix $X$ (or it does not have enough data), the problem arises even in closed-form solution $w = (X^TX)^{-1}X^Ty$ when ...
6
votes
2answers
3k views

How do I avoid computationally singular matrices in R?

I'm fitting a logistic regression model (with R's caret package) to data here. I aim to predict whether Hillary or Trump will ...
1
vote
1answer
1k views

Why is within class scatter matrix in LDA singular? [closed]

I read that when number of data points are much less than the dimension of data, the within class scatter matrix in singular? Can someone explain why this is the case? For example, while using LDA for ...
2
votes
2answers
425 views

Sampling from matrix-variate normal distribution with singular covariances? [duplicate]

The matrix-variate normal distribution can be sampled indirectly by utilizing the Cholesky decomposition of two positive definite covariance matrices. However, if one or both of the covariance ...
6
votes
1answer
79 views

Limit of Bernoulli R.V.s is a singular distribution

Working through an exercise in Probability (the question can be found in Lamperti). Let $X_1,\dots$ be independent Bernoulli random variables with $\mathbb{P}(X_i=1) = p$ and $\mathbb{P}(X_i=0)=1-p$. ...
0
votes
0answers
195 views

Covariance Matrix and Correlation Matrix - Singularity

If a covariance matrix is non-singular, does this implies that correlation matrix is also non-singular. My guess is it depends on mean vector in $K_{X} = R_{X} - m_X.{m_X}^H$ Not sure though.
5
votes
1answer
716 views

Does it make sense to use PCA when the determinant of the correlation matrix is (almost) zero?

I'm running a PCA over a data set of $N \times p$ size ($N\approx 1000$ being the number of measurements and $p\approx 200$ being the number of dimensions/predictors). I expect many of the predictors ...
8
votes
1answer
197 views

Bayesian regression with singular $(X'X)$ - Is the posterior well-defined?

SE community, I hope to get some insights into the following problem. Given a simple linear regression model $$Y=X\beta+\epsilon\text{ , where } Y\in\mathbb{R}^T,X\in\mathbb{R}^{T \times N}.$$ Under a ...
1
vote
0answers
320 views

Multinomial logistic regression on categorical data results in singular matrix

I have a categorical dataset from a biological sampling study consisting of six variables, see http://pastebin.com/ncLzw0Mt for the actual dataset. ...
0
votes
1answer
77 views

estimators with singular covariance matrix

Suppose I have 2 vectors of random variables $\boldsymbol\theta_1 \in \mathbb{R^n}$ and $\boldsymbol\theta_2 \in \mathbb{R^m}$ with asymptotic covariance $\Sigma_1$ and $\Sigma_2$ respectively. I want ...
1
vote
1answer
341 views

Identical observations in linear regression

I want to do a linear regression $Y = X\beta + e$, but some of the observations (rows in $X$) are identical (about 30 000 out of 50 000 remain after deleting all duplicates), so when I try to ...
0
votes
0answers
815 views

Singular Hessian/Observed information Matrix at optimal solution

I am trying to estimate the standard errors of an maximum likelihood estimate (multidimensional) in R'sfunction optim. I want to ...
5
votes
2answers
2k views

What is the relation between singular correlation matrix and PCA?

Can anyone kindly give me some information about the statement (last sentence) at the end of below definition. What does it mean by "It can be used when a correlation matrix is singular"? This quote ...
2
votes
0answers
233 views

Loss function for rank deficient covariance matrices?

I'm trying to compare the efficiency of different estimators of the covariance matrix of a particular type of multivariate normally distributed data. This comparison, as well as the estimation process ...
1
vote
0answers
2k views

R - system is computationally singular - dealing with small numbers

I'm working with a ~200x200 Markovian transition matrix of non-zero probabilities. Forcibly, these probabilities are, for the large part, going to be very small. I am trying to find the inverse of my ...
1
vote
1answer
917 views

Multivariate normal with singular covariance

I'm an undergraduate student. I read about multivariate normal distribution in hogg and craig. And i wonder why the covariance is allowed to be positive SEMI-definite. I read this Normal distribution ...
1
vote
0answers
518 views

Zero inflated model problem: system is computationally singular

I'm using R.After getting an error asking me to provide starting values for a glm (poisson family), I took a look at my data and realized I had quite a bit of zeroes. So, I tried zeroinfl from pscl. I ...
2
votes
0answers
281 views

Probit regression in R giving singular Hessian matrix

I am trying to run a probit regression using panel data in R by first computing the log likelihood and then using the optim function to optimize. Scale of ...
4
votes
2answers
2k views

R gls() vs. SAS proc mixed with interaction: Why does R complain about a singular matrix when SAS does not?

I like to keep analyses all in SAS or all in R when I can help it and lately have been using R more and more, but there's one analysis that I do somewhat routinely that has given me trouble in R. I ...
54
votes
1answer
53k views

What correlation makes a matrix singular and what are implications of singularity or near-singularity?

I am doing some calculations on different matrices (mainly in logistic regression) and I commonly get the error "Matrix is singular", where I have to go back and remove the correlated variables. My ...
0
votes
1answer
729 views

Formal errors from non-negative least-squares?

I am computing a standard linear regression subject to a positivity constraint using non-negative least squares (lsqnonneg in Matlab, actually). Is it possible to ...