Questions tagged [misspecification]

Problems with model specification, such as missing variables/predictors, wrong functional form, wrong variance or covariance structure, etc.

Filter by
Sorted by
Tagged with
74
votes
11answers
8k views

Why should I be Bayesian when my model is wrong?

Edits: I have added a simple example: inference of the mean of the $X_i$. I have also slightly clarified why the credible intervals not matching confidence intervals is bad. I, a fairly devout ...
28
votes
6answers
75k views

Inclusion of lagged dependent variable in regression

I'm very confused about if it's legitimate to include a lagged dependent variable into a regression model. Basically I think if this model focuses on the relationship between the change in Y and other ...
26
votes
2answers
5k views

Is it true that Bayesian methods don't overfit?

Is it true that Bayesian methods don't overfit? (I saw some papers and tutorials making this claim) For example, if we apply a Gaussian Process to MNIST (handwritten digit classification), but only ...
23
votes
2answers
1k views

Why doesn't Wilks' 1938 proof work for misspecified models?

In the famous 1938 paper ("The large-sample distribution of the likelihood ratio for testing composite hypotheses", Annals of Mathematical Statistics, 9:60-62), Samuel Wilks derived the asymptotic ...
15
votes
2answers
284 views

Statistical Inference Under Misspecification

The classical treatment of statistical inference relies on the assumption that that a correctly specified statistical is used exists. That is, the distribution $\mathbb{P}^*(Y)$ that generated the ...
10
votes
2answers
7k views

When to use (non)parametric test of homoscedasticity assumption?

If one is testing assumption of homoscedasticity, parametric (Bartlett Test of Homogeneity of Variances, bartlett.test) and non-parametric (Figner-Killeen Test of ...
9
votes
2answers
318 views

Statistical inference under model misspecification

I have a general methodological question. It might have been answered before, but I am not able to locate the relevant thread. I will appreciate pointers to possible duplicates. (Here is an excellent ...
7
votes
1answer
2k views

Distribution of random effects

Why do we usually assume that random effects come from a normal distribution? Can we assume another distribution? Or maybe because the CLT indicates that a random effect is normally distributed?
7
votes
1answer
247 views

Effects of model selection and misspecification testing on inference: Probabilistic Reduction approach (Aris Spanos)

This question is about pre-test bias, inference after model selection and data snooping within the Probabilistic Reduction (PR) methodology by Aris Spanos (which is related to the Error Statistics ...
7
votes
1answer
444 views

Test incorrect functional form when residuals have non-normal distribution

J. B. Ramsey (in "Tests for specification errors in classical linear least-squares regression analysis." Journal of the Royal Statistical Society. 1969) says that the RESET test assumes that the ...
6
votes
4answers
154 views

Could logistic regression be used to detect large errors in least squares regression?

I have the following linear model: $$w^*=\text{arg min}_w\sum_{i=1}^N \bigg(Y_i-\sum_{j=1}^M X_{i,j}\times w_j\bigg)^2$$ Let $T \in N^*$ and $e_i=|Y_i-\sum_{j=1}^M X_{i,j}\times w_j|$. It's ...
6
votes
2answers
4k views

Conditional vs. Marginal models

I have data with an outcome of 0 or 1 (binary) representing success or failure. I also have two comparison groups (Treatment vs. Control). Each subject in the study contributed 2 observations (the ...
6
votes
1answer
1k views

Fixed Regressor Conspiracy and Connection to Exchangeability

In simple regression model regressors are treated as fixed rather than stochastic. Whoever picks the experimental values for the regressors, decides in which frequency to include each value. This can ...
5
votes
2answers
2k views

Robust standard errors in econometrics

I keep hearing my professor try to explain that we can use robust standard errors when we run a regression to confront the issue of heteroskedasticity. However I don't quite understand how telling ...
5
votes
1answer
121 views

Sample $R^2$ consistent?

In a linear regression context: is the sample $\widehat{R^2}$ a consistent estimator of the population parameter $R^2$? Maybe this depends on distributional assumptions?
5
votes
3answers
4k views

Statistical test to determine if a relationship is linear?

What is the best statistical test to use if I measure the value of $Y$ (e.g. pH) for specific values of $X$ e.g. $X=0,10,20,30,...,100$ (e.g. temperature) and I want to test weather the relationship ...
5
votes
1answer
600 views

Statistical Significance or Unambiguous Direction of Influence?

TL; DR In the context of a linear regression model, we run a statistical test for whether an estimated coefficient is "statistically significant". We will say that it is if we reject the null of it ...
5
votes
1answer
1k views

Misspecified levels in multilevel mixed logit model

I'm estimating a couple 3 level logit models using Stata 12 and am faced with a dilemma about how (or if) I should specify my third level. The data is court cases nested within judges nested within ...
5
votes
0answers
103 views

When does the prediction of random effects matter?

In linear or generalized linear mixed effects models, random effects are incorporated to explain the within-unit correlation for repeated measures over time. In Bayesian modeling, conventional prior ...
4
votes
3answers
952 views

What is the interpretation/meaning of confidence intervals in misspecified models?

Consider the following model $Y_i = f(X_i) + e_i$ from which we observe n iid data points $\left( X_i, Y_i \right)_{i=1}^n$. Suppose that $X_i \in \mathbb{R}^d$ is a $d$ dimensional feature vector. ...
4
votes
1answer
1k views

Does the sandwich estimator in GEE protect against both correlation misspecification and heteroscedasticity?

The relative merits of GEE with exchangeable correlation or GEE with independence and the sandwich estimate have been discussed, but I couldn't find a post specifically addressing my question. I have ...
4
votes
1answer
69 views

Non-Linear regression and variance misspecification

Given a non-linear regression model for cross-section data $$y_i = f(x_i,\theta_0) + \epsilon_i,$$ where it is assumed that $\mathbb E[y_i\lvert x_i] = f(x_i,\theta_0)$, I understand that it is a ...
4
votes
1answer
100 views

Some logical questions about parameter estimation in the situations when the model is misspecified

When we have a parametric model, we can use many procedure to estimate the parameters in the model, i.e., obtain many different estimators. We usually focus on the set of consistent estimators. For ...
4
votes
2answers
324 views

Problems due to analyzing variables from different levels at one single level

Please ease the following paragraph from the first chapter , Introduction to Multilevel Analysis , p.3 of the book: Historically , multilevel problems have led to analysis approaches that moved all ...
4
votes
0answers
76 views

literature on small samples and parametric survival models

I have an abundance of small data sets with right-censored data. There are different groups in each data set and I'd like to get confidence intervals for the regression parameters. Each data set has 3-...
3
votes
3answers
177 views

Bayesian inference with false models: to what does it converge?

This is the second follow up question from these two previous questions: Bayesian inference and testable implications How do I perform an actual "posterior predictive check" in this model? ...
3
votes
1answer
122 views

Recommend monograph on statistical model misspecification

Is there a good book on statistical model misspecification in general? It should cover, for example, the behavior of estimators (e.g., maximum likelihood) when the specified parametric family does not ...
3
votes
1answer
3k views

Strictly positive response in regression: what should my “default” model be?

For unbounded continuous responses, Gaussian errors are the analyst's default model for many reasons, one of them being that their ML estimate coincides with the OLS estimate that has many desirable ...
3
votes
3answers
1k views

Endogeneity & IV = model misspecification?

I'd like to raise a controversial point: if you need instrumental variables, your model is wrong. Basic endogeneity problem and the IV solution Let us suppose the basic framework of endogeneity and ...
3
votes
1answer
107 views

Partial or incomplete Bayesian update

In Bayesian statistics, usually we have some prior distribution $P(M)$, some observations $X$, and we compute a posterior $P(M|X)$. The observation allows us to gain information about the generating ...
3
votes
2answers
383 views

Using simulated data to check when patterns in GLMM residual plots are acceptable

I have run the following Poisson GLMM: ...
3
votes
1answer
42 views

Consequences of choosing the wrong GLM response distribution

In Chapter 10 of McElreath's Statistical Rethinking (2nd edition), he argues that the response distribution for a GLM should be chosen to maximize entropy given a set of constraints on the response ...
3
votes
0answers
201 views

How do I perform an actual “posterior predictive check”?

This question is the follow-up of this previous question: Bayesian inference and testable implications. For concreteness, consider the following bayesian model. This model is not to be taken ...
3
votes
0answers
58 views

Over “specification” of a statistics model?

For a simple example, I am fitting data with a likelihood function generated by a normal distribution. The first model is the normal distribution with two-parameters. The second competing model is the ...
3
votes
0answers
76 views

Does a high Chi square p-value for a whole model mean it is insignificant if likelihood ratio tests indicated variables should be added?

I've been estimating lots of versions of the same model by incrementally adding variables. With some variables, if I add them to the model, the likelihood ratio test indicates that they are ...
3
votes
0answers
5k views

What to do when ovtest and linktest in Stata suggest model misspecification?

I have a sample that consists of 50 observations. The base model of the OLS-Regression with three control variables, two of them significant, has a $R^2=0.50$ and its F-Value is 7. Both ...
2
votes
1answer
557 views

How to calculate sandwich standard errors for generalized least squares models?

Dependent data can be modeled using covariance structures like compound symmetry, spherical, AR-1, and other. Using generalized least squares, inference can be made on the regression coefficients ...
2
votes
1answer
94 views

Why does $\mathbb{E}[s^2] \geq \sigma^2$ if we omit relevant variables (linear regression model)?

Suppose that the "true" model is $Y = X\beta + Z \gamma + \varepsilon$ with the standard assumptions of the linear model. However, we only perform an OLS-regression on the variables contained in $X$. ...
2
votes
1answer
295 views

Robust error estimation and hazard ratio with non-proportional hazards

I recall having heard that the hazard ratio, estimated in a Cox model, can be made robust against the parallel hazard functions assumption. The key to this is using a Huber-White, or Huber-Eicker-...
2
votes
2answers
1k views

Central limit theorem for maximum likelihood estimators when modelling assumptions are violated

Lehman's Element's of Statistical Learning Theory gives in Theorem 7.5.2 a central limit theorem for multiparamter maximum likelihood estimators. (Many other sources provide similar theorems.) The ...
2
votes
2answers
66 views

$R^2$ is too high- reasons? [closed]

What are the reasons of too high values of $R^2$? I only know that its value increases when the number of independent variables increases. What are the other factors?
2
votes
1answer
271 views

Does having a result that is “robust to specification” make it more likely to be true?

I'm wondering if there is any evidence (simulated or otherwise) demonstrating that robustness to specification actually means a result is more likely to be true. By robustness to specification I mean ...
2
votes
1answer
57 views

How to correct inference for searching over multiple specifications?

Correcting for multiple testing is easy, and there's a lot of literature around it. But now consider a different problem. Say you have $X$ and $Y$ and you want to estimate $E[Y|X=x]= f(x)$ where $f$ ...
2
votes
1answer
708 views

residuals increases with fitted values : what kind of model misspecification might it indicate?

As we know, plotting residuals against fitted values may indicate whether the model is misspecified or the variance is not constant, or both. Let's focus on model misspecification only and set aside ...
2
votes
1answer
179 views

How to quantify mis-specification bias and compare against smoothing bias for a non-parametric estimate of a randomly allocated continuous treatment?

Suppose that there is a data-generating process $$ y = \alpha + g(x) + \epsilon $$ which is to say that an outcome is some function of $x$. Suppose that $x$ is randomly assigned, so $\text{Cov}(x,\...
2
votes
2answers
2k views

How to test model misspecification using Johansen test?

There are three misspecification tests that I would like to perform on my model for a Johansen test before continuing with the cointegration tests themselves. Following Johansen's advice in Likelihood-...
2
votes
0answers
16 views

SEM: Why is there no dependable connection between the size of SEM residuals and the type or degree of model misspecification?

Kline (2016) p. 278 writes [Y]ou should know that there is actually no dependable or trustworthy connection between the size of the residuals and the type or degree of model misspecification. For ...
2
votes
0answers
27 views

Random effects for mutually inclusive grouping factors

I am trying to fit a model on a set of data (e.g. 10,000 observations with 20 explanatory variables). The observations belong to 30 groups, G1, G2, G3, ... G30, so I need to account for the grouping ...
2
votes
0answers
76 views

Relative importance different model diagnostics in LMM/GLMM

I may be over thinking things but I have been stuck on this issue for the last few days. I am currently modelling data with a large number of clusters but cluster size is limited to 2, with a skewed ...
2
votes
0answers
212 views

Maximum likelihood estimation for incorrect distribution parameter

Supposing I have a random variable $X$ with distribution $$f(x | \boldsymbol{\theta}, \boldsymbol{\varphi}).$$ Here $\boldsymbol{\theta} = (\theta_1,\theta_2,...,\theta_r)$ are $r$ parameters which ...