Questions tagged [diagnostic]

Diagnostic measures (such as residuals or some summary statistics calculated from residuals) are used to evaluate some aspect of quality of model fit to data.

Filter by
Sorted by
Tagged with
15
votes
3answers
5k views

Is my model any good, based on the diagnostic metric ($R^2$/ AUC/ accuracy/ RMSE etc.) value?

I've fitted my model and am trying to understand whether it's any good. I've calculated the recommended metrics to assess it ($R^2$/ AUC / accuracy / prediction error / etc) but do not know how to ...
36
votes
3answers
29k views

Interpreting residual diagnostic plots for glm models?

I am looking for guidelines on how to interpret residual plots of glm models. Especially poisson, negative binomial, binomial models. What can we expect from these plots when the models are "correct"...
11
votes
4answers
9k views

Diagonal straight lines in residuals vs fitted values plot for multiple regression

I am observing strange patterns in residuals for my data: [EDIT] Here are the partial regression plots for the two variables: [EDIT2] Added the PP Plot The distribution seems to be doing fine (see ...
12
votes
1answer
3k views

Prove the relation between Mahalanobis distance and Leverage?

I have seen formulas on Wikipedia. that relate Mahalanobis distance and Leverage: Mahalanobis distance is closely related to the leverage statistic, $h$, but has a different scale: $$D^2 = (N - 1)(...
12
votes
1answer
530 views

Why are diagnostics based on residuals?

In simple linear regression one often wants to verify if certain assumptions are met to be able to do inference (e.g. residuals are normally distributed). Is it reasonable to check the assumptions ...
29
votes
5answers
4k views

Why does my bootstrap interval have terrible coverage?

I wanted to do a class demonstration where I compare a t-interval to a bootstrap interval and calculate the coverage probability of both. I wanted the data to come from a skewed distribution so I ...
4
votes
0answers
9k views

Fitting a zero-inflated negative binomial regression with R

In this thread, I laid out a problem involving fitting a model that attempts to use minor league baseball statistics to predict success at the major league level (explained in full in the thread). ...
28
votes
2answers
148k views

Likelihood ratio test in R

Suppose I am going to do a univariate logistic regression on several independent variables, like this: ...
44
votes
2answers
95k views

How to read Cook's distance plots?

Does anyone know how to work out whether points 7, 16 and 29 are influential points or not? I read somewhere that because Cook's distance is lower than 1, they are not. Am, I right?
12
votes
4answers
32k views

Is there a test for omitted variable bias in OLS?

I am aware of the Ramsey Reset test which may detect nonlinear dependencies. However, if you just throw out one of the regression coefficients (merely linear dependencies), you may get a bias, ...
7
votes
1answer
5k views

How to identify outliers and do model diagnostics for an lme4 model?

I need to identify outliers and high leverage points, and perform model diagnostics, in an lme4 model. For outliers and high leverage points, simply making a plot ...
8
votes
3answers
2k views

How should I interpret this residual plot?

I am unable to interpret this graph. My dependent variable is total number of movie tickets that will be sold for a show. The independent variables are the number of days left before the show, ...
6
votes
3answers
2k views

What does this plot tell me about my linear model?

I have fit the following linear model, I tested the response by looking at a qq plot and it is almost perfectly linear. When i fit the model though, and study the predicted vs observed plot, It looks ...
1
vote
1answer
1k views

Apparent correlation between standardized residuals and predicted in regression

The following is the residuals vs predicted scatter plot for a regression model with two IVs. Initially, I thought it was evidence of heteroskedasticity. But, I reasoned that although there is a ...
7
votes
1answer
30k views

How to interpret model diagnostics when doing linear regression in R?

I ran lm() on my data with models selected by individual lm's of each characteristic and then combined the top $R^2$ based on $p$...
10
votes
1answer
1k views

What type of post-fit analysis of residuals do you use?

When carrying out OLS multiple linear regression, rather than plot the residuals against fitted values, I plot the (internal) Studentized residuals against fitted values (ditto for covariates). These ...
6
votes
2answers
17k views

Standardized residuals in R's lm output

If I plot the diagnostic plots to an R regression, a couple of them have "Standardized Residuals" as their y-axis such as in this plot: What are the residuals standardized over? That is, let us ...
6
votes
1answer
3k views

Negative Binomial Regression and Heteroskedasticity

What relationship does a negative binomial regression have to heteroskedasticity and if one still needs to check and/or correct for it how would this be done?
9
votes
1answer
6k views

How to manually calculate dfbetas

I am trying to replicate what the function dfbetas() does in R. dfbeta() is not an issue... Here is a set of vectors: ...
3
votes
1answer
2k views

Interpreting Regression Diagnostic Plots

I'm sorry if this is a broad question but can someone explain to me how to interpret these regression diagnostic plots? I understand the Normal Q-Q show's how normal the spread of the data is, but the ...
2
votes
1answer
1k views

Remaining heteroskedasticity even after GARCH estimation

This is according to the Ljung-Box $Q$ statistic of residuals squared and ARCH-LM test. Both suggest there are ARCH effects remaining after lag 1 even after I have estimated my GARCH (1,1) model. I ...
3
votes
1answer
313 views

Iterative outlier diagnostic

I am working on outlier diagnostics and I have a question about the best way to conduct them. Irrespective of the way used to define an outlier (i.e., statistical indexes, threshold), some of my ...
3
votes
1answer
2k views

Diagonal lines in residuals vs fitted values plot for ANOVA

I'm experiencing strange patterns of residuals. The following chart is a scatterplot of Standard residuals (Sres) versus Fits. I'm interested in the diagonal lines that mean that a higher fit leads to ...
2
votes
1answer
2k views

Poisson regression residuals diagnostic

I have done a poisson regression on my data set and am now looking to investigate the model fit. I notice that the fitted values from predict() in r give me the pre exp transformed values. For ...
0
votes
2answers
250 views

If the model fits well, nothing can be done?

I am playing a data without any background information. First, I try multiple linear regression. The model fits well, since the $r^2$ is larger than 90%. I deleted several variables by AIC. The fits ...
11
votes
2answers
2k views

Possible extensions to the default diagnostic plots for lm (in R and in general)?

I started digging a bit into the plot.lm function, this function gives six plots for lm, they are: a plot of residuals against fitted values a Scale-Location plot of sqrt(| residuals |) against ...
25
votes
1answer
3k views

What diagnostic plots exists for quantile regression?

Following on my question for OLS, I wonder: what diagnostic plots exists for quantile regression? (and are there R implementation of them?) A quick google search already came up with the worm plot (...
7
votes
1answer
9k views

How to validate & diagnose a gamma GLM in R?

I am fitting a generalized linear model in R with the log link and I need to validate and diagnose my model. I have never worked with the GLM in the past. Is there an article or any references I can ...
11
votes
1answer
12k views

How to interpret TBATS model results and model diagnostics

I have got a half hourly demand data, which is a multi-seasonal time series. I used tbats in forecast package in R, and got ...
6
votes
3answers
1k views

In linear regression, is the $R^2$ value enough to assess whether the relationship between the independent and dependent variable is linear?

In linear regression, is the $R^2$ value enough to assess whether the relationship between the independent and dependent variable is linear? It gives the amount of variability in the dependent ...
5
votes
4answers
1k views

Communicating Regression Model Results

I am concerned about how unequipped most people are (both within and without academia) to properly employ standard model building methods such as linear regression and to interpret the results of ...
5
votes
0answers
2k views

Diagnostics for GEE in R

I have been checking out which diagnostics to use for a GEE analysis. It seem that influence measures are appropriate (Preisser, 1996). Does anyone know of a package that can be used in R to examine ...
9
votes
0answers
251 views

When/why not to use studentized residuals for regression diagnostics?

Consider a linear regression $$ y=X\beta+\varepsilon. $$ Residuals $e:=y-X\hat\beta$ are often used as substitutes for the unobserved model errors $\varepsilon$ for validating assumptions such as ...
7
votes
1answer
2k views

What diagnostics for random effects logistic regression?

I'm wondering what checks/diagnostics you usually calculate and report if you do a random effects logistic regression. C-statistic/ROC curve Check multicollinearity? Check heteroscedasticity? ...
4
votes
1answer
180 views

Using splines to address non-linearity in logistic regression

I was wondering if the following is a reasonable way to proceed: I have a number of logistic models, fitted using glm, that I want to use to make predictions. The ...
3
votes
1answer
5k views

GARCH diagnostics: autocorrelation in standardized residuals but not in their squares

Fitting an ARMA-GARCH model, I checked the Weighted Ljung-Box test on standardized residuals and squared residuals to verify if the model is adeguate in describing the linear dependence in the return ...
10
votes
7answers
1k views

Is sensitivity or specificity a function of prevalence?

Standard teaching says that sensitivity and specificity are properties of the test and are independent of prevalence. But isn't this just an assumption? Harrison's principles of internal medicine ...
6
votes
2answers
6k views

Is residuals autocorrelation always a problem?

I read that OLS underestimates variance when residuals are autocorrelated. I see why autocorrelation would be a problem in time series analysis, in the sense that the coefficient are not efficient ...
6
votes
1answer
2k views

Interpreting case influence statistics (leverage, studentized residuals, and Cook's distance)

I just wanted to clarify some things about leverage, studentized residuals, and Cook's distance: Does a large (in absolute value) studentized residual mean that a case is an outlier? Does a large ...
14
votes
3answers
1k views

Is it valid to analyze signal detection data without employing metrics derived from signal detection theory?

A signal detection experiment typically presents the observer (or diagnostic system) with either a signal or a non-signal, and the observer is asked to report whether they think the presented item is ...
4
votes
1answer
1k views

Beta regression and regression diagnostics. Do we need to check for normality and other diagnostics?

I have a dependent variable which is a ratio and 0 < y < 1 condition holds. I will apply betareg in Stata but I am not sure what are the diagnostics that are ...
3
votes
1answer
10k views

R lmer Model Diagnosis qqnorm

I fitted this lmer model: m1 <- lmer(logR ~ N_g.m.2 * Year + (1|Wh/N_g.m.2), data = CO2_Ratio) Rendering the attached qqplot. ...
2
votes
1answer
399 views

Overdispersion tests from DHARMa and sjstats: conflicting results?

I ran some models for my count data, and did some diagnostics to check for overdispersion. Here is a dharma graph, which as I understand, indicates no overdispersion. And this is the result I get ...
5
votes
2answers
1k views

Which is the best method for meta-analysis of diagnostic test accuracy studies?

I am conducting a meta-analysis of diagnostic test accuracy studies focusing on myocardial perfusion imaging. I have used first Meta-Disc, but only for descriptive purposes, as it is clear that ...
4
votes
1answer
820 views

Interpret if residuals are “close enough” to a normal distribution

I'm working in Python with statsmodels. I estimate a multiple regression model (n=10763; 12 predictors; r^2=0.216; all coefficients have signs pointing the correct direction and are significant). Then ...
4
votes
1answer
4k views

Explanation of R diagnostic plot for logistic regression

I'm hoping someone can explain this bit of R code for me related to glm(). I don't understand the diagnostic plot that has been suggested. It seems a more ...
3
votes
1answer
619 views

Functions for regression diagnostics on mer objects in R

I'm particularly interested in plotting residuals against fitted values, and residuals against predictors. Often times I need to make boxplots of the residuals conditional on predictors. I'd be ...
3
votes
1answer
451 views

Confidence interval of AUC with Reitsma model in mada R package

I am conducting a meta-analysis of diagnostic test accuracy studies comparing myocardial perfusion scintigraphy vs coronary angiography using the mada R package. I ...
3
votes
1answer
541 views

Posterior predictive checks and the real world

In general, when performing posterior predictive checks, one calculates a posterior predictive p-value like so: $$p_B = \frac{1}{S}\sum_{s=1}^{S}\mathbb{1} (T(x^{(rep,s)},\theta^{s}) \ge T(x,\theta^{s}...
2
votes
1answer
104 views

If diagnostics for multiple linear regression are ok, are diagnostics of the component variables needed?

This is a follow-on question from here. I received two conflicting answers to the question posed in the title of this post. The diagnostics of the multiple regression looked okay (see link), but it ...