Questions tagged [diagnostic]

Diagnostic measures (such as residuals or some summary statistics calculated from residuals) are used to evaluate some aspect of quality of model fit to data.

Filter by
Sorted by
Tagged with
46
votes
2answers
98k views

How to read Cook's distance plots?

Does anyone know how to work out whether points 7, 16 and 29 are influential points or not? I read somewhere that because Cook's distance is lower than 1, they are not. Am, I right?
36
votes
3answers
29k views

Interpreting residual diagnostic plots for glm models?

I am looking for guidelines on how to interpret residual plots of glm models. Especially poisson, negative binomial, binomial models. What can we expect from these plots when the models are "correct"...
30
votes
5answers
4k views

Why does my bootstrap interval have terrible coverage?

I wanted to do a class demonstration where I compare a t-interval to a bootstrap interval and calculate the coverage probability of both. I wanted the data to come from a skewed distribution so I ...
29
votes
2answers
154k views

Likelihood ratio test in R

Suppose I am going to do a univariate logistic regression on several independent variables, like this: ...
25
votes
1answer
4k views

What diagnostic plots exists for quantile regression?

Following on my question for OLS, I wonder: what diagnostic plots exists for quantile regression? (and are there R implementation of them?) A quick google search already came up with the worm plot (...
17
votes
3answers
6k views

Is my model any good, based on the diagnostic metric ($R^2$/ AUC/ accuracy/ RMSE etc.) value?

I've fitted my model and am trying to understand whether it's any good. I've calculated the recommended metrics to assess it ($R^2$/ AUC / accuracy / prediction error / etc) but do not know how to ...
14
votes
3answers
1k views

Is it valid to analyze signal detection data without employing metrics derived from signal detection theory?

A signal detection experiment typically presents the observer (or diagnostic system) with either a signal or a non-signal, and the observer is asked to report whether they think the presented item is ...
14
votes
2answers
4k views

MCMC Geweke diagnostic

I'm running a Metropolis sampler (C++) and want to use the previous samples to estimate the convergence rate. One easy to implement diagnostic I found is the Geweke diagnostic, which computes the ...
12
votes
4answers
33k views

Is there a test for omitted variable bias in OLS?

I am aware of the Ramsey Reset test which may detect nonlinear dependencies. However, if you just throw out one of the regression coefficients (merely linear dependencies), you may get a bias, ...
12
votes
1answer
532 views

Why are diagnostics based on residuals?

In simple linear regression one often wants to verify if certain assumptions are met to be able to do inference (e.g. residuals are normally distributed). Is it reasonable to check the assumptions ...
12
votes
1answer
3k views

Prove the relation between Mahalanobis distance and Leverage?

I have seen formulas on Wikipedia. that relate Mahalanobis distance and Leverage: Mahalanobis distance is closely related to the leverage statistic, $h$, but has a different scale: $$D^2 = (N - 1)(...
11
votes
4answers
10k views

Diagonal straight lines in residuals vs fitted values plot for multiple regression

I am observing strange patterns in residuals for my data: [EDIT] Here are the partial regression plots for the two variables: [EDIT2] Added the PP Plot The distribution seems to be doing fine (see ...
11
votes
2answers
2k views

Possible extensions to the default diagnostic plots for lm (in R and in general)?

I started digging a bit into the plot.lm function, this function gives six plots for lm, they are: a plot of residuals against fitted values a Scale-Location plot of sqrt(| residuals |) against ...
11
votes
1answer
13k views

How to interpret TBATS model results and model diagnostics

I have got a half hourly demand data, which is a multi-seasonal time series. I used tbats in forecast package in R, and got ...
11
votes
2answers
2k views

What are some good exploratory analysis and diagnostic plots for count data? [duplicate]

Does anyone know of good reference material on exploratory analysis and diagnostic plots for count data?
11
votes
2answers
9k views

Residual Diagnostics and Homogeneity of variances in linear mixed model

Before asking this question, I did search our site and found a lot of similar questions, (like here, here, and here). But I feel those related questions were not well responded or discussed, thus ...
10
votes
7answers
1k views

Is sensitivity or specificity a function of prevalence?

Standard teaching says that sensitivity and specificity are properties of the test and are independent of prevalence. But isn't this just an assumption? Harrison's principles of internal medicine ...
10
votes
3answers
9k views

Residuals for logistic regression and Cook's distance

Are there any particular assumptions regarding the errors for logistic regression such as the constant variance of the error terms and the normality of the residuals? Also typically when you have ...
10
votes
1answer
1k views

What type of post-fit analysis of residuals do you use?

When carrying out OLS multiple linear regression, rather than plot the residuals against fitted values, I plot the (internal) Studentized residuals against fitted values (ditto for covariates). These ...
9
votes
1answer
7k views

How to manually calculate dfbetas

I am trying to replicate what the function dfbetas() does in R. dfbeta() is not an issue... Here is a set of vectors: ...
9
votes
1answer
1k views

What is gained from a scale-location plot?

The plot function in R provides four diagnostic plots for linear regression: It seems like the residuals vs fitted plot and the scale-location plot are basically ...
9
votes
0answers
264 views

When/why not to use studentized residuals for regression diagnostics?

Consider a linear regression $$ y=X\beta+\varepsilon. $$ Residuals $e:=y-X\hat\beta$ are often used as substitutes for the unobserved model errors $\varepsilon$ for validating assumptions such as ...
8
votes
3answers
2k views

How should I interpret this residual plot?

I am unable to interpret this graph. My dependent variable is total number of movie tickets that will be sold for a show. The independent variables are the number of days left before the show, ...
8
votes
3answers
2k views

Is there formal test of non-linearity in linear regression?

In logistic regression there is Box-Tidwell but I know of nothing like that in linear regression. I use partial residual plots to look for this, a graphical feature, but would love to find a formal ...
8
votes
1answer
5k views

How to identify outliers and do model diagnostics for an lme4 model?

I need to identify outliers and high leverage points, and perform model diagnostics, in an lme4 model. For outliers and high leverage points, simply making a plot ...
8
votes
1answer
5k views

How to interpret the direction of the Harvey-Collier test and Rainbow test for linearity?

I implemented both those tests with R, using the lmtest package. Both tests directionally say the same thing (I think) with a very similar p-value of very close to 0. But, are those tests saying ...
8
votes
2answers
2k views

How to combine the results of several binary tests?

First off let me say that I had one stats course in engineering school 38 years ago. So I'm flying blind here. I've got the results of what are essentially 18 separate diagnostic tests for a disease. ...
8
votes
1answer
150 views

Is there any meaningfully robust approach to conduct a network meta-analysis of diagnostic test accuracy studies?

Background: I am working on a systematic review including several imaging modalities for coronary artery disease, but the evidence network is quite large, including different modalities, often ...
8
votes
0answers
342 views

“Brute force” expected deviance for logistic regression?

A commonly used goodness of fit statistic for logistic regression is the deviance. This is also known as the likelihood ratio chi-square statistic. It is defined as: $$D=\sum_{i=1}^{N}d_i^2$$ $$d_i^...
7
votes
3answers
2k views

What does this plot tell me about my linear model?

I have fit the following linear model, I tested the response by looking at a qq plot and it is almost perfectly linear. When i fit the model though, and study the predicted vs observed plot, It looks ...
7
votes
1answer
30k views

How to interpret model diagnostics when doing linear regression in R?

I ran lm() on my data with models selected by individual lm's of each characteristic and then combined the top $R^2$ based on $p$...
7
votes
1answer
19k views

Logistic regression diagnostic plots in R

For testing purposes I made up some correlated data in R like this: ...
7
votes
1answer
10k views

How to calculate the confidence intervals for likelihood ratios from a 2x2 table in the presence of cells with zeroes

I am analysing a diagnostic test (against a gold standard, using a 2x2 table). I want to calculate likelihood ratios (sensitivity / (1-specificity) etc) however I have several sets of data with 0 ...
7
votes
1answer
9k views

How to validate & diagnose a gamma GLM in R?

I am fitting a generalized linear model in R with the log link and I need to validate and diagnose my model. I have never worked with the GLM in the past. Is there an article or any references I can ...
7
votes
1answer
431 views

Are normally distributed residuals not necessarily homoskedastic?

Let's say I've ran a linear regression and I'm checking the model diagnostics. I made a histogram of the residuals and they appear more or less normally distributed as below. I thought for a long ...
7
votes
1answer
2k views

What diagnostics for random effects logistic regression?

I'm wondering what checks/diagnostics you usually calculate and report if you do a random effects logistic regression. C-statistic/ROC curve Check multicollinearity? Check heteroscedasticity? ...
6
votes
3answers
1k views

In linear regression, is the $R^2$ value enough to assess whether the relationship between the independent and dependent variable is linear?

In linear regression, is the $R^2$ value enough to assess whether the relationship between the independent and dependent variable is linear? It gives the amount of variability in the dependent ...
6
votes
2answers
6k views

does rstandard standardize in z?

I'm new to R, so please be gentle. I was under the impression that rstandard(model) returns the z-scores of the residuals in ...
6
votes
2answers
18k views

Standardized residuals in R's lm output

If I plot the diagnostic plots to an R regression, a couple of them have "Standardized Residuals" as their y-axis such as in this plot: What are the residuals standardized over? That is, let us ...
6
votes
1answer
2k views

Interpreting case influence statistics (leverage, studentized residuals, and Cook's distance)

I just wanted to clarify some things about leverage, studentized residuals, and Cook's distance: Does a large (in absolute value) studentized residual mean that a case is an outlier? Does a large ...
6
votes
2answers
6k views

Is residuals autocorrelation always a problem?

I read that OLS underestimates variance when residuals are autocorrelated. I see why autocorrelation would be a problem in time series analysis, in the sense that the coefficient are not efficient ...
6
votes
4answers
1k views

Communicating Regression Model Results

I am concerned about how unequipped most people are (both within and without academia) to properly employ standard model building methods such as linear regression and to interpret the results of ...
6
votes
1answer
3k views

Negative Binomial Regression and Heteroskedasticity

What relationship does a negative binomial regression have to heteroskedasticity and if one still needs to check and/or correct for it how would this be done?
6
votes
1answer
4k views

Please help me refine this zero-inflated negative binomial model

I have been working on a baseball model to predict success at the major league level using minor league statistics. After posting multiple threads on this site (1, 2, 3) and receiving valuable ...
5
votes
1answer
2k views

Interpreting glm.diag.plots

Interpretation of GLM results is notoriously tricky. I'm new to GLM and have stumbled on the glm.diag.plots function from the boot package in R, which promises to make things easier. There's little ...
5
votes
1answer
142 views

When is it justified to “peek” at the outcome variable in model-building process?

I am referring to the following comment made in a 1996 paper by Dr Frank Harrell et al in Statistics in Medicine: Unless a formal penalized estimation technique is used, multiple comparisons ...
5
votes
3answers
966 views

Troublesome residual plot from linear mixed model

I have fitted the following linear mixed model based on the results of an economic game: lmer(TotalScore~perOOgivenP+Game+(1|Subject),REML=T,data=mdl1table)->m1 ...
5
votes
2answers
1k views

Diagnostics for General Linear Models

Pearson residuals follow normal distribution. We plot them against predicted values to see if the model is good. Why would we plot deviance residuals against predicted values? Deviance residuals don'...
5
votes
1answer
1k views

Is the goodness of fit test in JMP the Hosmer-Lemeshow goodness of fit test?

I'm working with an organization that is using JMP in their analysis, and I can't tell from the description in JMP's help files if the test for goodness of fit in their logistic regression is the ...
5
votes
1answer
2k views

What resolution should I be using for residuals vs fitted values plot from a linear regression?

I made this linear regression that shows how well estimated animal locations (longitude) predict actual animal locations. ...

1
2 3 4 5 6