Diagnostic measures (such as residuals or some summary statistics calculated from residuals) are used to evaluate some aspect of quality of model fit to data.

learn more… | top users | synonyms (1)

21
votes
4answers
759 views

Why does my bootstrap interval have terrible coverage?

I wanted to do a class demonstration where I compare a t-interval to a bootstrap interval and calculate the coverage probability of both. I wanted the data to come from a skewed distribution so I ...
18
votes
2answers
22k views

How to read Cook's distance plots?

Does anyone know how to work out whether points 7, 16 and 29 are influential points or not? I read somewhere that because Cook's distance is lower than 1, they are not. Am, I right?
18
votes
2answers
10k views

Interpreting residual diagnostic plots for glm models?

I am looking for guidelines on how to interpret residual plots of glm models. Especially poisson, negative binomial, binomial models. What can we expect from these plots when the models are ...
14
votes
2answers
1k views

What diagnostic plots exists for quantile regression?

Following on my question for OLS, I wonder: what diagnostic plots exists for quantile regression? (and are there R implementation of them?) A quick google search already came up with the worm plot ...
13
votes
3answers
677 views

Is it valid to analyze signal detection data without employing metrics derived from signal detection theory?

A signal detection experiment typically presents the observer (or diagnostic system) with either a signal or a non-signal, and the observer is asked to report whether they think the presented item is ...
12
votes
2answers
30k views

Likelihood ratio test in R

I'm not sure if I'm asking something stupid or off topic here, but I can't think where can I ask this question. suppose I am going to do a univariate logistic regression on several independent ...
10
votes
1answer
364 views

Why are diagnostics based on residuals?

In simple linear regression one often wants to verify if certain assumptions are met to be able to do inference (e.g. residuals are normally distributed). Is it reasonable to check the assumptions ...
8
votes
2answers
374 views

What are some good exploratory analysis and diagnostic plots for count data?

Does anyone know of good reference material on exploratory analysis and diagnostic plots for count data?
8
votes
1answer
604 views

What type of post-fit analysis of residuals do you use?

When carrying out OLS multiple linear regression, rather than plot the residuals against fitted values, I plot the (internal) Studentized residuals against fitted values (ditto for covariates). These ...
7
votes
4answers
923 views

Diagonal lines in residuals vs fitted values plot for multiple regression

I am observing strange patterns in residuals for my data: [EDIT] Here are the partial regression plots for the two variables: [EDIT2] Added the PP Plot The distribution seems to be doing ...
7
votes
2answers
917 views

Possible extensions to the default diagnostic plots for lm (in R and in general)?

I started digging a bit into the plot.lm function, this function gives six plots for lm, they are: a plot of residuals against fitted values a Scale-Location plot of sqrt(| residuals |) against ...
7
votes
3answers
2k views

Residuals for logistic regression and Cook's distance

Are there any particular assumptions regarding the errors for logistic regression such as the constant variance of the error terms and the normality of the residuals? Also typically when you have ...
7
votes
2answers
825 views

MCMC Geweke diagnostic

I'm running a Metropolis sampler (C++) and want to use the previous samples to estimate the convergence rate. One easy to implement diagnostic I found is the Geweke diagnostic, which computes the ...
6
votes
2answers
623 views

How to combine the results of several binary tests?

First off let me say that I had one stats course in engineering school 38 years ago. So I'm flying blind here. I've got the results of what are essentially 18 separate diagnostic tests for a ...
5
votes
3answers
544 views

In linear regression, is the $R^2$ value enough to assess whether the relationship between the independent and dependent variable is linear?

In linear regression, is the $R^2$ value enough to assess whether the relationship between the independent and dependent variable is linear? It gives the amount of variability in the dependent ...
5
votes
1answer
157 views

Are normally distributed residuals not necessarily homoskedastic?

Let's say I've ran a linear regression and I'm checking the model diagnostics. I made a histogram of the residuals and they appear more or less normally distributed as below. I thought for a long ...
5
votes
1answer
254 views

Is the goodness of fit test in JMP the Hosmer-Lemeshow goodness of fit test?

I'm working with an organization that is using JMP in their analysis, and I can't tell from the description in JMP's help files if the test for goodness of fit in their logistic regression is the ...
5
votes
1answer
1k views

Please help me refine this zero-inflated negative binomial model

I have been working on a baseball model to predict success at the major league level using minor league statistics. After posting multiple threads on this site (1, 2, 3) and receiving valuable ...
4
votes
1answer
2k views

Explanation of R diagnostic plot for logistic regression

I'm hoping someone can explain this bit of R code for me related to glm(). I don't understand the diagnostic plot that has been suggested. It seems a more ...
4
votes
1answer
3k views

How to calculate the confidence intervals for likelihood ratios from a 2x2 table in the presence of cells with zeroes

I am analysing a diagnostic test (against a gold standard, using a 2x2 table). I want to calculate likelihood ratios (sensitivity / (1-specificity) etc) however I have several sets of data with 0 ...
4
votes
1answer
2k views

How to validate & diagnose a gamma GLM in R?

I am fitting a generalized linear model in R with the log link and I need to validate and diagnose my model. I have never worked with the GLM in the past. Is there an article or any references I ...
4
votes
1answer
68 views

Geweke diagnostic of a Markov chain: why does the first window have to cover the burn-in?

I read the following statement in this document$^{[1]}$, at the bottom of page 11: Too wide A will some times “hide” the burn in part within the converged part of the chain and the difference in ...
4
votes
1answer
434 views

What diagnostics for random effects logistic regression?

I'm wondering what checks/diagnostics you usually calculate and report if you do a random effects logistic regression. C-statistic/ROC curve Check multicollinearity? Check heteroscedasticity? ...
4
votes
1answer
2k views

Diagnostic plots for lmer

I am trying to produce a glmm using the lme4 package in R. To validate my model I would like to produce some diagnostic plots ...
4
votes
0answers
390 views

Diagnostics for GEE in R

I have been checking out which diagnostics to use for a GEE analysis. It seem that influence measures are appropriate (Preisser, 1996). Does anyone know of a package that can be used in R to examine ...
4
votes
0answers
3k views

Fitting a zero-inflated negative binomial regression with R

In this thread, I laid out a problem involving fitting a model that attempts to use minor league baseball statistics to predict success at the major league level (explained in full in the thread). ...
3
votes
1answer
14k views

How to interpret model diagnostics when doing linear regression in R?

I ran lm() on my data with models selected by individual lm's of each characteristic and then combined the top $R^2$ based on ...
3
votes
1answer
90 views

How to treat this OLS based on residual diagnostics

I am struggling already a couple of days with this simple OLS, can you help? Outcome years in function of predictor score, very simple linear model. The residual plot does absolutely not look good ...
3
votes
2answers
3k views

Standardized residuals in R's lm output

I have a quick question: if I plot the diagnostic plots to an R regression, a couple of them have "Standardized Residuals" as their y-axis such as in this plot: My question is this: what are the ...
3
votes
4answers
553 views

Communicating Regression Model Results

I am concerned about how unequipped most people are (both within and without academia) to properly employ standard model building methods such as linear regression and to interpret the results of ...
3
votes
1answer
48 views

How can we have non-random patterns in the plot of simple linear regression residuals vs the predictor variable?

A) When considering a simple linear regression model, it is important to check the linearity assumption. Graphing the residuals vs the predictor variable can often give a good idea of whether or not ...
3
votes
3answers
475 views

Troublesome residual plot from linear mixed model

I have fitted the following linear mixed model based on the results of an economic game: lmer(TotalScore~perOOgivenP+Game+(1|Subject),REML=T,data=mdl1table)->m1 ...
3
votes
1answer
105 views

What are the three forms of the Park test for heteroskedasticity?

I understand the Park test for heteroskedasticity has three different forms. The best known one is in a log form: LN(Residual^2) = intercept + slope (LN(X)). The second one is in a linear form: ...
3
votes
1answer
413 views

What resolution should I be using for residuals vs fitted values plot from a linear regression?

I made this linear regression that shows how well estimated animal locations (longitude) predict actual animal locations. ...
3
votes
3answers
89 views

Too many predictors to manually check linearity

Say I have 1000 potential predictors in a logistic regression. I don't have time to check each predictor manually for linearity. I could wait till after variable selection, but in that case I wonder ...
3
votes
1answer
166 views

How to interpret the direction of the Harvey-Collier test and Rainbow test for linearity?

I implemented both those tests with R, using the lmtest package. Both tests directionally say the same thing (I think) with a very similar p-value of very close to 0. But, are those tests saying ...
3
votes
0answers
59 views

Can I compare different estimations approaches with AIC?

I'm running two different panel models. Model 1 is a Random Effects regression estimated using Maximum Likelihood and bootstrapped standard errors. Model 2 has the same main dependent variables and ...
3
votes
0answers
286 views

How to compute sample size to compare two diagnostic tests

I will be performing two diagnostic tests (one is the gold standard, one is novel) on the same subject aiming to establish sensitivity, specificity, PPV and NPV. What formula may be used to compute ...
3
votes
0answers
241 views

“Brute force” expected deviance for logistic regression?

A commonly used goodness of fit statistic for logistic regression is the deviance. This is also known as the likelihood ratio chi-square statistic. It is defined as: $$D=\sum_{i=1}^{N}d_i^2$$ ...
2
votes
2answers
159 views

Are the model residuals well-behaved (homoscedasticity)?

Can I say looking at this residuals-vs-fitted plots, that my residuals are homoscedastic?
2
votes
2answers
676 views

does rstandard standardize in z?

I'm new to R, so please be gentle. I was under the impression that rstandard(model) returns the z-scores of the residuals in ...
2
votes
2answers
347 views

Is residuals autocorrelation always a problem?

I read that OLS underestimates variance when residuals are autocorrelated. I see why autocorrelation would be a problem in time series analysis, in the sense that the coefficient are not efficient ...
2
votes
1answer
196 views

Diagonal lines in residuals vs fitted values plot for ANOVA

I'm experiencing strange patterns of residuals. The following chart is a scatterplot of Standard residuals (Sres) versus Fits. I'm interested in the diagonal lines that mean that a higher fit leads to ...
2
votes
1answer
376 views

Logistic regression and complementary log log model

is there like a diagnostic checking(like those in the time series) done for logistic regression and complementary loglog model?
2
votes
1answer
119 views

Interpreting case influence statistics (leverage, studentized residuals, and Cook's distance)

I just wanted to clarify some things about leverage, studentized residuals, and Cook's distance: Does a large (in absolute value) studentized residual mean that a case is an outlier? Does a large ...
2
votes
1answer
107 views

Explanation of a step in derivation of residuals for R lm diagnostic?

I'm reading Faraway's book (http://cran.r-project.org/doc/contrib/Faraway-PRA.pdf) to try to understand R's lm diagnostic plots. On page 72 of the book is this: I have been trying to understand a ...
2
votes
1answer
496 views

Functions for regression diagnostics on mer objects in R

I'm particularly interested in plotting residuals against fitted values, and residuals against predictors. Often times I need to make boxplots of the residuals conditional on predictors. I'd be ...
2
votes
0answers
15 views

Independence of “residuals” in a Bayesian multilevel hierarchical model

So i'm having some problems realising what model checks I should do after fitting a bayesian model other than convergence diagnostics. Lets say i'm fitting a hierarchical bayesian regression model, I ...
2
votes
0answers
47 views

what to do with ridiculous but valid leverage points

So I'm having some difficulty fitting a linear model to the data (see other post here glm model fit - can't find a family/link combination that produces good fit). In particular, I'm worried ...
2
votes
0answers
28 views

How to proceed with nonstationary variables in panels?

In most of the emprical papers using panel data, authors do not seem to "worry" too much aboout the non-stationarity of the individual variables. Yes, there is asymptotic theory for N and T going to ...