Diagnostic measures (such as residuals or some summary statistics calculated from residuals) are used to evaluate some aspect of quality of model fit to data.

learn more… | top users | synonyms (1)

1
vote
1answer
78 views

What does this plot tell me about my linear model?

I have fit the following linear model, I tested the response by looking at a qq plot and it is almost perfectly linear. When i fit the model though, and study the predicted vs observed plot, It looks ...
0
votes
1answer
270 views

GARCH model diagnostics: how to interpret test results?

I'm new here and also new in Ox environment. Those below are results I obtained from estimating AR(5)-FIGARCH(1,$d$,1) models. ...
3
votes
0answers
65 views

Verification of assumptions in TBATS model

I have a question about using BATS/TBATS models implemented in the forecast package for R. In De Liv­era, Hyndman & Snyder (2011) the models are used without any following analysis. Is it OK to ...
1
vote
1answer
15 views

Meta-analysis of multiple diagnostic tests accuracies

I want to perform a meta-analysis of multiple diagnostic test at once. Outcomes are sensitivity, specificity, positive predictive value and negative predictive value. All the test have the same ...
1
vote
1answer
62 views

How to interpret TBATS model results and model diagnostics

I have got a half hourly demand data, which is a multiseaonal time series. I used tbats in forecast package in R, and got ...
0
votes
1answer
141 views

Statistical tests for comparing Positive/Negative likelihood ratios in two independent diagnsotic tests

I have a problem regarding the comparison of the likelihood ratios in two diagnostic tests. The definition of positive likelihood ratio in screening tests is "the probability of a person who has the ...
2
votes
0answers
31 views

Is there any meaningfully robust approach to conduct a network meta-analysis of diagnostic test accuracy studies?

I am working on a systematic review including several imaging modalities for coronary artery disease, but the evidence network is quite large, including different modalities, often compared one to the ...
1
vote
0answers
10 views

MCMC diagnostics for EMC or SMC

Are diagnostics developed for MCMC (e.g. Gelman-Rubin, Geweke) suitable for output from Evolutionary Monte Carlo (EMC) or Sequential Monte Carlo (SMC)?
1
vote
0answers
38 views

How to interpret regression diagnostic plots for multiple linear regression analysis (with specific example)

I recently using the day.csv file which is downloaded from http://archive.ics.uci.edu/ml/machine-learning-databases/00275/ to build a regression model for the last column “cnt” in R. This is the ...
1
vote
1answer
79 views

Estimate ROC curve using binormal distribuiton

I am conducting a meta-analysis on diagnostic studies but for each study I have only mean and standard deviation reported. How can I estimate the ROC curve using the binormal assumption in R ? Tks,
1
vote
0answers
25 views

Diagnostic device hypothesis testing for superiority to a fixed value

I am designing a study to demonstrate that a diagnostic medical device is more sensitive than a fixed performance goal of 70%. Currently I am attempting to determine the appropriate sample size. My ...
9
votes
2answers
531 views

What are some good exploratory analysis and diagnostic plots for count data?

Does anyone know of good reference material on exploratory analysis and diagnostic plots for count data?
1
vote
1answer
33 views

Diagnostics for generalized additive model vs linear model

I am doing an analysis in R and I have the model: lm(birthrate~education+employment+lo(latitude, longitude),data=data2). It seems to be a generalized additive ...
1
vote
0answers
27 views

How to interpret diagnostic plots for GEV distribution

I produced the following Diagnostic plots, from the following R Code: ...
1
vote
1answer
55 views

Are these diagnostic plots from lmer too far away from normal and showing heteroscedasticity?

I have read similar posts in this website to help me assess whether my diagnostic plots are too far away from normal and if they are showing heteroscedasticity (Interpretation of residuals vs fitted ...
4
votes
0answers
78 views

what to do with ridiculous but valid leverage points

So I'm having some difficulty fitting a linear model to the data (see other post here glm model fit - can't find a family/link combination that produces good fit). In particular, I'm worried ...
7
votes
1answer
231 views

Prove the relation between Mahalanobis distance and Leverage?

I have seen formulas on Wikipedia. that relate Mahalanobis distance and Leverage: Mahalanobis distance is closely related to the leverage statistic, $h$, but has a different scale: $$D^2 = (N - 1)(...
3
votes
1answer
373 views

Interpreting case influence statistics (leverage, studentized residuals, and Cook's distance)

I just wanted to clarify some things about leverage, studentized residuals, and Cook's distance: Does a large (in absolute value) studentized residual mean that a case is an outlier? Does a large ...
3
votes
1answer
84 views

Mixed model or ANOVA on differences in pre-post design

I want to analyse the effect of different treatment types (control, treatment1, ..., treatment4) on the surface of specimens made of certain materials (...
0
votes
0answers
16 views

Autocorrelation of MAPE in case of lack of autocorrelation in residuals?

I'm building a forecasting tool for electricity market (spot power market). The model forecasts spot prices of power for each of 24 hours of next day. I measure its performance for whole days (for 24 ...
1
vote
0answers
25 views

Standardized residuals vs fitted values for Poisson regression

McCullagh and Nelder's book on glm suggest to plot standardized deviance residuals against either the linear predictor ($\hat{\eta}$) or the fitted values ($\hat{\mu}$) transformed to the constant ...
4
votes
2answers
122 views

Which is the best method for meta-analysis of diagnostic test accuracy studies?

I am conducting a meta-analysis of diagnostic test accuracy studies focusing on myocardial perfusion imaging. I have used first Meta-Disc, but only for descriptive purposes, as it is clear that ...
0
votes
0answers
25 views

Lag length for the Ljung-Box test

I have an ARIMA model applied on hourly data: Arima.fit2 <- Arima(tsTrain, order=c(17,1,0)) The length of my training set is 60 hours. In the end I plan to ...
3
votes
1answer
55 views

Confidence interval of AUC with Reitsma model in mada R package

I am conducting a meta-analysis of diagnostic test accuracy studies comparing myocardial perfusion scintigraphy vs coronary angiography using the mada R package. I ...
2
votes
0answers
23 views

When creating a multiple regression model for a subgroup, is it necessary to test all assumptions again?

My results section consists of a multiple regression analysis considering 3 factors, containing all of my participants. Following this, I have considered males and females separately, by construction ...
1
vote
1answer
176 views

GARCH diagnostics: autocorrelation in standardized residuals but not in their squares

Fitting an ARMA-GARCH model, I checked the Weighted Ljung-Box test on standardized residuals and squared residuals to verify if the model is adeguate in describing the linear dependence in the return ...
2
votes
0answers
35 views

How should I interpret/follow-up on mixed logistic regression (GLMM) diagnostics?

I have experimental data (n subjects = 64) in which the response variable, accuracy (0 or 1), was measured 9 times within subjects. My predictor is Condition (A vs B) measured between subjects. I ...
0
votes
0answers
17 views

Assessing the residual independence assumption (nonlinear least squares regression diagnostics)

I would like to assess the assumptions underlying nonlinear regression models using statistical tests rather than graphical methods since I have thousands of fitting results. I am not certain ...
1
vote
0answers
18 views

How to decide between quasi-poisson and negative binomial?

I tried quasi-poisson and negative binomial glms on my counted data in R. The estimates are pretty much the same but p-value are different. Quasi-poisson gives insignificant result. NB give ...
0
votes
0answers
13 views

Same error on all sets

My concern is the classification of data splitted into training, cross validation and test sets. By evaluating my model, the values of precision, recall, f-score and auroc are nearly the same for all ...
21
votes
2answers
36k views

How to read Cook's distance plots?

Does anyone know how to work out whether points 7, 16 and 29 are influential points or not? I read somewhere that because Cook's distance is lower than 1, they are not. Am, I right?
2
votes
1answer
53 views

Posterior predictive checks and the real world

In general, when performing posterior predictive checks, one calculates a posterior predictive p-value like so: $$p_B = \frac{1}{S}\sum_{s=1}^{S}\mathbb{1} (T(x^{(rep,s)},\theta^{s}) \ge T(x,\theta^{s}...
1
vote
1answer
122 views

How to identify outliers and do model diagnostics for an lme4 model?

I need to identify outliers and high leverage points, and perform model diagnostics, in an lme4 model. For outliers and high leverage points, simply making a plot ...
3
votes
1answer
68 views

Tilted rectangle in residuals vs fitted plot

When performing diagnostics on an OLS model, what can make a plot of the predicted responses vs the residuals. Ideally we want a horizontal rectangle shape. But what does it mean when the plot is a ...
16
votes
2answers
47k views

Likelihood ratio test in R

Suppose I am going to do a univariate logistic regression on several independent variables, like this: ...
1
vote
0answers
30 views

GLM Diganostic Analysis - What plots actually matter?

In regards to this Poisson approach (How can the Poisson GLM be used instead of logistic regression for the Titanic survival data?). I always thought that Normal Q-Q, Cook's D distance plots were ...
0
votes
0answers
14 views

Diagnostic measure not influenced by transformations of response variable

What are the diagnostic measures (like Cook's distance, H matrix, DFFITS, DFBETAS) that are not affected by the transformation of the response/dependent variable? And why?
0
votes
0answers
31 views

Logit model simulation - binned residual plot seems to exhibit bias

To understand how to perform diagnostics, I've been experimenting simulating logit models. The binned residual plot makes sense to me as a simple way of analysing whether residuals are distributed as ...
1
vote
1answer
76 views

Evaluation of diagnostic test ($R^2$ vs AUC)

A sort of a risk evaluation system is offered to us (sort of advertising). The output value is lethality (yes/no). The evaluators are several diagnoses, emergency status, age etc. No detailed ...
0
votes
0answers
58 views

Positive predictive value versus false positive rate

A colleague of mine has quoted to me the following: For a given population with a known specificity and a known event rate where a positive test is thought to predict the presence of an event: ...
1
vote
1answer
22 views

Is Box and Pierce test on residual or squared residual?

In a GARCH model, is it better to test the autocorrelation of squared residuals rather than the autocorrelation of residuals? Thanks!
5
votes
1answer
2k views

How to manually calculate dfbetas

I am trying to replicate what the function dfbetas() does in R. dfbeta() is not an issue... Here is a set of vectors: ...
1
vote
1answer
85 views

Diagnostics for quasipoisson glm for continuous data

I'm a little confused about how to use the quasipoisson family in the glm function. It was recommended by someone that I use it for my analysis, even though the data are continuous - and as such, I ...
1
vote
0answers
64 views

Why is my logistic regression fit that bad?

I was requested by a professor to analyze a high dimensional dataset (92 cases, 400+ variables, and a lot of NAs...) of various patient outcomes (gaussian distributed, counts, and binary) and their ...
6
votes
3answers
12k views

Is there a test for omitted variable bias in OLS?

I am aware of the Ramsey Reset test which may detect nonlinear dependencies. However, if you just throw out one of the regression coefficients (merely linear dependencies), you may get a bias, ...
8
votes
2answers
2k views

Residual Diagnostics and Homogeneity of variances in linear mixed model

Before asking this question, I did search our site and found a lot of similar questions, (like here, here, and here). But I feel those related questions were not well responded or discussed, thus ...
0
votes
0answers
27 views

Logistic Regression Diagnostics

I am a bit confused how to assess the results of a logistic regression procedure I ran. Using SAS (on a 60% training set) - The LRT, Score and Wald test and got very low p-values. However, this model ...
2
votes
0answers
41 views

Problems in finding outliers and leverage points in non-linear regression

I'm implementing diagnostic of non-linear regression model $(y=ax^b)$. I'm trying to find out where outliers and leverage points in my model ...
0
votes
1answer
15 views

How can model misfit introduce dependency?

I read somewhere recently that mis-fitting a regression model can introduce dependency into the model. Unfortunately I cannot find the where I read this. Can anyone explain how mis-fitting a ...
0
votes
0answers
153 views

ARIMA diagnostic testing in Stata

I am using an ARIMA(1,1,0) in Stata. I have already executed estat aroots and wntestq (white noise test) for the residuals. I ...