Diagnostic measures (such as residuals or some summary statistics calculated from residuals) are used to evaluate some aspect of quality of model fit to data.

learn more… | top users | synonyms (1)

1
vote
0answers
25 views

Diagnostic device hypothesis testing for superiority to a fixed value

I am designing a study to demonstrate that a diagnostic medical device is more sensitive than a fixed performance goal of 70%. Currently I am attempting to determine the appropriate sample size. My ...
1
vote
1answer
23 views

Diagnostics for generalized additive model vs linear model

I am doing an analysis in R and I have the model: lm(birthrate~education+employment+lo(latitude, longitude),data=data2). It seems to be a generalized additive ...
1
vote
0answers
22 views

How to interpret diagnostic plots for GEV distribution

I produced the following Diagnostic plots, from the following R Code: ...
1
vote
1answer
43 views

Are these diagnostic plots from lmer too far away from normal and showing heteroscedasticity?

I have read similar posts in this website to help me assess whether my diagnostic plots are too far away from normal and if they are showing heteroscedasticity (Interpretation of residuals vs fitted ...
3
votes
1answer
71 views

Mixed model or ANOVA on differences in pre-post design

I want to analyse the effect of different treatment types (control, treatment1, ..., treatment4) on the surface of specimens made of certain materials (...
1
vote
0answers
22 views

Standardized residuals vs fitted values for Poisson regression

McCullagh and Nelder's book on glm suggest to plot standardized deviance residuals against either the linear predictor ($\hat{\eta}$) or the fitted values ($\hat{\mu}$) transformed to the constant ...
0
votes
0answers
21 views

Lag length for the Ljung-Box test

I have an ARIMA model applied on hourly data: Arima.fit2 <- Arima(tsTrain, order=c(17,1,0)) The length of my training set is 60 hours. In the end I plan to ...
3
votes
1answer
49 views

Confidence interval of AUC with Reitsma model in mada R package

I am conducting a meta-analysis of diagnostic test accuracy studies comparing myocardial perfusion scintigraphy vs coronary angiography using the mada R package. I ...
4
votes
2answers
95 views

Which is the best method for meta-analysis of diagnostic test accuracy studies?

I am conducting a meta-analysis of diagnostic test accuracy studies focusing on myocardial perfusion imaging. I have used first Meta-Disc, but only for descriptive purposes, as it is clear that ...
2
votes
0answers
17 views

When creating a multiple regression model for a subgroup, is it necessary to test all assumptions again?

My results section consists of a multiple regression analysis considering 3 factors, containing all of my participants. Following this, I have considered males and females separately, by construction ...
2
votes
0answers
25 views

How should I interpret/follow-up on mixed logistic regression (GLMM) diagnostics?

I have experimental data (n subjects = 64) in which the response variable, accuracy (0 or 1), was measured 9 times within subjects. My predictor is Condition (A vs B) measured between subjects. I ...
0
votes
1answer
84 views

GARCH diagnostics: autocorrelation in standardized residuals but not in their squares

Fitting an ARMA-GARCH model, I checked the Weighted Ljung-Box test on standardized residuals and squared residuals to verify if the model is adeguate in describing the linear dependence in the return ...
0
votes
0answers
16 views

Assessing the residual independence assumption (nonlinear least squares regression diagnostics)

I would like to assess the assumptions underlying nonlinear regression models using statistical tests rather than graphical methods since I have thousands of fitting results. I am not certain ...
1
vote
0answers
15 views

How to decide between quasi-poisson and negative binomial?

I tried quasi-poisson and negative binomial glms on my counted data in R. The estimates are pretty much the same but p-value are different. Quasi-poisson gives insignificant result. NB give ...
0
votes
1answer
159 views

GARCH model diagnostics: how to interpret test results?

I'm new here and also new in Ox environment. Those below are results I obtained from estimating AR(5)-FIGARCH(1,$d$,1) models. ...
0
votes
0answers
12 views

Same error on all sets

My concern is the classification of data splitted into training, cross validation and test sets. By evaluating my model, the values of precision, recall, f-score and auroc are nearly the same for all ...
7
votes
1answer
189 views

Prove the relation between Mahalanobis distance and Leverage?

I have seen formulas on Wikipedia. that relate Mahalanobis distance and Leverage: Mahalanobis distance is closely related to the leverage statistic, $h$, but has a different scale: $$D^2 = (N - ...
2
votes
1answer
47 views

Posterior predictive checks and the real world

In general, when performing posterior predictive checks, one calculates a posterior predictive p-value like so: $$p_B = \frac{1}{S}\sum_{s=1}^{S}\mathbb{1} (T(x^{(rep,s)},\theta^{s}) \ge ...
0
votes
1answer
76 views

How to identify outliers and do model diagnostics for an lme4 model?

I need to identify outliers and high leverage points, and perform model diagnostics, in an lme4 model. For outliers and high leverage points, simply making a plot ...
3
votes
1answer
57 views

Tilted rectangle in residuals vs fitted plot

When performing diagnostics on an OLS model, what can make a plot of the predicted responses vs the residuals. Ideally we want a horizontal rectangle shape. But what does it mean when the plot is a ...
1
vote
0answers
26 views

GLM Diganostic Analysis - What plots actually matter?

In regards to this Poisson approach (How can the Poisson GLM be used instead of logistic regression for the Titanic survival data?). I always thought that Normal Q-Q, Cook's D distance plots were ...
0
votes
0answers
13 views

Autocorrelation of MAPE in case of lack of autocorrelation in residuals?

I'm building a forecasting tool for electricity market (spot power market). The model forecasts spot prices of power for each of 24 hours of next day. I measure its performance for whole days (for 24 ...
0
votes
0answers
14 views

Diagnostic measure not influenced by transformations of response variable

What are the diagnostic measures (like Cook's distance, H matrix, DFFITS, DFBETAS) that are not affected by the transformation of the response/dependent variable? And why?
0
votes
0answers
24 views

Logit model simulation - binned residual plot seems to exhibit bias

To understand how to perform diagnostics, I've been experimenting simulating logit models. The binned residual plot makes sense to me as a simple way of analysing whether residuals are distributed as ...
1
vote
1answer
63 views

Evaluation of diagnostic test ($R^2$ vs AUC)

A sort of a risk evaluation system is offered to us (sort of advertising). The output value is lethality (yes/no). The evaluators are several diagnoses, emergency status, age etc. No detailed ...
0
votes
0answers
45 views

Positive predictive value versus false positive rate

A colleague of mine has quoted to me the following: For a given population with a known specificity and a known event rate where a positive test is thought to predict the presence of an event: ...
1
vote
1answer
21 views

Is Box and Pierce test on residual or squared residual?

In a GARCH model, is it better to test the autocorrelation of squared residuals rather than the autocorrelation of residuals? Thanks!
1
vote
0answers
58 views

Why is my logistic regression fit that bad?

I was requested by a professor to analyze a high dimensional dataset (92 cases, 400+ variables, and a lot of NAs...) of various patient outcomes (gaussian distributed, counts, and binary) and their ...
1
vote
1answer
68 views

Diagnostics for quasipoisson glm for continuous data

I'm a little confused about how to use the quasipoisson family in the glm function. It was recommended by someone that I use it for my analysis, even though the data are continuous - and as such, I ...
0
votes
0answers
26 views

Logistic Regression Diagnostics

I am a bit confused how to assess the results of a logistic regression procedure I ran. Using SAS (on a 60% training set) - The LRT, Score and Wald test and got very low p-values. However, this model ...
2
votes
0answers
35 views

Problems in finding outliers and leverage points in non-linear regression

I'm implementing diagnostic of non-linear regression model $(y=ax^b)$. I'm trying to find out where outliers and leverage points in my model ...
0
votes
1answer
15 views

How can model misfit introduce dependency?

I read somewhere recently that mis-fitting a regression model can introduce dependency into the model. Unfortunately I cannot find the where I read this. Can anyone explain how mis-fitting a ...
0
votes
0answers
132 views

ARIMA diagnostic testing in Stata

I am using an ARIMA(1,1,0) in Stata. I have already executed estat aroots and wntestq (white noise test) for the residuals. I ...
1
vote
0answers
18 views

Diagnostic plots for logistic regression [duplicate]

I want to check for the assumptions of linearity in my logistic regression model, which states that the logit of the probability is a linear combination of predictors. It might be the case though, ...
0
votes
1answer
57 views

Zero-inflated models how to get predicted values = 0 stata

I'm fairly new at Stata and this is the main reason of my question. I did a longitudinal zero-inflated poisson model: ...
0
votes
0answers
21 views

Does it make sense to standardize residual using $SSE/(n-1)$?

See[Dean and Voss]Design and Analysis of Experiments,1999. pp.105 The authors proposed that we standardize residuals using: $$z_{it}=\frac{\hat{e_{it}}}{\sqrt{ssE/(n-1)}}$$ where ...
1
vote
0answers
66 views

Using Half-Normal Plot to Identify Outlying Observations

When we perform linear regression diagnostic, we usually utilize the half-normal plot (available in the faraway package in R) to identify outlying observations. ...
4
votes
1answer
88 views

When is it justified to “peek” at the outcome variable in model-building process?

I am referring to the following comment made in a 1996 paper by Dr Frank Harrell et al in Statistics in Medicine: Unless a formal penalized estimation technique is used, multiple comparisons ...
0
votes
0answers
75 views

Tukey Test in Linear Regression Model Development

I am not able to interpret properly the Tukey Test results from a linear regression model i have built. The Tukey Test for the model i believe is to test is the model is linear. Please correct me if i ...
0
votes
0answers
29 views

Externally studentized residuals

Assume the our model is simply $$Y = X\beta + \epsilon$$ with $\epsilon$~ $N_n(0,{\sigma}^2I)$. If we want to identify the existing outlier in our dataset, we use the externally studentized ...
0
votes
1answer
107 views

Statistical tests for comparing Positive/Negative likelihood ratios in two independent diagnsotic tests

I have a problem regarding the comparison of the likelihood ratios in two diagnostic tests. The definition of positive likelihood ratio in screening tests is "the probability of a person who has the ...
0
votes
1answer
58 views

Estimate ROC curve using binormal distribuiton

I am conducting a meta-analysis on diagnostic studies but for each study I have only mean and standard deviation reported. How can I estimate the ROC curve using the binormal assumption in R ? Tks,
3
votes
0answers
102 views

Can I compare different estimations approaches with AIC?

I'm running two different panel models. Model 1 is a Random Effects regression estimated using Maximum Likelihood and bootstrapped standard errors. Model 2 has the same main dependent variables and ...
2
votes
0answers
34 views

Independence of “residuals” in a Bayesian multilevel hierarchical model

So i'm having some problems realising what model checks I should do after fitting a bayesian model other than convergence diagnostics. Lets say i'm fitting a hierarchical bayesian regression model, I ...
0
votes
0answers
75 views

Practical beginners resource for building a dynamic OLS model

I need to model the current account balance of a country. The regressors are the real effective exchange rate, the domestic GDP and the GDP of the world. I am using data for 30 years (in logs). It is ...
1
vote
2answers
61 views

What visualizations do people use to debug a machine learning model?

Imaging I am refining a model. What visualization can I rely on to help me identify an error, e.g. feature deficiency, data bias? A few graphs I am aware of include: confusion matrix, ROC, learning ...
1
vote
1answer
45 views

Questionable diagnostics for a binary logistic model

The model including one binary outcome (0/1; incident rate ~1.2%), one main exposure, and 13 covariates. The whole model is significant and the goodness-of-fit is OK. However, model diagnostic is ...
3
votes
1answer
174 views

How can we have non-random patterns in the plot of simple linear regression residuals vs the predictor variable?

A) When considering a simple linear regression model, it is important to check the linearity assumption. Graphing the residuals vs the predictor variable can often give a good idea of whether or not ...
1
vote
0answers
48 views

How to distinguish suspicious leverages?

Given a linear model and the following hatvalues and influence.measures, how can I say which measurements are suspicious? I mean ...
0
votes
2answers
154 views

Test for endogeneity in regressions model?

In a regression model are there tests to detect the possibility of endogeneity in the model? For example, we have the White's test for heteroskedasticity. Is there something for endogeneity?