Diagnostic measures (such as residuals or some summary statistics calculated from residuals) are used to evaluate some aspect of quality of model fit to data.

learn more… | top users | synonyms (1)

5
votes
0answers
264 views

“Brute force” expected deviance for logistic regression?

A commonly used goodness of fit statistic for logistic regression is the deviance. This is also known as the likelihood ratio chi-square statistic. It is defined as: $$D=\sum_{i=1}^{N}d_i^2$$ $$d_i^...
4
votes
0answers
78 views

what to do with ridiculous but valid leverage points

So I'm having some difficulty fitting a linear model to the data (see other post here glm model fit - can't find a family/link combination that produces good fit). In particular, I'm worried ...
4
votes
0answers
602 views

Diagnostics for GEE in R

I have been checking out which diagnostics to use for a GEE analysis. It seem that influence measures are appropriate (Preisser, 1996). Does anyone know of a package that can be used in R to examine ...
4
votes
0answers
4k views

Fitting a zero-inflated negative binomial regression with R

In this thread, I laid out a problem involving fitting a model that attempts to use minor league baseball statistics to predict success at the major league level (explained in full in the thread). ...
3
votes
0answers
111 views

Can I compare different estimations approaches with AIC?

I'm running two different panel models. Model 1 is a Random Effects regression estimated using Maximum Likelihood and bootstrapped standard errors. Model 2 has the same main dependent variables and ...
3
votes
0answers
65 views

Verification of assumptions in TBATS model

I have a question about using BATS/TBATS models implemented in the forecast package for R. In De Liv­era, Hyndman & Snyder (2011) the models are used without any following analysis. Is it OK to ...
3
votes
0answers
468 views

How to compute sample size to compare two diagnostic tests

I will be performing two diagnostic tests (one is the gold standard, one is novel) on the same subject aiming to establish sensitivity, specificity, PPV and NPV. What formula may be used to compute ...
2
votes
0answers
31 views

Is there any meaningfully robust approach to conduct a network meta-analysis of diagnostic test accuracy studies?

I am working on a systematic review including several imaging modalities for coronary artery disease, but the evidence network is quite large, including different modalities, often compared one to the ...
2
votes
0answers
23 views

When creating a multiple regression model for a subgroup, is it necessary to test all assumptions again?

My results section consists of a multiple regression analysis considering 3 factors, containing all of my participants. Following this, I have considered males and females separately, by construction ...
2
votes
0answers
35 views

How should I interpret/follow-up on mixed logistic regression (GLMM) diagnostics?

I have experimental data (n subjects = 64) in which the response variable, accuracy (0 or 1), was measured 9 times within subjects. My predictor is Condition (A vs B) measured between subjects. I ...
2
votes
0answers
41 views

Problems in finding outliers and leverage points in non-linear regression

I'm implementing diagnostic of non-linear regression model $(y=ax^b)$. I'm trying to find out where outliers and leverage points in my model ...
2
votes
0answers
38 views

Independence of “residuals” in a Bayesian multilevel hierarchical model

So i'm having some problems realising what model checks I should do after fitting a bayesian model other than convergence diagnostics. Lets say i'm fitting a hierarchical bayesian regression model, I ...
2
votes
0answers
41 views

How to proceed with nonstationary variables in panels?

In most of the emprical papers using panel data, authors do not seem to "worry" too much aboout the non-stationarity of the individual variables. Yes, there is asymptotic theory for N and T going to ...
2
votes
0answers
47 views

Refining “good” mixing time estimate

Fix a Markov chain $\{ X_{t} \}_{t \in \mathbb{N}}$ with mixing time $\tau_{\mathrm{mix}}$. Assume that I know some finite bound on the mixing time $\tau_{\mathrm{mix}} < \tau < \infty$, and ...
2
votes
0answers
666 views

How to verify linearity assumption in linear regression with categorical predictors?

I have used simple linear regression, and I'm now checking that the model meets the assumption of linearity. The model used a continuous response variable and categorical explanatory variables. How ...
2
votes
0answers
4k views

Collinearity diagnostics disagree - VIF, condition index, and correlation matrix

I'm working with a large dataset consisting of just over 1 million cases. The data are longitudinal covering 14 years and hierarchical with about 500 of the level 2 units. Each case is a criminal ...
1
vote
0answers
10 views

MCMC diagnostics for EMC or SMC

Are diagnostics developed for MCMC (e.g. Gelman-Rubin, Geweke) suitable for output from Evolutionary Monte Carlo (EMC) or Sequential Monte Carlo (SMC)?
1
vote
0answers
38 views

How to interpret regression diagnostic plots for multiple linear regression analysis (with specific example)

I recently using the day.csv file which is downloaded from http://archive.ics.uci.edu/ml/machine-learning-databases/00275/ to build a regression model for the last column “cnt” in R. This is the ...
1
vote
0answers
25 views

Diagnostic device hypothesis testing for superiority to a fixed value

I am designing a study to demonstrate that a diagnostic medical device is more sensitive than a fixed performance goal of 70%. Currently I am attempting to determine the appropriate sample size. My ...
1
vote
0answers
27 views

How to interpret diagnostic plots for GEV distribution

I produced the following Diagnostic plots, from the following R Code: ...
1
vote
0answers
25 views

Standardized residuals vs fitted values for Poisson regression

McCullagh and Nelder's book on glm suggest to plot standardized deviance residuals against either the linear predictor ($\hat{\eta}$) or the fitted values ($\hat{\mu}$) transformed to the constant ...
1
vote
0answers
18 views

How to decide between quasi-poisson and negative binomial?

I tried quasi-poisson and negative binomial glms on my counted data in R. The estimates are pretty much the same but p-value are different. Quasi-poisson gives insignificant result. NB give ...
1
vote
0answers
30 views

GLM Diganostic Analysis - What plots actually matter?

In regards to this Poisson approach (How can the Poisson GLM be used instead of logistic regression for the Titanic survival data?). I always thought that Normal Q-Q, Cook's D distance plots were ...
1
vote
0answers
64 views

Why is my logistic regression fit that bad?

I was requested by a professor to analyze a high dimensional dataset (92 cases, 400+ variables, and a lot of NAs...) of various patient outcomes (gaussian distributed, counts, and binary) and their ...
1
vote
0answers
69 views

Using Half-Normal Plot to Identify Outlying Observations

When we perform linear regression diagnostic, we usually utilize the half-normal plot (available in the faraway package in R) to identify outlying observations. ...
1
vote
0answers
51 views

How to distinguish suspicious leverages?

Given a linear model and the following hatvalues and influence.measures, how can I say which measurements are suspicious? I mean ...
1
vote
0answers
22 views

I want to compare two calibrations with R… But cannot find the right answer

I first want to precise that I spent 2 hours searching for an answer, couldn't find something that was answering my question. So basically, I ran two calibration with my diagnostic test, so I ...
1
vote
0answers
44 views

How do you approach transformations when modeling?

I'm working with a simple univariate dataset and I've built several models for it. Some I think are fairly decent given that datas structure. In order to get a decent model I had to do some ...
1
vote
0answers
148 views

What convergence diagnostics are appropriate for a Bayesian hierarchical logistic regression model?

Using WinBUGS, I fit several Bayesian hierarchical logistic regression models for the mean of a binary response variable conditional on a set of criteria. I am now using CODA in R to determine if my ...
1
vote
0answers
97 views

R lmer Model Diagnosis qqnorm

I fitted this lmer model: m1 <- lmer(logR ~ N_g.m.2 * Year + (1|Wh/N_g.m.2), data = CO2_Ratio) Rendering the attached qqplot. ...
1
vote
0answers
251 views

Model assumptions and diagnostics for proportional hazard regression model with frailty in R

I am wondering are there any assumptions that must be met in the proportional hazard regression model with frailty? I remember that in regular proportional hazard model without frailty all variables ...
1
vote
0answers
296 views

VECM Diagnostic Test

I have got few questions about VECM and cointegration test. Basically I conducted a cointegration test on two time series (spot vs forward price) by using the Johansen procedure. The results suggest ...
1
vote
0answers
106 views

What plots should be used for diagnostics for linear mixed model?

Before fitting a linear mixed model, can any plots be used to show a random intercept/slope is justifiable in the model? I.e. these plots may indicate a different pattern for each individual over time....
1
vote
0answers
69 views

Help in interpreting AUC values from ordinal variables

I would appreciate if someone could explain to me the benefit of using area under ROC for evaluating agreements between two raters. Here is an example from two raters on, let's say, clinical time ...
1
vote
0answers
58 views

Simulation for model checks for sample size

In the book Bayesian and Frequentist Regression Methods, Wakefield notes that estimators for coefficients in a linear model will be normal if the error terms are normal or if the sample size is ...
1
vote
0answers
203 views

Has anyone written a package in R to calculate diagnostic plots after clogit (conditional logistic regression)? e.g. leverage

Has anyone written a package in R to calculate diagnostic plots after clogit, conditional logistic regression? e.g. leverage. Or ...
1
vote
0answers
103 views

How do I go about conducting model diagnostics on WLS?

I'm familiar with the diagnostics required for OLS, however I'm in new territory with a model I'm fitting to data in R, using Poisson regression with GLM. What are the standard methods in evaluating ...
1
vote
0answers
92 views

Failure detection method

I receive 1000 points per day from installations who produces electricity. Every installation must proportionally produce the same amount of energy. I have to spot failures in those data. The actual ...
0
votes
0answers
25 views

Lag length for the Ljung-Box test

I have an ARIMA model applied on hourly data: Arima.fit2 <- Arima(tsTrain, order=c(17,1,0)) The length of my training set is 60 hours. In the end I plan to ...
0
votes
0answers
17 views

Assessing the residual independence assumption (nonlinear least squares regression diagnostics)

I would like to assess the assumptions underlying nonlinear regression models using statistical tests rather than graphical methods since I have thousands of fitting results. I am not certain ...
0
votes
0answers
12 views

Same error on all sets

My concern is the classification of data splitted into training, cross validation and test sets. By evaluating my model, the values of precision, recall, f-score and auroc are nearly the same for all ...
0
votes
0answers
16 views

Autocorrelation of MAPE in case of lack of autocorrelation in residuals?

I'm building a forecasting tool for electricity market (spot power market). The model forecasts spot prices of power for each of 24 hours of next day. I measure its performance for whole days (for 24 ...
0
votes
0answers
14 views

Diagnostic measure not influenced by transformations of response variable

What are the diagnostic measures (like Cook's distance, H matrix, DFFITS, DFBETAS) that are not affected by the transformation of the response/dependent variable? And why?
0
votes
0answers
30 views

Logit model simulation - binned residual plot seems to exhibit bias

To understand how to perform diagnostics, I've been experimenting simulating logit models. The binned residual plot makes sense to me as a simple way of analysing whether residuals are distributed as ...
0
votes
0answers
56 views

Positive predictive value versus false positive rate

A colleague of mine has quoted to me the following: For a given population with a known specificity and a known event rate where a positive test is thought to predict the presence of an event: ...
0
votes
0answers
27 views

Logistic Regression Diagnostics

I am a bit confused how to assess the results of a logistic regression procedure I ran. Using SAS (on a 60% training set) - The LRT, Score and Wald test and got very low p-values. However, this model ...
0
votes
0answers
153 views

ARIMA diagnostic testing in Stata

I am using an ARIMA(1,1,0) in Stata. I have already executed estat aroots and wntestq (white noise test) for the residuals. I ...
0
votes
0answers
22 views

Does it make sense to standardize residual using $SSE/(n-1)$?

See[Dean and Voss]Design and Analysis of Experiments,1999. pp.105 The authors proposed that we standardize residuals using: $$z_{it}=\frac{\hat{e_{it}}}{\sqrt{ssE/(n-1)}}$$ where $\hat{e_{it}}=y_{it}-...
0
votes
0answers
88 views

Tukey Test in Linear Regression Model Development

I am not able to interpret properly the Tukey Test results from a linear regression model i have built. The Tukey Test for the model i believe is to test is the model is linear. Please correct me if i ...
0
votes
0answers
29 views

Externally studentized residuals

Assume the our model is simply $$Y = X\beta + \epsilon$$ with $\epsilon$~ $N_n(0,{\sigma}^2I)$. If we want to identify the existing outlier in our dataset, we use the externally studentized ...