Diagnostic measures (such as residuals or some summary statistics calculated from residuals) are used to evaluate some aspect of quality of model fit to data.

learn more… | top users | synonyms (1)

0
votes
1answer
24 views

Should cohort studies in diagnosis research be age-matched?

It is clear that cohort studies in epidemiology should be matched for such variables as age, gender, etc. However, cross-sectional cohort studies may also be used to assess the accuracy of a new ...
3
votes
1answer
33 views

Tilted rectangle in residuals vs fitted plot

When performing diagnostics on an OLS model, what can make a plot of the predicted responses vs the residuals. Ideally we want a horizontal rectangle shape. But what does it mean when the plot is a ...
1
vote
0answers
17 views

GLM Diganostic Analysis - What plots actually matter?

In regards to this Poisson approach (How can the Poisson GLM be used instead of logistic regression for the Titanic survival data?). I always thought that Normal Q-Q, Cook's D distance plots were ...
0
votes
0answers
8 views

Autocorrelation of MAPE in case of lack of autocorrelation in residuals?

I'm building a forecasting tool for electricity market (spot power market). The model forecasts spot prices of power for each of 24 hours of next day. I measure its performance for whole days (for 24 ...
0
votes
0answers
13 views

Diagnostic measure not influenced by transformations of response variable

What are the diagnostic measures (like Cook's distance, H matrix, DFFITS, DFBETAS) that are not affected by the transformation of the response/dependent variable? And why?
0
votes
0answers
14 views

Logit model simulation - binned residual plot seems to exhibit bias

To understand how to perform diagnostics, I've been experimenting simulating logit models. The binned residual plot makes sense to me as a simple way of analysing whether residuals are distributed as ...
1
vote
1answer
50 views

Evaluation of diagnostic test ($R^2$ vs AUC)

A sort of a risk evaluation system is offered to us (sort of advertising). The output value is lethality (yes/no). The evaluators are several diagnoses, emergency status, age etc. No detailed ...
0
votes
0answers
18 views

Positive predictive value versus false positive rate

A colleague of mine has quoted to me the following: For a given population with a known specificity and a known event rate where a positive test is thought to predict the presence of an event: ...
1
vote
1answer
19 views

Is Box and Pierce test on residual or squared residual?

In a GARCH model, is it better to test the autocorrelation of squared residuals rather than the autocorrelation of residuals? Thanks!
1
vote
0answers
50 views

Why is my logistic regression fit that bad?

I was requested by a professor to analyze a high dimensional dataset (92 cases, 400+ variables, and a lot of NAs...) of various patient outcomes (gaussian distributed, counts, and binary) and their ...
0
votes
1answer
39 views

Diagnostics for quasipoisson glm for continuous data

I'm a little confused about how to use the quasipoisson family in the glm function. It was recommended by someone that I use it for my analysis, even though the data are continuous - and as such, I ...
0
votes
0answers
21 views

Logistic Regression Diagnostics

I am a bit confused how to assess the results of a logistic regression procedure I ran. Using SAS (on a 60% training set) - The LRT, Score and Wald test and got very low p-values. However, this model ...
1
vote
0answers
21 views

Problems in finding outliers and leverage points in non-linear regression

I'm implementing diagnostic of non-linear regression model $(y=ax^b)$. I'm trying to find out where outliers and leverage points in my model ...
0
votes
1answer
10 views

How can model misfit introduce dependency?

I read somewhere recently that mis-fitting a regression model can introduce dependency into the model. Unfortunately I cannot find the where I read this. Can anyone explain how mis-fitting a ...
0
votes
0answers
60 views

ARIMA diagnostic testing in Stata

I am using an ARIMA(1,1,0) in Stata. I have already executed estat aroots and wntestq (white noise test) for the residuals. I ...
1
vote
0answers
18 views

Diagnostic plots for logistic regression [duplicate]

I want to check for the assumptions of linearity in my logistic regression model, which states that the logit of the probability is a linear combination of predictors. It might be the case though, ...
0
votes
1answer
38 views

Zero-inflated models how to get predicted values = 0 stata

I'm fairly new at Stata and this is the main reason of my question. I did a longitudinal zero-inflated poisson model: ...
0
votes
0answers
15 views

Does it make sense to standardize residual using $SSE/(n-1)$?

See[Dean and Voss]Design and Analysis of Experiments,1999. pp.105 The authors proposed that we standardize residuals using: $$z_{it}=\frac{\hat{e_{it}}}{\sqrt{ssE/(n-1)}}$$ where ...
1
vote
0answers
43 views

Using Half-Normal Plot to Identify Outlying Observations

When we perform linear regression diagnostic, we usually utilize the half-normal plot (available in the faraway package in R) to identify outlying observations. ...
3
votes
1answer
78 views

When is it justified to “peek” at the outcome variable in model-building process?

I am referring to the following comment made in a 1996 paper by Dr Frank Harrell et al in Statistics in Medicine: Unless a formal penalized estimation technique is used, multiple comparisons ...
0
votes
0answers
48 views

Tukey Test in Linear Regression Model Development

I am not able to interpret properly the Tukey Test results from a linear regression model i have built. The Tukey Test for the model i believe is to test is the model is linear. Please correct me if i ...
0
votes
0answers
24 views

Externally studentized residuals

Assume the our model is simply $$Y = X\beta + \epsilon$$ with $\epsilon$~ $N_n(0,{\sigma}^2I)$. If we want to identify the existing outlier in our dataset, we use the externally studentized ...
0
votes
1answer
65 views

Statistical tests for comparing Positive/Negative likelihood ratios in two independent diagnsotic tests

I have a problem regarding the comparison of the likelihood ratios in two diagnostic tests. The definition of positive likelihood ratio in screening tests is "the probability of a person who has the ...
0
votes
0answers
41 views

Estimate ROC curve using binormal distribuiton

I am conducting a meta-analysis on diagnostic studies but for each study I have only mean and standard deviation reported. How can I estimate the ROC curve using the binormal assumption in R ? Tks,
3
votes
0answers
88 views

Can I compare different estimations approaches with AIC?

I'm running two different panel models. Model 1 is a Random Effects regression estimated using Maximum Likelihood and bootstrapped standard errors. Model 2 has the same main dependent variables and ...
2
votes
0answers
28 views

Independence of “residuals” in a Bayesian multilevel hierarchical model

So i'm having some problems realising what model checks I should do after fitting a bayesian model other than convergence diagnostics. Lets say i'm fitting a hierarchical bayesian regression model, I ...
0
votes
0answers
61 views

Practical beginners resource for building a dynamic OLS model

I need to model the current account balance of a country. The regressors are the real effective exchange rate, the domestic GDP and the GDP of the world. I am using data for 30 years (in logs). It is ...
1
vote
2answers
58 views

What visualizations do people use to debug a machine learning model?

Imaging I am refining a model. What visualization can I rely on to help me identify an error, e.g. feature deficiency, data bias? A few graphs I am aware of include: confusion matrix, ROC, learning ...
1
vote
1answer
40 views

Questionable diagnostics for a binary logistic model

The model including one binary outcome (0/1; incident rate ~1.2%), one main exposure, and 13 covariates. The whole model is significant and the goodness-of-fit is OK. However, model diagnostic is ...
3
votes
1answer
128 views

How can we have non-random patterns in the plot of simple linear regression residuals vs the predictor variable?

A) When considering a simple linear regression model, it is important to check the linearity assumption. Graphing the residuals vs the predictor variable can often give a good idea of whether or not ...
1
vote
0answers
42 views

How to distinguish suspicious leverages?

Given a linear model and the following hatvalues and influence.measures, how can I say which measurements are suspicious? I mean ...
0
votes
2answers
110 views

Test for endogeneity in regressions model?

In a regression model are there tests to detect the possibility of endogeneity in the model? For example, we have the White's test for heteroskedasticity. Is there something for endogeneity?
2
votes
0answers
56 views

what to do with ridiculous but valid leverage points

So I'm having some difficulty fitting a linear model to the data (see other post here glm model fit - can't find a family/link combination that produces good fit). In particular, I'm worried ...
1
vote
1answer
45 views

Definition of 'Model Diagnostics'

Can anyone help me out with explaining what the term 'model diagnostics' refers to when applied to multiple regression please? In particular, what tests are necessary to check whether your estimated ...
0
votes
0answers
24 views

Model diagnostic tests for local linear regression

What are some of the model diagnostic tests which are used/ would be suitable to use for a local linear regression model? Note that this is not the least squares linear model, but rather a special ...
0
votes
0answers
126 views

How to apply diagnostics to regression model from FactoMineR

Many diagnostics to assess regression models are listed on this page: http://www.statmethods.net/stats/rdiagnostics.html ...
0
votes
0answers
23 views

Combining LR+ and LR- in naive Bayes

I've come across a colleagues who advocates combining both the positive likehood ratios and negative likelihood ratios in the calculation of posterior probability of the likelihood of a ...
4
votes
1answer
1k views

How to manually calculate dfbetas

I am trying to replicate what the function dfbetas() does in R. dfbeta() is not an issue... Here is a set of vectors: ...
0
votes
0answers
145 views

Arima Models Diagnostics

I'm doing a forecasting using seasonal ARIMA method. I'm using astsa package in r and I'm testing two models that I can't decide which one is better to use than the other The ACf and PACF for the ...
0
votes
2answers
140 views

What does 8.6e-28 mean in the result from a White Test testing for heteroscedasticity?

I used a White Test for testing the homoscedasticity assumption of the linear regression I am working on. I have a problem with the interpretation as I have a result from the test in which the p-value ...
2
votes
1answer
249 views

Interpreting case influence statistics (leverage, studentized residuals, and Cook's distance)

I just wanted to clarify some things about leverage, studentized residuals, and Cook's distance: Does a large (in absolute value) studentized residual mean that a case is an outlier? Does a large ...
1
vote
0answers
21 views

I want to compare two calibrations with R… But cannot find the right answer

I first want to precise that I spent 2 hours searching for an answer, couldn't find something that was answering my question. So basically, I ran two calibration with my diagnostic test, so I ...
0
votes
0answers
118 views

How to compare diagnostic accuracy of two tests in unpaired data?

I am wondering whether I can compare diagnostic accuracy, sensitivity, specificity, positive and negative predictive values, and likelihood ratios, between two diagnostic procedures employed in two ...
3
votes
1answer
457 views

How to interpret the direction of the Harvey-Collier test and Rainbow test for linearity?

I implemented both those tests with R, using the lmtest package. Both tests directionally say the same thing (I think) with a very similar p-value of very close to 0. But, are those tests saying ...
2
votes
0answers
34 views

How to proceed with nonstationary variables in panels?

In most of the emprical papers using panel data, authors do not seem to "worry" too much aboout the non-stationarity of the individual variables. Yes, there is asymptotic theory for N and T going to ...
1
vote
2answers
346 views

Normality of residuals - contradiction between 'symplot' and 'qnorm'?

After running a multiple linear regression analysis, I wanted to assess normality of residuals. I plotted a histogram which showed an almost normal distribution of residuals. I also used ...
2
votes
2answers
285 views

Are the model residuals well-behaved (homoscedasticity)?

Can I say looking at this residuals-vs-fitted plots, that my residuals are homoscedastic?
1
vote
0answers
39 views

How do you approach transformations when modeling?

I'm working with a simple univariate dataset and I've built several models for it. Some I think are fairly decent given that datas structure. In order to get a decent model I had to do some ...
2
votes
1answer
332 views

Diagonal lines in residuals vs fitted values plot for ANOVA

I'm experiencing strange patterns of residuals. The following chart is a scatterplot of Standard residuals (Sres) versus Fits. I'm interested in the diagonal lines that mean that a higher fit leads to ...
3
votes
1answer
187 views

What are the three forms of the Park test for heteroskedasticity?

I understand the Park test for heteroskedasticity has three different forms. The best known one is in a log form: LN(Residual^2) = intercept + slope (LN(X)). The second one is in a linear form: ...