Questions tagged [diagnostic]

Diagnostic measures (such as residuals or some summary statistics calculated from residuals) are used to evaluate some aspect of quality of model fit to data.

Filter by
Sorted by
Tagged with
0 votes
0 answers
37 views

Model diagnostic plots for anova test

Could you please help me in interpreting the model diagnostic plots? I feel there is a problem with the scale-location plot?
2 votes
1 answer
24 views

Levene's Test and residuals vs. fitted plot lead to different interpretations about heteroscedasticity

I am performing a one-way ANOVA in R with the following data: Cu Day CC Cu1 49 30934500 Cu1 49 26860125 Cu1 49 46524750 Cu10 49 15272561 Cu10 49 31601659 Cu10 49 17627634 Cu100 49 3718127 ...
0 votes
0 answers
13 views

Testing and training data for LMM/GLMM

I was reading this article on logistic regression and ML and noticed that they explicitly mention the need for using training sets of data and testing sets of data: However, I have never seen this ...
0 votes
1 answer
19 views

Intuition behind martingale residuals and their properties in survival modeling

I am reading Tutz & Schmid "Modeling Discrete Time-to-Event Data" (2016) chapter 4 Evaluation and Model Choice section 4.2 Residuals and Goodness-of-Fit. A martingale residual is defined ...
1 vote
1 answer
28 views

Intuition behind the null distribution of the deviance statistic in survival models

I am reading Tutz & Schmid "Modeling Discrete Time-to-Event Data" (2016) chapter 4 Evaluation and Model Choice section 4.2 Residuals and Goodness-of-Fit. A goodness-of-fit statistic ...
0 votes
0 answers
4 views

How do I interpret this Scale-location graph based on an anova?

I ran an anova using 1 categorical factor with 2 levels (Habitat: 1 and 2) and 1 continous factor (leaf carbon concentration)as the explanatory variables. The response variable is continous (...
1 vote
1 answer
23 views

What is the hat matrix and why is it inappropriate for GLMM standardized residuals?

When I run this code to plot standardized residuals for a standard logistic regression: ...
10 votes
2 answers
777 views

Is higher AUC always better?

Let's say we measure binary classifier performance by ROC graph, and we have two separate models with distinct AUC (The Area Under the Curve) values. Is the model with the higher AUC value always ...
  • 317
3 votes
0 answers
28 views

GLMM and BLUPs: high correlation between random effects in a logistic GLMM

Background: In an experiment, subjects had to choose whether they wanted an immediate reward or to wait for a larger reward (dichotomous dependent variable: yes/no). This choice was made multiple ...
1 vote
0 answers
25 views

What kind of transformation should I do to improve my linear mixed model?

I had asked in a previous post how to check the validity conditions of a linear mixed model I made, and here is the link After additional diagnostics I have the impression that the distribution of the ...
2 votes
1 answer
69 views

How to properly diagnose a linear mixed model

I am trying to study the relationship between the type of bariatric surgery and weight loss. This is longitudinal data with different BMI measurements per patient. This the formula of the model: ...
1 vote
0 answers
24 views

Finding a fit for left-skewed continuous data

I am looking at various plant community traits and how they relate to a number of environmental variables in a woodland. I am using mixed models as some samples come from the same woodlands so I'm ...
0 votes
1 answer
32 views

Confusing presentation of survival model diagnostics in Tutz & Schmid (2016)

I am reading Tutz & Schmid "Modeling Discrete Time-to-Event Data" (2016) chapter 4 Evaluation and Model Choice section 4.2 Residuals and Goodness-of-Fit. I got stuck on p. 78: The first ...
0 votes
0 answers
23 views

OLS diagnostics in R

I conducted a OLS regression (N=2046) in R and I am a little unsure about the diagnostics. I reject the Shapiro-Wilk test and the Breusch-Pagan test, so this would mean no normality and ...
user avatar
0 votes
0 answers
9 views

When is single factor ANOVA regression checking out for model diagnostics?

Consider a simple randomized experiment with 4 treatments(no control) where the outcome of interest is difference between pre-treatment and post-treatment count. I want to conclude 4 treatments does ...
  • 1,169
1 vote
0 answers
19 views

Is there some standard way to diagnose a structural time series model (also called simple unobserved components model)?

I am dealing with a structural time series model (also called a simple unobserved components model), and I wonder if there is some standard way to diagnose this sort of models. In most reference books ...
  • 123
2 votes
2 answers
335 views

Analysis of forecast errors from Facebook Prophet

I created a forecasting model Facebook Prophet and now trying to analyse the forecast errors (yhat - forecasted). Following are 3 graphs I plotted First one is raw forecast errors, second one is ...
0 votes
1 answer
30 views

non-constant variance of $\hat{\epsilon}$ against $\hat{y}$

In Faraways Linear Models (2ed.) page 77, he mention: "when non-constant variance is seen in the plot of $\hat{\epsilon}$ against $\hat{y}$", a transformation of the response $y$ to $h(y)$ ...
0 votes
0 answers
15 views

Adjusting weighting and df for constant variance

Faraways Linear models (2ed), page 75, does the following: ...
0 votes
0 answers
44 views

negative binomial glm: issues with model diagnostics when using offset term

I'm trying to model count data using a negative binomial glm with an offset in R and am having some issues with getting the model to fit properly when I use an offset term. Here is a reproducible ...
  • 1
0 votes
0 answers
26 views

Residual diagnostic for Boston Housing Dataset

I have some questions regarding the procedures for proper analysis of the "Boston Housing" dataset. My problem concerns the dysgnostics of the residuals and how to correct possible ...
  • 1
0 votes
0 answers
45 views

Linear and Ridge regression yield the same R² and MSE

I'm currently practicing on the NY taxi dataset but I'm having an issue and I'm sure it's because of some stupid mistake. After cleaning the dataset, I'm taking the following features and try to ...
  • 25
8 votes
2 answers
289 views

Objective criteria for assumption violations that do not utilize p-values?

Suppose we have a standard regression model and want to identify whether we have violated the assumptions of the model. Traditionally, we might utilize a significance test to determine whether (for ...
  • 569
0 votes
0 answers
65 views

DHARMa problems: Poisson or Zero-inflated negative binomial model?

I have a dataset of abundance of of rodent (AA) in 63 sites, which located in two main area (northern and southern part, NS). AA ...
3 votes
1 answer
74 views

What does it mean when there is a pattern in residuals related to the dependent variable?

I created a linear regression model and realized I first need to check some assumptions. Autocorrelation not existing -> valid, since it is a between-subject experiment. Low collinearity between ...
1 vote
1 answer
171 views

parallel lines (cluster) in PCA plots (PC1 vs PC2)

There are about 20 subjects, 3 treatment groups, and 1000+ genes in my data; the 1000+ genes were processed in two batches. Could anyone comment on why I am seeing parallel lines/clusters in the PCA ...
  • 63
0 votes
0 answers
8 views

Which mixed model to use in this scenario?

I conducted an experiment in which I am trying to model the relationship between my response yield [dt/ha] and the predictors soil moisture [%] + weed coverage [%]+ treatment + distance (+ date) and ...
  • 83
0 votes
0 answers
252 views

Fixed effect instrumental variable (IV) regression with available diagnostic tests

May I please know an R package and code to run fixed effect instrumental variable (IV) regression with available diagnostic tests (e.g., weak instrument test, exogeneity test (using Wu-Hausman), ...
  • 404
0 votes
0 answers
9 views

Is there any Statistic method which could reflect the diagnose value when the prevalence of special type of characteristic is lower

Our present study is conducting a novel diagnosis test for predicting a disease by the X-ray. The gold-standard is a minimally invasive procedure can test a tissue sample for the disease. In our ...
1 vote
0 answers
28 views

Can we objectively determine whether assumptions have been violated in R?

I'm testing statistical assumptions in R and so far I've been using plot(model, which = c(1:6)), which produces six graphs for linearity, normality of errors, ...
1 vote
0 answers
46 views

OLS linearity assumptions

Do we have to check on stata that the OLS assumptions of linearity of a regression if the regression is already linear in parameters? I'm having trouble with this concept. Thanks in advance.
0 votes
0 answers
46 views

Sample size for F1 score

I am developing a binary test and will be evaluating it based on the F1-score against a perfect reference standard. I want to calculate the sample size I'd need to have 80% power to be 95% confident ...
1 vote
1 answer
21 views

Impact of residuals on forecast

I'm working with ARMA models right now and I was wondering about the following case: If we have late significant lags in the residuals ACF and the rest of the earlier residual lags weren't significant,...
  • 57
0 votes
0 answers
12 views

Positive Predictive Value given one positive and two negative test results

I am wondering how to calculate the positive predictive value in case of one positive and two negative test results. Put differently, what is the probability that a person has a disease given that one ...
  • 1
0 votes
2 answers
151 views

Optimal lag length in VAR/VECM: IC or Residual test?

I read so many answers in here that I should use IC(information criteria) to determine the optimal lag length in VAR/VECM. But also it is important to check the residual of VAR/VECM has no-...
  • 9
1 vote
2 answers
224 views

What are the assumptions of MLE and how to test them using residuals?

When using ordinary least square (OLS) method, there are certain residual diagnostics that need to be performed. In a similar manner, what diagnostics should be performed when using Maximum Likelihood ...
  • 109
1 vote
0 answers
89 views

Are ARIMA model residuals normally distributed

I am running the ARIMA model using python's package: sm.tsa.statespace.SARIMAX. I get the following results. As we can see, the ...
1 vote
1 answer
84 views

Is correlation matrix of residuals , a method to check for autocorrelation of residuals in the model?

I am modelling and forecasting using a factor augmented VAR. I got the following residual matrix after fitting the model. My question is how to interpret it and does it indicate any autocorrelations ...
  • 109
3 votes
1 answer
41 views

The different usage of ROC between diagnotic tests and machine learning

I am currently very confused about the ROC usage in diagnostic tests and machine learning. In the scenario of medical diagnostic studies, many tutorial does not mention the data split procedure as ...
  • 45
0 votes
0 answers
14 views

Concept clarification: which pre-test probability to use for the Fagan’s nomogram

I am currently testing the diagnostic accuracy of my predictive model. I hope to visualize my model using Fagan’s nomogram. I want to ask for the pre-test probability in the nomogram, should I plug in ...
0 votes
0 answers
18 views

Model diagnostic plots [duplicate]

i have built a predictive model and these are the model diagnostic. How do i interpret them? it looks like the model is linear based on the residuals vs fitted plot. but what about others?
user avatar
0 votes
1 answer
172 views

Detecting autocorrelation of residuals using ACF and PACF plots

How to identify autocorrelation of residuals in the fitted VAR model. I have provided the ACF and PACF plots below. There are some significant lags in the PACF plot. Does it mean that my model has ...
  • 109
0 votes
0 answers
62 views

(Explanatory) Modelling with PMG and MG estimators

Hello kind people of Stackexchange. I happen to be kind of stuck and in need of help. See, I want to make an ARDL model with PMG and/or MG estimator (Pesaran, 1999) (okay, in fact it may be the CCE ...
  • 21
1 vote
1 answer
116 views

Why should a Li-Mak Test on Squared Standardized Residuals be preferred over a ARCH LM Test or Ljung-Box Test on Squared Residuals?

If I didn't misunderstand the literature, the predominant approach to test for autoregressive conditional heteroscedasticity in (G)ARCH models is to apply the ARCH LM test of Engle or the Ljung-Box ...
0 votes
1 answer
51 views

Residual diagnostics in Dynamic Factor Models (DFM)

I am quite new to Dynamic Factor Models. My main task is to estimate the model on my training data and test the model on my test data set. My question is, should we perform residual diagnostics to an ...
  • 109
0 votes
0 answers
16 views

Why do regression through the origin when taking residuals of response variable against fitted values regressed against residuals of added var?

I know my title may be a little off, but I'm not sure how to describe the terms I'm talking about in a more concise manner so if there is one do let me know. I finished working through an exercise in ...
1 vote
1 answer
36 views

Different results for residual normal distribution between Jarque-Bera test and Q-Q Plot

I am trying to test for normality of residuals using 2 different ways: Using Jarque-Bera test Q-Q Plot I can see different results, for the JB test the value is 19.9553 with a probability of 0.00005....
0 votes
0 answers
16 views

Diagnostics for Biprobit Model

What is the procedure for assessing the fit of a biprobit model? Unfortunately, I couldn't find this material being covered in the standard statistics or econometric textbooks, which is odd, given the ...
  • 11
0 votes
1 answer
29 views

Box-Jenkins Methodology: Statistical model checking

According to https://en.wikipedia.org/wiki/Box%E2%80%93Jenkins_method : Statistical model checking by testing whether the estimated model conforms to the specifications of a stationary univariate ...
  • 91
0 votes
1 answer
348 views

Diagnostic Accuracy: AUC vs cross-validation accuracy

Can someone please explain the difference between the diagnostic accuracy from the AUC model vs diagnostic accuracy in the below code? Note: I found the below code at the bottom of this forum/page: ...

1
2 3 4 5
8