Questions tagged [goodness-of-fit]

Goodness of fit tests indicate whether or not it is reasonable to assume that a random sample comes from a specific distribution.

452 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
2k views

Very different scale parameter estimates in Poisson regression

The background: I'm analysing survival data using a Poisson model. I've splitted the data on 2 time-scales (attained age and calendar year). Attained age is modelled using flexible parametric ...
• 1,626
367 views

A commonly used goodness of fit statistic for logistic regression is the deviance. This is also known as the likelihood ratio chi-square statistic. It is defined as: $$D=\sum_{i=1}^{N}d_i^2$$ $$d_i^... • 22.8k 8 votes 0 answers 4k views Chi-squared calculation for (nonlinear) model goodness of fit for data with trials I would like to be certain of how to calculate the \chi^2 value for this scenario: We desire a measure of the goodness of fit of a nonlinear model to a data set. The data consists of many data ... • 81 7 votes 0 answers 994 views Goodness-of-fit for Discrete Distributions I've been doing some data analysis with Scipy. So far I accomplished this with continuous distributions: I can fit a probability distribution to a set of data points using a maximum likelihood fit. ... 7 votes 0 answers 1k views What techniques are there to measure goodness of fit of Deming (orthogonal) regression? Questions: Even if there is no "widely accepted" technique, is there a useful-and-above-average technique for estimating goodness of fit in orthogonal regressions? What are the pros/cons of this ... • 113 7 votes 0 answers 604 views Calculate goodness-of-fit (with deviance) to compare averaged models? I need to compare the goodness of fit of several averaged logistic regression models by calculating the deviance explained. I'm using the MuMIn package in R to ... 6 votes 0 answers 233 views Goodness of fit to a fitted distribution Assume we have a sample of data \{x_1, \dots, x_n\} and a family of distributions (f_\theta)_\theta indexed by some parameter vector \theta. We would like to fit \theta to \{x_1, \dots, x_n\}... • 99.4k 6 votes 0 answers 70 views Interpreting contradictory goodness of fit I am studying the effect of categorical independent variables on a binary outcome. To do this, I have fitted a glm as following : ... • 321 5 votes 0 answers 177 views What are the pros and cons of different metrics for evaluating a logistic regression model? In the data science world, I have always evaluated the performance of logistic regression models simply using ROC/AUC. However recently, I've read from some traditional statistics source about some ... • 421 5 votes 0 answers 2k views chi-squared goodness-of-fit: effect size and power How can effect size and power of the chi-square goodness-of-fit test be calculated such that: Effect size is linear and standardized (given effect size E_{x^2}, 0 \le E_{x^2} \le 1) so that ... 5 votes 1 answer 226 views How to test if many samples are from the same distribution For each day of the week I observe n_i independently chosen values from some process. I would like to be able to answer the following two questions. Are the distributions from which the samples are ... • 1,889 5 votes 0 answers 3k views Interpreting MANOVA and redundancy analysis of a canonical correlation analysis I have done a canonical correlation analysis using the American Community Survey Dataset. The analysis is done between Ancestry and ... • 51 4 votes 0 answers 308 views How to compare model fits between datasets of different sizes? A measure like mean square error appears to tolerate some variation in dataset size but, for example, a two-parameter model is still likely to produce a great (yet meaningless) fit to any dataset with ... • 205 4 votes 0 answers 286 views What is the difference between Shapiro-Francia and Shapiro-Wilk tests of normality? I apologize for this easy/basic question. Please forgive me if this is a duplicate question. I've been researching the Shapiro-Wilk test (1965) of normality for a project. In doing so, I found an ... • 141 4 votes 0 answers 142 views Assessing fit for a logistic regression I've been working on fitting a logistic regression model to a data set. I've attached the data set below. It includes data about the occurrence of snow, SnowBinary... • 26.9k 4 votes 0 answers 86 views Basics relative to chi square, likelihood, fits, I'm confused to separate all the different meanings and connections. The background of my question: On the one hand related to lmer models and on the other hand to the goodness of a fit. And their ... • 2,309 4 votes 0 answers 1k views How to specify the parameter df in the goodness-of-fit test for a t-copula? I have read the Q&A, and Consultation Paper, Article 26, p. 47-48. I tried to fit a t-copula. I have found the coefficients rho.1 and ... • 850 4 votes 0 answers 93 views Are there statistical tests for detecting model miscalibration that can be used as an alternative to binning probabilities? I have a model which gives success probability estimates for a sequence of non-identically distributed Bernoulli trials. When the model is not constrained to be well calibrated with respect to ... • 3,858 4 votes 0 answers 699 views Am I calculating the Anderson-Darling statistic correctly for comparing PDFs? I want to compute the Anderson-Darling statistic to use as a goodness of fit test for some PDF distributions. I am using R. I want to compare some models to the observations, which are my baseline. ... • 91 4 votes 0 answers 207 views Generalization of one-sample Kolmogorov-Smirnov test for non identically distributed data? Is there a standard approach for testing goodness-of-fit to a probability distribution when the samples are independent but not identically distributed? In other words, given the data \{\{x_1, y_1\}, ... • 433 4 votes 0 answers 969 views Goodness of fit test for a normal distribution In the example from the web site I was trying out this problem in page 107 about Goodness of fit test for a normal distribution.Question is about analysis of fat content of hamburgers. I understand ... • 1,045 4 votes 0 answers 4k views Measure of goodness-of-fit in Ordinal Logistic Regression with continuous independent variable In case of the ordinal logistic regression, both of the goodness-of-fit statistics, Pearson and Deviance goodness-of-fit measures, should be used only for models that have reasonably large expected ... • 3,385 4 votes 0 answers 1k views How to compare two power law distributions? Disclaimer: I am a biologist, so please don't hesitate to correct me if I am making unwise assumptions. I am looking at the distribution of a unique 6mer in a series of 2100mer (I have 415 ... • 61 3 votes 0 answers 107 views Question about statement in Oster (2019): variation in a control In Oster (2019), she discusses how authors typically include controls and examine coefficient stability as a way to test for presence of confounding, and points out that researchers should consider ... • 415 3 votes 0 answers 84 views Joint asymptotic distribution and measures of fit I have been looking for something similar but couldn't find any, unfortunately. Some help would be much appreciated. Consider a simple bivariate linear mean regression$$y=\beta x + e where $E[e|x]=... 3 votes 0 answers 178 views Improving fit of underdispered beta regression model in glmmtmb I have survey data where the outcome is the proportion of a research budget interviewees wished to assign to one of three different "types" of research into solutions for various issues. I ... • 31 3 votes 0 answers 60 views Is it really possible to get p-value as exactly 1 for Goodness-of-Fit for Poisson Regression in R? I am building a Poisson regression (Model name fit_4) with a composite count (score) data as the dependent variable and other several variables as an independent. ... 3 votes 0 answers 49 views Is there any anova-like approach for calculating contingency tables across multiple levels within a factor I want to compare success rates across a large number of different levels within a third factor to detect if there are statistically significant differences for at least one of the groups. I'm not ... • 153 3 votes 0 answers 61 views Multivariable residual analysis and goodness of fit I am reading the paper Statistical Modeling: The Two Cultures (2001) by Leo Breiman. In section 5.2 he claims: Residual analysis is similarly unreliable. In a discussion after a presentation of ... 3 votes 0 answers 239 views Over "specification" of a statistics model? For a simple example, I am fitting data with a likelihood function generated by a normal distribution. The first model is the normal distribution with two-parameters. The second competing model is the ... • 719 3 votes 0 answers 810 views Random forest [R]: why is my OOB RMSE so much smaller than test RMSE? I'm doing the kaggle challenge on timetravel predictions where the task is to predict the duration (Y) of a uber trip given some information about the start and end coordinates and the time the trip ... • 1,434 3 votes 0 answers 65 views A goodness of fit test for two discrete distributions with unequal variance? I have a computer simulation in which a virtual agent end up in different areas of a layout based on several factors. There are 18 conditions, so the data (you can find the csv file for a toy dataset ... • 290 3 votes 0 answers 772 views Evaluating model fit for glmmTMB with beta distribution I am looking for a way to evaluate model fit (e.g.$R^2$, %deviance explained) for the following model: ... 3 votes 0 answers 3k views Goodness-of-fit glm: Pearson's residuals or deviance residuals? I want to evaluate the goodness-of-fit (or badness-of-fit) of a negative binomial glm. However, even here within CV, I've seen multiple different approaches for doing so. Some use the the residual ... • 89 3 votes 0 answers 427 views Determining the type of data distribution I am looking on expanding my introductory knowledge in stat, and I have come up with this challenge. I have an application that is designed to analyze and index news articles. The goal of the ... • 31 3 votes 0 answers 161 views Log-likelihood calculation on separate test set I'm looking for a "hack" in R that would allow me to calculate the log-likelihood of a GLM fit on a separate test set easily regardless of the distribution. For instance for a Gamma GLM, this is how ... 3 votes 0 answers 253 views Estimating model fit in multiple mediator SEM I'm building a multiple mediation model in lavaan. My research question concerns to what extent a relationship between a continuous X and a binary Y is mediated by two continuous mediators M1 and M2. ... • 31 3 votes 0 answers 391 views Why is the Kolmogorov–Smirnov (KS) test more popular than the Overlapping Coefficient (OVL)? The Overlapping Coefficient (OVL) measures the common area between two Probability Density Functions (see this question for more details). Intuitively, this seems like a good way to gauge the ... • 1,118 3 votes 0 answers 113 views Use cases for chi square goodness of fit tests In the answer to the question: Chi squared goodness of fit test, user glen_b comments that: The circumstances where it really makes much sense to formally test goodness of fit are surprisingly few. ... • 83 3 votes 1 answer 264 views Detect the correct distribution from a small sample size by using fitdistrplus in R The simplest version of the issue that I am looking for help is: How to detect the correct distribution from a small sample size in R by using fitdistrplus A simpler version: I am generating some ... 3 votes 0 answers 6k views Pseudo R2 Values For Negative Binomial Regression Model in R Yields Inconsistent, Sometimes Negative Values I am getting some odd results and I feel like I have been banging my head against the wall for a while, so here I am. I am running a negative binomial regression analysis using MASS:glm.nb: ... • 171 3 votes 0 answers 823 views glmmADMB- Pseudo R^2 and residual deviance criteria I’ve been using glmmADMB to fit zero-inflated negative binomial models with a random effect (as far as I can tell, this is the only package that will allow me to do this). I have been trying to do ... 3 votes 0 answers 157 views time series model selection in exploratory research I work in the field of behavioural interventions and I use dynamic models to gain better insight into (explain) the process of behaviour change and to help inform future behavioural interventions (... 3 votes 0 answers 473 views Evaluating hetroskedasticity in a binomial residuals vs. fitted plot in glmer? I am trying to validate the goodness of fit of a model in glmer using residuals plotting. I went through many threads here related to this but still I am not sure that the solutions offered apply to ... • 141 3 votes 0 answers 116 views How to measure the goodness of fit of a 3D line? I have data points in 3D space, and I'd like to quantify how well they conform to a straight line. Specifically, this is a body tracking problem. I have multiple options available for the 3D ... • 211 3 votes 0 answers 261 views Comparing confirmatory factor analysis models with different number of observed variables I have a 14 item scale that is hypothesized to measure self-esteem. The first model I specify uses all 14 items and predicts one latent factor (i.e., self-esteem). I would then like to specify a ... • 33 3 votes 1 answer 110 views To what extent are the different versions of$R^2\$ comparable, with and without an intercept

I have been trying to fit some data with a linear regression. I don't have any theoretical assumptions on the regression, and from what I know both an intercept or no intercept can be plausible ...
• 41
3k views

R - How do the methods in goodfit() (package vcd) calculate degrees of freedom?

I have counts and I am trying to determine if they conform to the negative binomial distribution. I am using goodfit() from the vcd package, however the two methods are giving me conflicting results: ...
• 61
433 views

Estimating probability distribution function of data stream

Although a similar question exists, I couldn't find my answer. I'm not a statistician hence please neglect if some terminologies aren't correct and let me know if I am interpreting something wrong. ...
• 83