Questions tagged [goodness-of-fit]

Goodness of fit tests indicate whether or not it is reasonable to assume that a random sample comes from a specific distribution.

452 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
10 votes
0 answers
2k views

Very different scale parameter estimates in Poisson regression

The background: I'm analysing survival data using a Poisson model. I've splitted the data on 2 time-scales (attained age and calendar year). Attained age is modelled using flexible parametric ...
user avatar
  • 1,626
9 votes
0 answers
367 views

"Brute force" expected deviance for logistic regression?

A commonly used goodness of fit statistic for logistic regression is the deviance. This is also known as the likelihood ratio chi-square statistic. It is defined as: $$D=\sum_{i=1}^{N}d_i^2$$ $$d_i^...
user avatar
8 votes
0 answers
4k views

Chi-squared calculation for (nonlinear) model goodness of fit for data with trials

I would like to be certain of how to calculate the $\chi^2$ value for this scenario: We desire a measure of the goodness of fit of a nonlinear model to a data set. The data consists of many data ...
user avatar
  • 81
7 votes
0 answers
994 views

Goodness-of-fit for Discrete Distributions

I've been doing some data analysis with Scipy. So far I accomplished this with continuous distributions: I can fit a probability distribution to a set of data points using a maximum likelihood fit. ...
user avatar
7 votes
0 answers
1k views

What techniques are there to measure goodness of fit of Deming (orthogonal) regression?

Questions: Even if there is no "widely accepted" technique, is there a useful-and-above-average technique for estimating goodness of fit in orthogonal regressions? What are the pros/cons of this ...
user avatar
  • 113
7 votes
0 answers
604 views

Calculate goodness-of-fit (with deviance) to compare averaged models?

I need to compare the goodness of fit of several averaged logistic regression models by calculating the deviance explained. I'm using the MuMIn package in R to ...
user avatar
6 votes
0 answers
233 views

Goodness of fit to a fitted distribution

Assume we have a sample of data $\{x_1, \dots, x_n\}$ and a family of distributions $(f_\theta)_\theta$ indexed by some parameter vector $\theta$. We would like to fit $\theta$ to $\{x_1, \dots, x_n\}$...
user avatar
6 votes
0 answers
70 views

Interpreting contradictory goodness of fit

I am studying the effect of categorical independent variables on a binary outcome. To do this, I have fitted a glm as following : ...
user avatar
  • 321
5 votes
0 answers
177 views

What are the pros and cons of different metrics for evaluating a logistic regression model?

In the data science world, I have always evaluated the performance of logistic regression models simply using ROC/AUC. However recently, I've read from some traditional statistics source about some ...
user avatar
  • 421
5 votes
0 answers
2k views

chi-squared goodness-of-fit: effect size and power

How can effect size and power of the chi-square goodness-of-fit test be calculated such that: Effect size is linear and standardized (given effect size $E_{x^2}$, $0 \le E_{x^2} \le 1$) so that ...
user avatar
5 votes
1 answer
226 views

How to test if many samples are from the same distribution

For each day of the week I observe $n_i$ independently chosen values from some process. I would like to be able to answer the following two questions. Are the distributions from which the samples are ...
user avatar
  • 1,889
5 votes
0 answers
3k views

Interpreting MANOVA and redundancy analysis of a canonical correlation analysis

I have done a canonical correlation analysis using the American Community Survey Dataset. The analysis is done between Ancestry and ...
user avatar
  • 51
4 votes
0 answers
308 views

How to compare model fits between datasets of different sizes?

A measure like mean square error appears to tolerate some variation in dataset size but, for example, a two-parameter model is still likely to produce a great (yet meaningless) fit to any dataset with ...
user avatar
  • 205
4 votes
0 answers
286 views

What is the difference between Shapiro-Francia and Shapiro-Wilk tests of normality?

I apologize for this easy/basic question. Please forgive me if this is a duplicate question. I've been researching the Shapiro-Wilk test (1965) of normality for a project. In doing so, I found an ...
user avatar
  • 141
4 votes
0 answers
142 views

Assessing fit for a logistic regression

I've been working on fitting a logistic regression model to a data set. I've attached the data set below. It includes data about the occurrence of snow, SnowBinary...
user avatar
4 votes
0 answers
86 views

Basics relative to chi square, likelihood, fits,

I'm confused to separate all the different meanings and connections. The background of my question: On the one hand related to lmer models and on the other hand to the goodness of a fit. And their ...
user avatar
  • 2,309
4 votes
0 answers
1k views

How to specify the parameter df in the goodness-of-fit test for a t-copula?

I have read the Q&A, and Consultation Paper, Article 26, p. 47-48. I tried to fit a t-copula. I have found the coefficients rho.1 and ...
user avatar
  • 850
4 votes
0 answers
93 views

Are there statistical tests for detecting model miscalibration that can be used as an alternative to binning probabilities?

I have a model which gives success probability estimates for a sequence of non-identically distributed Bernoulli trials. When the model is not constrained to be well calibrated with respect to ...
user avatar
  • 3,858
4 votes
0 answers
699 views

Am I calculating the Anderson-Darling statistic correctly for comparing PDFs?

I want to compute the Anderson-Darling statistic to use as a goodness of fit test for some PDF distributions. I am using R. I want to compare some models to the observations, which are my baseline. ...
user avatar
  • 91
4 votes
0 answers
207 views

Generalization of one-sample Kolmogorov-Smirnov test for non identically distributed data?

Is there a standard approach for testing goodness-of-fit to a probability distribution when the samples are independent but not identically distributed? In other words, given the data $\{\{x_1, y_1\}, ...
user avatar
  • 433
4 votes
0 answers
969 views

Goodness of fit test for a normal distribution

In the example from the web site I was trying out this problem in page 107 about Goodness of fit test for a normal distribution.Question is about analysis of fat content of hamburgers. I understand ...
user avatar
  • 1,045
4 votes
0 answers
4k views

Measure of goodness-of-fit in Ordinal Logistic Regression with continuous independent variable

In case of the ordinal logistic regression, both of the goodness-of-fit statistics, Pearson and Deviance goodness-of-fit measures, should be used only for models that have reasonably large expected ...
user avatar
  • 3,385
4 votes
0 answers
1k views

How to compare two power law distributions?

Disclaimer: I am a biologist, so please don't hesitate to correct me if I am making unwise assumptions. I am looking at the distribution of a unique 6mer in a series of 2100mer (I have 415 ...
user avatar
  • 61
3 votes
0 answers
107 views

Question about statement in Oster (2019): variation in a control

In Oster (2019), she discusses how authors typically include controls and examine coefficient stability as a way to test for presence of confounding, and points out that researchers should consider ...
user avatar
  • 415
3 votes
0 answers
84 views

Joint asymptotic distribution and measures of fit

I have been looking for something similar but couldn't find any, unfortunately. Some help would be much appreciated. Consider a simple bivariate linear mean regression $$y=\beta x + e$$ where $E[e|x]=...
user avatar
3 votes
0 answers
178 views

Improving fit of underdispered beta regression model in glmmtmb

I have survey data where the outcome is the proportion of a research budget interviewees wished to assign to one of three different "types" of research into solutions for various issues. I ...
user avatar
  • 31
3 votes
0 answers
60 views

Is it really possible to get p-value as exactly 1 for Goodness-of-Fit for Poisson Regression in R?

I am building a Poisson regression (Model name fit_4) with a composite count (score) data as the dependent variable and other several variables as an independent. ...
user avatar
3 votes
0 answers
49 views

Is there any anova-like approach for calculating contingency tables across multiple levels within a factor

I want to compare success rates across a large number of different levels within a third factor to detect if there are statistically significant differences for at least one of the groups. I'm not ...
user avatar
3 votes
0 answers
61 views

Multivariable residual analysis and goodness of fit

I am reading the paper Statistical Modeling: The Two Cultures (2001) by Leo Breiman. In section 5.2 he claims: Residual analysis is similarly unreliable. In a discussion after a presentation of ...
user avatar
3 votes
0 answers
239 views

Over "specification" of a statistics model?

For a simple example, I am fitting data with a likelihood function generated by a normal distribution. The first model is the normal distribution with two-parameters. The second competing model is the ...
user avatar
  • 719
3 votes
0 answers
810 views

Random forest [R]: why is my OOB RMSE so much smaller than test RMSE?

I'm doing the kaggle challenge on timetravel predictions where the task is to predict the duration (Y) of a uber trip given some information about the start and end coordinates and the time the trip ...
user avatar
  • 1,434
3 votes
0 answers
65 views

A goodness of fit test for two discrete distributions with unequal variance?

I have a computer simulation in which a virtual agent end up in different areas of a layout based on several factors. There are 18 conditions, so the data (you can find the csv file for a toy dataset ...
user avatar
  • 290
3 votes
0 answers
772 views

Evaluating model fit for glmmTMB with beta distribution

I am looking for a way to evaluate model fit (e.g. $R^2$, %deviance explained) for the following model: ...
user avatar
3 votes
0 answers
3k views

Goodness-of-fit glm: Pearson's residuals or deviance residuals?

I want to evaluate the goodness-of-fit (or badness-of-fit) of a negative binomial glm. However, even here within CV, I've seen multiple different approaches for doing so. Some use the the residual ...
user avatar
  • 89
3 votes
0 answers
427 views

Determining the type of data distribution

I am looking on expanding my introductory knowledge in stat, and I have come up with this challenge. I have an application that is designed to analyze and index news articles. The goal of the ...
user avatar
  • 31
3 votes
0 answers
161 views

Log-likelihood calculation on separate test set

I'm looking for a "hack" in R that would allow me to calculate the log-likelihood of a GLM fit on a separate test set easily regardless of the distribution. For instance for a Gamma GLM, this is how ...
user avatar
3 votes
0 answers
253 views

Estimating model fit in multiple mediator SEM

I'm building a multiple mediation model in lavaan. My research question concerns to what extent a relationship between a continuous X and a binary Y is mediated by two continuous mediators M1 and M2. ...
user avatar
3 votes
0 answers
391 views

Why is the Kolmogorov–Smirnov (KS) test more popular than the Overlapping Coefficient (OVL)?

The Overlapping Coefficient (OVL) measures the common area between two Probability Density Functions (see this question for more details). Intuitively, this seems like a good way to gauge the ...
user avatar
3 votes
0 answers
113 views

Use cases for chi square goodness of fit tests

In the answer to the question: Chi squared goodness of fit test, user glen_b comments that: The circumstances where it really makes much sense to formally test goodness of fit are surprisingly few. ...
user avatar
  • 83
3 votes
1 answer
264 views

Detect the correct distribution from a small sample size by using fitdistrplus in R

The simplest version of the issue that I am looking for help is: How to detect the correct distribution from a small sample size in R by using fitdistrplus A simpler version: I am generating some ...
user avatar
3 votes
0 answers
6k views

Pseudo R2 Values For Negative Binomial Regression Model in R Yields Inconsistent, Sometimes Negative Values

I am getting some odd results and I feel like I have been banging my head against the wall for a while, so here I am. I am running a negative binomial regression analysis using MASS:glm.nb: ...
user avatar
3 votes
0 answers
823 views

glmmADMB- Pseudo R^2 and residual deviance criteria

I’ve been using glmmADMB to fit zero-inflated negative binomial models with a random effect (as far as I can tell, this is the only package that will allow me to do this). I have been trying to do ...
user avatar
3 votes
0 answers
157 views

time series model selection in exploratory research

I work in the field of behavioural interventions and I use dynamic models to gain better insight into (explain) the process of behaviour change and to help inform future behavioural interventions (...
user avatar
3 votes
0 answers
473 views

Evaluating hetroskedasticity in a binomial residuals vs. fitted plot in glmer?

I am trying to validate the goodness of fit of a model in glmer using residuals plotting. I went through many threads here related to this but still I am not sure that the solutions offered apply to ...
user avatar
  • 141
3 votes
0 answers
116 views

How to measure the goodness of fit of a 3D line?

I have data points in 3D space, and I'd like to quantify how well they conform to a straight line. Specifically, this is a body tracking problem. I have multiple options available for the 3D ...
user avatar
3 votes
0 answers
261 views

Comparing confirmatory factor analysis models with different number of observed variables

I have a 14 item scale that is hypothesized to measure self-esteem. The first model I specify uses all 14 items and predicts one latent factor (i.e., self-esteem). I would then like to specify a ...
user avatar
  • 33
3 votes
1 answer
110 views

To what extent are the different versions of $R^2$ comparable, with and without an intercept

I have been trying to fit some data with a linear regression. I don't have any theoretical assumptions on the regression, and from what I know both an intercept or no intercept can be plausible ...
user avatar
  • 41
3 votes
0 answers
3k views

R - How do the methods in goodfit() (package vcd) calculate degrees of freedom?

I have counts and I am trying to determine if they conform to the negative binomial distribution. I am using goodfit() from the vcd package, however the two methods are giving me conflicting results: ...
user avatar
  • 61
3 votes
0 answers
433 views

Estimating probability distribution function of data stream

Although a similar question exists, I couldn't find my answer. I'm not a statistician hence please neglect if some terminologies aren't correct and let me know if I am interpreting something wrong. ...
user avatar
  • 83
3 votes
0 answers
168 views

Tracy-Widom distribution - Phase transitions - catastrophe/chaos - 'surface-fit'/'curve-fit' software

Is there an algorithm to determine the fit of a sample set of data to a saddle curve? I'd like to know the variance from the closest fit the sample has to: \begin{align} z &= x^3 - 3xy^2 \\ z &...
user avatar

1
2 3 4 5
10