Questions tagged [goodness-of-fit]

Goodness of fit tests indicate whether or not it is reasonable to assume that a random sample comes from a specific distribution.

Filter by
Sorted by
Tagged with
1 vote
0 answers
67 views

Understanding $\chi^2$ in plain language [closed]

Suppose I was trying to explain what $\chi^2$ is and why it's important to my grandma. I want to give core intuition to this formula: $$\chi^2 = \sum_{i}\frac{(O_i-E_i)^2}{E_i}$$ I would tell her ...
Swike's user avatar
  • 129
0 votes
0 answers
27 views

Sample Size for Adaptive Lasso

Be gentle, I'm learning here. I have a fairly simple adaptive lasso regression that I'm trying to test for a minimum sample size. I used cross-validated mean squared error as the "score" of ...
JRW's user avatar
  • 1
0 votes
0 answers
17 views

How to calculate AIC of a spatial panel model in R?

This may be a better question for Stack Overflow, but I think a statistics answer may be warranted here: I have built some spatial panel models in R using the ...
geoscience123's user avatar
0 votes
0 answers
9 views

external internal validation and chi square

The internal test set, created from 20 percent of the train data set, consists of 'a' and 'b' labels. I take the 'a' labels from the internal test and combine them with the 'c' group of another ...
Nemo's user avatar
  • 1
1 vote
0 answers
17 views

Poor RMSEA/Fit for Simple Poisson Regression

I am running a simple Poisson regression. $X$ = time, $Y$ = count data. This is a huge dataset with many years. There is significance between $X$ and $Y$. But model shows poor fit via high RMSEA value....
mmt1026's user avatar
  • 11
2 votes
0 answers
61 views

How well does my model fit? Specifying a null-model in non-linear mixed models

I want to fit a model y ~ b * exp(-exp(a) * x), but including a random effect, with this data: ...
quak's user avatar
  • 31
0 votes
0 answers
13 views

How to determine goodness-of-fit between non-parametric 2d-datasets

Lets say I have a set of paired x' and y' values and I have a N sets of reference values also consisting of paired x and y values. I would like to determine which reference set best matches by x'y' ...
James London's user avatar
2 votes
1 answer
49 views

Use of weights in non-linear least square fitting

I would like to have your suggestions and help concerning my problem. I have images generated on a position sensitive detector. The signal for each pixels corresponds to the amount of 'particles' ...
toto's user avatar
  • 21
1 vote
1 answer
84 views

Deriving Sample version of Anderson Darling test statistic from the theoretical version

In literature, I have seen two types of Anderson-Darling test statistic. One is expressed as $A_T^2 = n\int_{-\infty}^{\infty}\frac{(F_n(x)-F(x))^2}{F(x)(1-F(x))}dF(x)$ and the other is given by $A_s^...
DevD's user avatar
  • 115
2 votes
0 answers
40 views

Error metric for regression of count data: Poisson Deviance or Mean Square Error?

I would like to understand what difference it makes, if I use, for example, either Mean Square Error or Poisson Deviance as error metric/loss function for a regression of count data. Are there any a-...
g g's user avatar
  • 2,686
3 votes
2 answers
59 views

Appropriate test to use when each sample is a permutation from $S_k$

Suppose I have a dataset $x_1, \ldots, x_n$ where each $x_i$ is a permutation of $\{1, \ldots, k\}$. [For example, if $k=4$ the data might be $x_1 = (2, 1, 3, 4)$, $x_2 = (3, 2, 4, 1)$, $x_3 = (4, 3, ...
angryavian's user avatar
  • 2,328
3 votes
1 answer
32 views

Comparing $R^2$ values that measure different predictions

I have a question regarding $R^2$. Say, a model A predicts $y$ based on $x$, and a model B predicts $x$ based on y. I want to test which model is a better fit to the data. Since model A predicts $y$, ...
Erik's user avatar
  • 31
2 votes
1 answer
51 views

Goodness of fit in GLM produces NANs

I have a GLM model which has a quadratic term and looks like this. ...
Saad Khan's user avatar
  • 101
0 votes
0 answers
37 views

What is the number of "explanatory variables" in the adjusted R2 formula

I am trying to assess the goodness of fit of a surface I've developed (i.e., model predicts a variable y based on 2 variables x1 and x2). My model has 5 parameters which are estimated (and is likely ...
Erik's user avatar
  • 31
2 votes
0 answers
63 views

Which dependent variable is mostly impacted by predictor?

Usually one wants to identify the most important predictors (x1, x2, x3..., xn) in a regression model. My question is reversed: I have a data set that contains a risk factor risk and several outcomes ...
a.henrietty's user avatar
0 votes
0 answers
9 views

Goodness-of-fit test for single but high-dimensional datum

Are there any goodness-of-fit tests for samples consisting of only one datum that is high-dimensional? Usually, G-o-F tests are run using samples with multiple datums, each having a single dimension. ...
user654123's user avatar
0 votes
0 answers
44 views

Poor fitted vs. actual values

I'm using a BART model (Bayesian additive regression tree) to predict the relative risk of an outcome (21,384 observations) controlling for 388 features and I'm getting a really poor actual vs. fitted ...
Tim's user avatar
  • 13
0 votes
0 answers
22 views

How should MacKenzie and Bailey goodness-of-fit test be used for several models at a time in model selection?

I am running single-season occupancy models and I usually have many selected "best models" by AICc criterion. At the time, I am running MacKenzie and Bailey's (2004) goodness-of-fit test for ...
Magdalena Salas's user avatar
1 vote
3 answers
178 views

Poisson regression for rare events?

Poisson regression is commonly used to analyse count data. However, when we deal with rare events it does not seem to be appropriate any more. At least, graphical criteria to assess the model fit like ...
Student's user avatar
  • 21
0 votes
1 answer
45 views

CFA: chi-square value is 0 but with degrees of freedom

I want to do a SEM analysis with an actor-partner interdependence model in Mplus. I managed to calculate it and everything seems right if I look at the means, SD's, ...
Axenox's user avatar
  • 1
0 votes
0 answers
39 views

testing difference in distributions with population values (using python)

I am looking for the best method to test for a difference between two distributions. I have data from a patient group, and my aim is to compare the distribution to a reference distribution. However, ...
Ana's user avatar
  • 101
3 votes
2 answers
353 views

Goodness-of-fit test for very skewed data [closed]

I am working with two quite large datasets (9000 obs and 3800 obs). Each observation has been grouped into 1 of 12 categories and I am looking to test if the frequency of categories in the smaller ...
Elin Sjoeholm's user avatar
2 votes
1 answer
171 views

Negative Binomial Regression Model - Effect of Removing Significant Covariates

I have been dabbling in NB regression for less than a year now. I have applied the well known g.o.f. tests. Lately I started using the Conditional Moment (CM) test, described in Cameron and Trivedi ...
Barton's user avatar
  • 21
0 votes
1 answer
115 views

Fitting data with a beta distribution using fitdist from fitdistrplus package in R

I have a variable x that corresponds to data from a database. I've been trying to find the best distribution to fit it and looking at the histogram I figured either a normal, weibull or logistic ...
Ismael ancona's user avatar
2 votes
2 answers
64 views

Importance of goodness of fit in Hypothesis testing

I am struggling with one question. Is goodness of fit of a model necessary when your purpose is to test hypothesis regarding a coefficient? To be specific, I am regressing formal credit access on ...
Rupon's user avatar
  • 21
0 votes
0 answers
72 views

What is this nonparametric goodness-of-fit test?

I wrote down a goodness-of-fit test that I have not seen before. However, it is quite elementary and has many applications, so I bet it must have been known. Could someone tell me its name? Setup. The ...
Student's user avatar
  • 235
0 votes
1 answer
66 views

obtaining a goodness of fit indices for an analysis with tweedie distribution

I performed an analysis specifying a dependent variable with a tweedie distribution using the glmmTMB package in R. I got several fit indices, specifically: AIC, BIC, Logliklihood. What is the most ...
Lior's user avatar
  • 41
3 votes
1 answer
61 views

Should Kullback-Leibler as an R2 value be large or small for better goodness-of-fit

I am trying to use the Kullback-Leibler as an R2 value for goodnes-of-fit for GLM models. The R package performance defines their function as: ...
simpson's user avatar
  • 133
0 votes
0 answers
9 views

Variations of Correlation Coefficient of Simple Linear Regression with Estimators [duplicate]

Suppose we are using an Ordinary Least Squares (OLS) estimator of $\alpha_{0}$ and $\alpha_{1}$ for the simple linear regression below: $$ H_{i} = \alpha_{0} + \alpha_{1}X_{i} + \epsilon_{i} $$ How ...
Plesiozaurus's user avatar
0 votes
0 answers
40 views

C-statistic (or AUC) for fractional logistic regression (i.e. continuous regression)

I have proportional data to which I have fit a logistic regression (i.e. fractional logistic regression). The statistician in our group wants me to provide a c-statistic for the regression. My ...
Brandon's user avatar
  • 356
2 votes
1 answer
427 views

Goodness of fit of structural equation modelling

I am currently working on a structural equation modeling project using the lavaan package in R. The model satisfied all the goodness of fit tests (GFI, AGFI, CFI, ...
Suba's user avatar
  • 21
0 votes
0 answers
15 views

Can I move on to the next level (test) of invariance when my fit gets better?

I am running a group invariance test. The data consists of data from students from grades 5 and 6 for one school (School A) and only from grade 5 in another school (School B). I have already run group ...
EmH's user avatar
  • 113
1 vote
0 answers
48 views

Interpretation of Anderson–Darling test

Assume that I am using Anderson–Darling test to evaluate whether a given sample of data is drawn from a normal distribution with some mean and standard deviation. If you accept the null hypothesis in ...
Barbab's user avatar
  • 333
1 vote
0 answers
44 views

Yes or no: is Pearson’s r is a measure of goodness of fit to an affine function? [duplicate]

Is the statement "Pearson’s r is a measure of goodness of fit to an affine function" literally true? Why or why not?
BigMistake's user avatar
1 vote
0 answers
19 views

Is it possible average an information criterion across models?

Is it possible to take the average of information criterion like the AIC? For my model comparison, I have 24 different models. I use 4 different GARCH models each with 6 different distributions for ...
Hello there's user avatar
0 votes
0 answers
63 views

How Anderson-Darling test with Braun's method works in R?

I generally work with skewed and heavy-tailed lifetime distributions, so I need to check the goodness-of-fit of certain data to such distributions. After some exploration, I came to know that the AD ...
DevD's user avatar
  • 115
0 votes
0 answers
41 views

Easy goodness of fit tests when using svyolr package (ordered logit regression on complex survey data)?

for a uni project I am running an ordered logit regression on survey data, using the svyolr package in R. I am looking to report some sort of goodness of fit for the model but am struggling to find ...
Ollie's user avatar
  • 1
3 votes
1 answer
254 views

Pearson chi squared test vs deviance test in GLM

From my understanding, both Pearson chi squared test and deviance test can be used to assess the goodness of fit for GLM, but they have different alternative hypotheses. For the Pearson chi-squared ...
user344849's user avatar
1 vote
1 answer
138 views

Comparing the goodness of fit for multiple GLS models applied to different data sets

I fitted simple linear models to several (6) data sets of different sizes, using GLS. All models have the same predictors (but the coefficients are different). I would like to compare the goodness of ...
soungalo's user avatar
  • 111
0 votes
0 answers
41 views

Is Pearson's chi-squared appropriate for models with low deviance explained?

I'm working on fitting a binomial GLM using LASSO in R (package glmnet). My response variable is a proportion which is generated using count data (successes and failures). The main purpose of my model ...
pfadenhw's user avatar
0 votes
2 answers
41 views

Is goodness-of-fit a reasonable way to test the usefulness of correlated precitors

Under a hypothesis testing framework, I would like to model $Y \sim X_1 + X_2 + Z$ where X_1 and X_2 are two related, likely ...
Jeremy's user avatar
  • 1,439
1 vote
2 answers
60 views

How can you convert sum of squares deviation to an r-squared value?

I would like to calculate the r-squared value for some regressions. The model (its in a GUI) I am using gives me "goodness of fit" in terms of the sum of squares deviation. I am using ...
vermicellion's user avatar
0 votes
0 answers
36 views

Influence of Large Data Size on Logistic Regression Fit and McFadden's $R^2$

I am currently working on a logistic regression analysis and have encountered a situation where I have approximately 16 million data points. I am interested in understanding the influence of such a ...
LeterPeko's user avatar
0 votes
0 answers
48 views

Recent goodness-of-fit tests related to proper scoring rules, CRPS and work of Gneiting

In a conference, I overheard a casual discussion about testing goodness of fit (GOF). Kolmogorov-Smirnov, Cramer-von Mises and Anderson-Darling were mentioned as some established GOF tests, but then ...
Richard Hardy's user avatar
3 votes
1 answer
49 views

Does the null distribution matter when choosing between goodness-of-fit tests?

There are multiple goodness of fit tests such as Kolmogorov-Smirnov, Cramer-von Mises, Anderson-Darling, and Kuiper's. Which one shall one use? There are threads (e.g. 1, 2, 3) and a nice textbook ...
Richard Hardy's user avatar
0 votes
0 answers
76 views

Keep or remove insignificant covariance values when modifying CFA model?

I am working to improve goodness of fit in a CFA model and am wondering about if I should retain added covariance values that are insignificant. I am adding covariances that are indicated by ...
Rachelb's user avatar
  • 11
1 vote
0 answers
35 views

ELPD and number of observations

Bit of a Bayesian noob here with a very simple question - does ELPD of a model depend on the number of observations / datapoints? Longer version: I am comparing several models for the same underlying ...
TY Lim's user avatar
  • 171
0 votes
1 answer
31 views

Are these trials identically distributed? These trials from a panel data are all Bernoulli experiments; do they have same probability of success p?

Scenario: I have an ordered/indexed sample that supposedly comes from a binomial distribution with fixed probability of success p. (By ordered/indexed I mean that not only I care about the percentage ...
Zmod2Z's user avatar
  • 9
1 vote
0 answers
24 views

An Alternative In-sample-goodness-of-fit Measure in Machine Learning Methods for Estimating Heterogeneous Causal Effects Paper

In 3.5.2 of Machine Learning Methods for Estimating Heterogeneous Causal Effects by Susan Athey and Guido Imbens it is stated: "If the models include an intercept, as they usually do, most ...
fsafe's user avatar
  • 43
0 votes
0 answers
34 views

Goodness of fit for in-sample predictions

This is probably a stupid question that I'm overthinking, and apologies if it's super obvious, but - Say you have some regression model y ~ f(x1, x2...) that you ...
TY Lim's user avatar
  • 171

1
2 3 4 5
24