Questions tagged [goodness-of-fit]
Goodness of fit tests indicate whether or not it is reasonable to assume that a random sample comes from a specific distribution.
978
questions
0
votes
0answers
5 views
Checking Conway-Maxwell-Poisson model adequacy
I am trying to troubleshoot model adequacy problems for underdispersed count data (number of correct responses in a simple task; dispersion ratio is 0.3) that I modeled with Conway-Maxwell-Poisson. ...
0
votes
0answers
45 views
Evaluate the quality of the negative binomial regression model fit
Our response variable is highly skewed and there is evidence of overdispersion as well. We tried with the Poisson, and Quasi-Poisson models. Both Poisson and Quasi-Poisson models failed to satisfy ...
0
votes
0answers
11 views
Goodness-of-fit test for conditional distribution
The chi-square goodness-of-fit test allows us to test if a data sample $(y_n)_{n=1,\ldots,N}$ agrees with some proposed model distribution $P(y)$. However, in a typical machine learning setting we ...
1
vote
0answers
36 views
Appropriate goodness-of-fit test for the negative binomial regression
I have used the following Pearson $χ2$ test and the deviance test to assess the negative binomial regression using R as
#########################################
...
1
vote
1answer
11 views
Best practice for establishing configural invariance?
I was hoping for some perspectives on the best practices for establishing configural invariance using ordered indicators with WLSMV estimation. I have a 3-factor scale and I am assessing invariance ...
0
votes
0answers
9 views
Reduced chi-squared for models fit to different subsets of data
I have some data to which I am fitting piecewise linear models. I want to select different subsets of the data, fit a model to each of them, and then compare which subsets are best able to be ...
0
votes
0answers
10 views
Which goodness of fit test to use?
I have two different vectors and I would like to compare these with each other so I could say that statistically the distributions come from the same sample. The data is below:
X1: 149 144 140 135 131 ...
2
votes
0answers
38 views
goodness of fit for psychometric data (perceptual threshold)
I'm running an experiment on perceptual thresholds in audio. I'll try not to bog you down with too many details:
The experiment is about vibrato speed; specifically, when can you tell the difference ...
1
vote
1answer
30 views
goodness of fit test to check for normality
I have task about hearing ability and here is frequency table
bad ability
average
good ability
total
19
64
17
100
It is said that average ability is everything +/- 1sd. This is all data I have. I ...
3
votes
1answer
58 views
Is there a G-test equivalent for continuous variables?
The G-test is similar to the chi-square test for goodness of fit. It is proportional to the kl-divergence.
I am wondering if there is a similar test that is applicable to continuous variables. Since ...
1
vote
0answers
13 views
Calculating the tail bounds for a beta-binomial regression
I have a beta-binomial regression model that depends on a probability $p$ and a given over-dispersion $\beta$ and is used to parametrise the distribution of $Y$ in the following way
$$ Y(x) \sim ...
0
votes
0answers
32 views
How to select and interpret Goodness of fit test for an epidemiological study with small sample size?
I have a study that found an association between exposure to antidepressants and the risk of preeclampsia.
The number of women who were exposed and had an outcome (i.e. preeclampsia) was small:
10 ...
0
votes
0answers
12 views
Joint estimation of OLS and Logistic Model
I ran two equations for panel data, the Q (OLS model), the second is ERM (Logistic model), the main independent variable in equation 1 is the main dependent var in equation 2.
To solve the endogeneity ...
0
votes
0answers
13 views
Compute sigma significance from chi2 goodness of fit
Let's assume I have some binned data, which I have fitted with two curves (two hypotheses). Each fit yields a different chi^2 value and degrees of freedom, such that I can already see clearly that one ...
1
vote
0answers
11 views
How do I calculate effect size for Anderson-Darling test?
I'm looking at the list of tests and effect sizes on https://cran.r-project.org/web/packages/statsExpressions/vignettes/stats_details.html, but I can find any way to calculate effect size for Anderson-...
0
votes
0answers
16 views
Goodness of fits tests. Multiple sample test
I have a question. Please help me:
Assume we need to test at a particular level (say, 10%) whether clients are equally likely to complain about quality of service at three firm branches. In the ...
2
votes
0answers
13 views
Improving fit of underdispered beta regression model in glmmtmb
I have survey data where the outcome is the proportion of a research budget interviewees wished to assign to one of three different "types" of research into solutions for various issues. I ...
0
votes
0answers
15 views
Goodness-of-fit for conditional logistic regression in a 1:1 matched case-control study
Dear Stackexchange community;
I would appreciate if someone would guide me on this matter.
On data analysis of a 1:1 matched case control study based on age and gender through using conditional ...
0
votes
0answers
8 views
How to include random effects in a Chi-square Goodness of Fit test
I'm trying to figure out how to include subject identity (random effect) in an analysis where subjects make a choice between two concurrent options.
I found this website, which shows how to code ...
0
votes
0answers
18 views
Test if a sample comes from a theoretical distribution (with a specific type of censoring)
My situation is the following. I am estimating certain numbers from a dataset split in $n$ parts, one number for each part. How and why exactly is not relevant to the current question, but as a toy ...
2
votes
0answers
33 views
What are the best methods for comparing Torgerson (Classical) Vs. Metric Vs. Non-Metric MDS results?
I am trying to contrast results of various MDS approaches applied on the same dataset and understand their comparative interpretation.
I calculate the goodness of fit for the various models with the ...
1
vote
0answers
11 views
Why are the 'discrete' one-sample Cramer-von Mises tests in Choulakian et al. 1994 written as such
Given a set of data $X$ of length $N$, with a theoretical probability distribution $F$ and empirical distribution $F_n$, the Cramer-von Mises statistic is essentially the sum of the squared distance ...
0
votes
0answers
21 views
different results between the published paper and my code to calculate the p-value of Kolmogorov–Smirnov (K–S) test using R
I want to know if the following distributions fit the given data well or not. I used the Kolmogorov–Smirnov (K–S) statistic for the following two distributions but, i obtained different p-value of k-s ...
0
votes
0answers
79 views
How to interpret NRMSE (Normalised Root Mean Square Error) without comparing models?
I have fitted some robust mixed effects linear regression models (using robustlmm::rlmer in R).
I have calculated the normalised root mean square error (NRMSE) for these models but I want to make sure ...
0
votes
0answers
19 views
Statistical power of the Lilliefors test: how to compare results for samples of different sizes?
The power of the Lilliefors test (LT) strongly depends on the sample size.
I need to apply LT to a discrete distribution to find the minimum value m_min above which it is exponential.
I need to do it ...
0
votes
0answers
11 views
Determining trend statistic for two time series
I have two time series: one with reported scatter plot points, and a solid line that represents a fit to the data based on other variables. Here:
https://docs.google.com/document/d/1gizOAV8ZjaATLm-...
0
votes
0answers
27 views
Effect size for goodness of fit test
What effect size measure can be used for a goodness of fit test (given data and theoretical proportions)?
I read about the Cohen's w (https://en.wikipedia.org/wiki/Effect_size#Cohen's_w) but I also ...
0
votes
0answers
40 views
Why do I have a negative R squared if my model has an intercept?
Don't know what else to say, I am running a first difference panel IV model and getting a negative R squared. I imagine it has something to do with instrumental variables but can't figure out what. I ...
1
vote
1answer
67 views
Akaike Information Criterion I cannot interpret the result
Maybe is a silly question, or maybe I'm doing something wrong.
I've tried to implement AIC criterion to estimate the optimum number of parameters using Auto Regressive (AR) linear models using white ...
0
votes
1answer
24 views
If adjusted R-squared is zero, can individual coefficients have meaningful interpretation in multiple regression?
It can happen that adjust R-squared metric in multiple regression is zero (or very close to zero), but individual coefficients are statistically significant.
Under these circumstances, can I still ...
0
votes
1answer
43 views
Testing Benford's law using Chi Squared
I am attempting to show that the uniform distribution is NOT a good distribution to use when determining the distribution of leading significant digit for number. Here is my code:
...
1
vote
1answer
66 views
ROC for testing goodness of fit
I'm interested in using ROC to test for goodness of fit for binary models such as logistic regression.
I'm a bit confused by the literature where it is mostly just explained as a valid technique to ...
3
votes
0answers
55 views
Is it really possible to get p-value as exactly 1 for Goodness-of-Fit for Poisson Regression in R?
I am building a Poisson regression (Model name fit_4) with a composite count (score) data as the dependent variable and other several variables as an independent. ...
0
votes
0answers
33 views
I have a very high $R^2 (0.96)$ for my ARDL time series model, is this problematic?
All my variables are stationary, so cointigration can't be the problem. I have included one lag of the dependent variable and 5 explanatory variables. When I remove the explanatory variables, the R^2 ...
0
votes
0answers
15 views
Goodness of fit non linear models when errors are unknown
If I understand correctly.
a R^2 (coefficient of determination) is a good parameter to estimate the goodness of fit when performing a linear regression.
when errors are given, a reduced chi^2 is ...
1
vote
2answers
122 views
Kolmogorov-Smirnov and AIC giving opposite goodness-of-fit results. Is this possible/surprising/normal?
I have some data on the duration of several activities (rounded to the nearest half hour). I'm trying to add up these random variables (one per activity) so that I can calculate the total duration of ...
0
votes
0answers
14 views
Goodness of fit of a model with N*M responses
I have a model that predicts human responses. Both the model and the human classify some objects with confidence ratings and the output is in a form of a N*M table, e.g.:
...
0
votes
0answers
35 views
Why is this method used to find out whether a Poisson distribution is appropriate?
I have a problem which is concerned with figuring out whether a Poisson distribution is a suitable model or not for a certain scenario.
"The distribution of the length of the first word of each ...
2
votes
1answer
91 views
Goodness of fit test for LASSO
How would you do a goodness of fit test for Lasso regression?
Im guessing that the $R^2$ value, as for linear regression, wont work anymore. Why is that?
1
vote
0answers
39 views
Anderson-Darling test: p-value reliability when testing fitted distribution
I have a question regarding the A-D test, and perhaps goodness-of-fit tests altogether. I fitted a dataset to a long list of distributions. According to A-D, a Wakeby distribution provides the closest ...
0
votes
0answers
15 views
Compare goodness of fit for GAMs and (multiple) ordinary least squares
I'd be super grateful for any help. I have a dataset with a continuous response variable and various predictors. I want to fit two models.
A "simple" linear multiple least squares regression....
3
votes
1answer
67 views
Log transformation in GLM and model fit
For a negative binomial GLM, are we allowed to write the log transformation in the following way?
...
0
votes
0answers
17 views
Goodness of fit methods for density estimation
If we want to estimate the probability distribution function (pdf) of finite-sampled real continuous data using one of the following approaches:
Parametric density estimation: fit a well-known ...
2
votes
2answers
143 views
Regression and the CEF
I recently read in this page (https://www.timlrx.com/2018/02/26/notes-on-regression-approximation-of-the-conditional-expectation-function/#fn1) that:
"Regression offers a way of approximating ...
0
votes
0answers
28 views
Nice fitted values but lack of fit or vice versa?
What would you do if you had two Negative Binomial models (say 1 and 2) where Model_1 has nice fitted values but higher Pearson residuals (and the overdispersion ratio, that is Residuals over DF, ...
1
vote
1answer
28 views
Can anyone tell me more about this Prediction quality measure, Prediction Accuracy, used for regression evaluation?
I am trying to find more info about the attached Prediction Accuracy measure used for regression. It is quiet similar to R2 and Nash-sutcliffe Efficiency but not exactly. Googling leads to ...
3
votes
0answers
46 views
How to compare model fits between datasets of different sizes?
A measure like mean square error appears to tolerate some variation in dataset size but, for example, a two-parameter model is still likely to produce a great (yet meaningless) fit to any dataset with ...
1
vote
0answers
20 views
Is the repeated G-test of goodness-of-fit the appropriate statistical analysis?
I made observations of animal social behavior in which I categorized interactions as either competitive, cooperative, or neutral (i.e. one nominal variable with three possible outcomes). I want to ...
0
votes
1answer
20 views
Is a reversed path analysis a nested model? (SEM)
I'm trying to compare whether a forward/direct path analysis is a better fit to the same data than a reversed model. I'm using the SEM function of the ...
0
votes
0answers
16 views
How do I interpret model fit for ordinal regression when AICc and likelihood ratio test conflict?
I'm working with 4 nested models using ordinal regression (same sample, n=344, and dependent variable across models). The -2LL for each successive model increases and becomes statistically significant....
