Questions tagged [goodness-of-fit]
Goodness of fit tests indicate whether or not it is reasonable to assume that a random sample comes from a specific distribution.
945
questions
0
votes
0answers
6 views
Goodness of fit of a model with N*M responses
I have a model that predicts human responses. Both the model and the human classify some objects with confidence ratings and the output is in a form of a N*M table, e.g.:
...
0
votes
0answers
34 views
Why is this method used to find out whether a Poisson distribution is appropriate?
I have a problem which is concerned with figuring out whether a Poisson distribution is a suitable model or not for a certain scenario.
"The distribution of the length of the first word of each ...
1
vote
1answer
16 views
Goodness of fit test for LASSO
How would you do a goodness of fit test for Lasso regression?
Im guessing that the $R^2$ value, as for linear regression, wont work anymore. Why is that?
1
vote
0answers
25 views
Anderson-Darling test: p-value reliability when testing fitted distribution
I have a question regarding the A-D test, and perhaps goodness-of-fit tests altogether. I fitted a dataset to a long list of distributions. According to A-D, a Wakeby distribution provides the closest ...
0
votes
0answers
10 views
Compare goodness of fit for GAMs and (multiple) ordinary least squares
I'd be super grateful for any help. I have a dataset with a continuous response variable and various predictors. I want to fit two models.
A "simple" linear multiple least squares regression....
2
votes
1answer
58 views
Log transformation in GLM and model fit
For a negative binomial GLM, are we allowed to write the log transformation in the following way?
...
0
votes
0answers
15 views
Goodness of fit methods for density estimation
If we want to estimate the probability distribution function (pdf) of finite-sampled real continuous data using one of the following approaches:
Parametric density estimation: fit a well-known ...
1
vote
2answers
50 views
Regression and the CEF
I recently read in this page (https://www.timlrx.com/2018/02/26/notes-on-regression-approximation-of-the-conditional-expectation-function/#fn1) that:
"Regression offers a way of approximating ...
0
votes
0answers
28 views
Nice fitted values but lack of fit or vice versa?
What would you do if you had two Negative Binomial models (say 1 and 2) where Model_1 has nice fitted values but higher Pearson residuals (and the overdispersion ratio, that is Residuals over DF, ...
1
vote
1answer
27 views
Can anyone tell me more about this Prediction quality measure, Prediction Accuracy, used for regression evaluation?
I am trying to find more info about the attached Prediction Accuracy measure used for regression. It is quiet similar to R2 and Nash-sutcliffe Efficiency but not exactly. Googling leads to ...
2
votes
0answers
17 views
How to compare model fits between datasets of different sizes?
A measure like mean square error appears to tolerate some variation in dataset size but, for example, a two-parameter model is still likely to produce a great (yet meaningless) fit to any dataset with ...
1
vote
0answers
20 views
Is the repeated G-test of goodness-of-fit the appropriate statistical analysis?
I made observations of animal social behavior in which I categorized interactions as either competitive, cooperative, or neutral (i.e. one nominal variable with three possible outcomes). I want to ...
0
votes
1answer
19 views
Is a reversed path analysis a nested model? (SEM)
I'm trying to compare whether a forward/direct path analysis is a better fit to the same data than a reversed model. I'm using the SEM function of the ...
0
votes
0answers
13 views
How do I interpret model fit for ordinal regression when AICc and likelihood ratio test conflict?
I'm working with 4 nested models using ordinal regression (same sample, n=344, and dependent variable across models). The -2LL for each successive model increases and becomes statistically significant....
0
votes
0answers
13 views
Goodness of fit of stochastic differential equation
I have a discrete time series and I fit Ornstein-Uhlenbeck sde to it using "R" "sde" package. It gives me parameters of sde and likelihood. How do I know if data fits well OU sde? ...
5
votes
1answer
115 views
Why is the chi-square test more popular than the G-test?
Pearson's chi-square test and the G-test are two goodness-of-fit hypothesis tests for categorical data -- i.e., testing whether a sample came from a given distribution on a finite set. The respective ...
0
votes
0answers
70 views
Chi-squared test, Poisson distribution, type I error overestimated - well-suited test for discrete distributions?
UPDATE I edited my original question to make it as clear as possible.
My goal is to find a reliable goodness-of-fit test for Poisson-distributed samples. There are a few discussions here related to ...
1
vote
0answers
18 views
Can you use a test statistic like Anderson–Darling for parameter estimation?
I was looking at the Wikipedia article for maximum spacing estimation and this got me thinking. The idea behind this method is that if you know the true distribution of the data, then its CDF should ...
1
vote
1answer
32 views
goodness-of-fit and bootstrap
Assume one has two data samples: $X = \{ x_{1}, \dots, x_{n} \}$ and $Y = \{y_{1}, \dots, y_{m}\}$. Next, we aim to check if the data $Y$ was generated by the same data generating process (DGP) as $X$ ...
2
votes
1answer
39 views
Is it possible to take the Root Mean Square Error of a continuous function?
I am familiar with the Root Mean Square Error (RMSE) of discrete data: $$\text{RMSE} = \sqrt{\frac{1}{N} \sum_{i = i}^N |{\hat{y_i} - y_i}|^2}$$ where $\hat{y_i}$ are "predicted" or measured ...
3
votes
0answers
15 views
Is there any anova-like approach for calculating contingency tables across multiple levels within a factor
I want to compare success rates across a large number of different levels within a third factor to detect if there are statistically significant differences for at least one of the groups. I'm not ...
0
votes
0answers
26 views
p-values not uniform in KS simulation with estimated parameters
I want to perform a bootstrap/simulation of the significance of a KS test in some observed data and have come up with some synthetic examples to make sure my code works, which I base off Greg Snow's ...
0
votes
0answers
31 views
Using chisq.test() to compare two exponential distribution variables in R
I want to use the chi-square test to judge if the variable X follows the exponential distribution.
My plan is:
1] generate the X
2] using the rexp() to generate an exponential distribution vector
3] ...
0
votes
0answers
8 views
What is the practical way to test for goodness-of-fit of alternative curves?
I have fitted (that is, I have found the best parameters) two curves (Holling type III and Gomperz) to the data (red dots) and obtained the following regression:
How can I say which one is a better ...
2
votes
0answers
19 views
Meaning of adjusted correlation coefficient
It is well-known that for simple linear regression, $R^2 = \rho^2$, where $$R^2 = 1-\frac{\sum (y_i-\hat{y}_i)^2}{\sum (y_i - \mu_y)^2}$$
and $$\rho = \frac{\sum (y_i-\mu_y)(\hat{y}_i-\mu_{\hat{y}})}{\...
0
votes
1answer
32 views
t test vs goodness of fit test
Suppose I have two groups of data. I want to know if their distributions and / or means are different or not. Is it appropriate to do a two-sample t-test? Or should I go for a 2 sample goodness of ...
0
votes
1answer
20 views
Is it possible to calculate a p-value from a modified test?
I was asked to fit a distribution to some data and calculate the goodness-of-fit and a corresponding p-value. I've been using Pearsons chi-squared test (chi2gof in matlab) to do this but my advisor ...
1
vote
2answers
38 views
Alternative to R$^2$ in linear regression without intercept
It has been extensively described in this website the reason why one cannot properly calculate the $R^2$ - neither the Adjusted $R^2$ - in regression models fitted without an intercept. What is a good ...
1
vote
1answer
45 views
GARCH diagnostics: autocorrelation in standardised residuals and poor results of Goodness-of-Fit Test
I am trying to fit best ARMA - GARCH model using rugarch in Python on financial data 5 min returns series. I am using last 10k observations for this purpose. The goal is to predict next return and its ...
0
votes
0answers
23 views
Simulating p-value statistics for Liliefors test (Python statsmodels)
Python statsmodels has an implementation of Lilliefors' test for goodness of fit (i.e. if the parameters of the distribution were obtained from fitting the data and not per-determined as in the ...
1
vote
1answer
42 views
Can an interaction term be justified with substantive theory alone?
Let's say we have immense literature theoretical justification to expect that X1 predicts Y, even though X1 contributes only slightly to explaining variation in Y. We want to know if X1 is more ...
1
vote
0answers
14 views
How would I assess goodness-of-fit considering single participants?
I am designing a psychology experiment which is designed to look at the proportion of different types of errors people make.
There will be four response categories: correct; incorrect [type 1; type 2; ...
0
votes
0answers
9 views
How to interpret big RMSR despite of all other EFA indicators being OK?
During the EFA, I have Tucker Lewis Index of factoring reliability higher than 0.93, RMSEA index at 0.08, and CFI at 0.95 while my RMSR is 0.16.
How come my CFI is acceptable while RMSR is ...
0
votes
0answers
16 views
Fitting a distribution to my timeseries: two R-packages, two contrary results
typical R-User here, applying a bunch of packages to my data, hoping for a convincing result although I understand only half of what I do at most (and then getting rejected by the reviewers, surprise)....
0
votes
4answers
57 views
Accounting for uncertainty in evaluating a forecast
I'm new to forecasting and I have some (probably very) basic and generic questions. I'd appreciate some references that get into details of this too.
Using some model to forecast a time series, I ...
2
votes
1answer
51 views
Goodness of fit for a logit-transformed linear random-effects model?
(After re-reading my question here, I realize my notation is a mess... apologies. I hope the question is clear enough.)
There is an examination that students (indexed by $i$) can take once annually in ...
1
vote
1answer
28 views
3
votes
1answer
54 views
bootstrap KS 2 sample test
I have two datasets $S_1$ and $S_2$. I can run KS 2 samples test on these datasets to obtain the value of the KS 2 samples test.
Is there a correct way to bootstrap the KS 2 sample test using the ...
0
votes
1answer
26 views
Wrong answer in basic goodness-of-fit test
I am following my lecture notes on this test:
However, when I calculate the expression $2\log \Lambda$ (Python script attached below), I get $21.8$ instead of $44.9$, which is quite far off. The ...
5
votes
2answers
75 views
Can I use the Kolmogorov-Smirnov test with estimated parameters?
If I estimate the parameters of a distribution $\theta$ using MLE. Can I use the Kolmogorov Smirnov test to check the goodness of the fitted model?
0
votes
1answer
33 views
Identifying the best distribution to this data?
I'm trying to fit an appropriate distribution to a data with 216 values and estimate parameters. From Cullen and Frey graph, it looks like lognormal could be a good fit. From q-q plot, Weibull seems ...
1
vote
1answer
37 views
Standardized GARCH-residuals, distributions and AIC
So, I have been wondering about an interesting observation. My data contains 1006 log-returns of the SP500-index and I've estimated a GARCH(1,1)-process with Gaussian quasi-maximum likelihood - ...
2
votes
1answer
40 views
Is the cross-validation a goodness-of-fit test?
I read the Wikipedia article on the Goodness of fit(GOF) and it explains GOF as
The goodness of fit of a statistical model describes how well it fits
a set of observations. Measures of goodness ...
5
votes
1answer
173 views
Understanding why a $p$-value is too small
I have a data set with counts of particles, and I want to test if they follow a distribution. For a certain species, I make the $\chi^2$-test, and everything seems reasonable, finding a $p$-value of $...
0
votes
0answers
37 views
Testing the fit of polychoric correlation
I’m using polychoric correlations for my work. According to Kampen and Weeren (2017):
In order to prevent this questionable research practice, it is recommended that in applications of the ...
0
votes
0answers
30 views
Assessing CFI in Exploratory factor analysis (EFA) in R?
I'm currently conducting EFAs to analyse the internal structure of a construct. I've found a nine factor model with acceptable fit, but only regarding to RMSEA and SRMR. I'm using the program R to ...
1
vote
0answers
27 views
What is the motivation for using a second order approximation for KL divergence in this example? [closed]
Consider the KL divergence between two discrete distributions P and Q:
with probabilities $p_1,...,p_k$, and $q_1,...,q_k$
$I^{KL}(P;Q)= \sum^{k}_{i=1}p_ilog\frac{p_i}{q_i}$.
The notes then say ...
0
votes
0answers
7 views
Goodness of fit test as a random variable
I have two datasets coming from two different distributions and I want to apply goodness of fit on these two datasets to check if they are coming from the same distribution. For the sake of argument, ...
0
votes
0answers
10 views
pureErrorAnova(alr3 package on R) would work for two-way factorial anova
I have a balanced 2x5 anova factorial which 4 replicates each group. I'm trying to check the Lack of fit and I came accross the pureErrorAnova - alr3 package (Im using R software). My independent ...
0
votes
1answer
26 views
Why does the Kolmogorov distribution table have entries for $n$?
In my textbook it is said, that if the hypothesis of same population distributions is true, then $\sqrt{n}D\rightarrow K$ in distribution, where $D$ is the Kolmogorov-Smirnov statistic, and $K$ is ...
