Questions tagged [goodness-of-fit]
Goodness of fit tests indicate whether or not it is reasonable to assume that a random sample comes from a specific distribution.
862
questions
0
votes
1answer
21 views
SEM: High SRMR in model with binary predictor. Other fit indices OK [closed]
In this post I may display severe lack of understanding in SEM and the math behind it so please be kind.
I am testing the following structural model with lavaan using MLM (due to non-normality) with ...
2
votes
1answer
62 views
How to choose and perform a goodness-of-fit test?
I notice there are a lot of questions on the topic (most are along the lines of: how to do a Chi-squared goodness-of-fit test), but mine is perhaps more general.
Using an example I recently ...
0
votes
1answer
33 views
How do I get the sample mean and standard deviation? [duplicate]
I am trying to get the mean and standard deviation for the following sample means with the given sample frequencies for each mean. So in a histogram, the x-axis will show the mean range and sample ...
1
vote
0answers
17 views
Degree of freedom of different distributions: Chi-Square Goodness of Fit Test
I try to understand why the number of constraints differ for the distributions in the table below, for the Chi-square goodness of fit test. I found the content in the following link.
https://www....
2
votes
0answers
35 views
Over “specification” of a statistics model?
For a simple example, I am fitting data with a likelihood function generated by a normal distribution. The first model is the normal distribution with two-parameters. The second competing model is the ...
0
votes
0answers
13 views
Anderson-Darling test Critical Values for Log Logistic (asymptotically using Monte Carlo Sim)
I am trying to to compute the AD test critical values for a Log Logistic distribution (not a normality test, but an EDF) . I am trying to do a Monte Carlo simulation in R.
The steps are the following:...
1
vote
0answers
29 views
Goodness of fit ranking for several models
I have some data that I am trying to fit with different models: exponential, sum of two exponentials, hyperbolic secant, etc.
I would like to have a criteria to be able to rank the different models ...
1
vote
1answer
33 views
Goodness of fit for Copula
I was wondering if the graphical methods can be used instead of formal tests such as Cramer-Von Mises test as the GOF for copulas?
The scatter plot of pseudo-observations is as follows:
The Q-Q ...
2
votes
0answers
13 views
Goodness of Fit Test vs Testing RMSE
I have a bit of a broad question. It seems to be that there are two different approaches (Borrowing a bit of Breiman's Two Culture's paper) when it comes to testing if our data looks like our model.
...
3
votes
2answers
138 views
Non-significant result in CFA model
I've fitted a one-factor model to data originating from a unidimensional 8-item scale; sample size is 400. Because the scale uses a Likert response format, and data does not follow a multivariate ...
1
vote
0answers
14 views
ERGM with larger log-likelihood but worse fit than reference model?
I'm using the R package statnet to fit some ERGMs to the Faux Dixon High simulated network data provided with the package. The first model I fit is almost identical ...
0
votes
1answer
26 views
How to assess the quality and goodness of fit of a lasso model?
So far I evaluate the lasso model by its out-of-sample predictions and by the $R^2$ for the goodness of fit. What other measures should I look at? Particularly, do I need to examine the residuals for ...
2
votes
0answers
51 views
Different loss functions for estimation and cross-validation
Assume that the goal is to estimate some parameter $\theta$ (finite or infinite dimensional) based on some data available. Also, assume that there are other nuisance parameters are present in the ...
1
vote
0answers
32 views
Fish counts and Poisson [closed]
I am having troubles with fitting a Poisson distribution to my data, let me explain:
I have fish counts of different species from a closed list, at 5 sites, 2 different depths, and across 10 years. ...
1
vote
0answers
16 views
Does the large number of sample is the reason for low p-value in GOF test of Copula? (Cramer_Von mises)
I would like to determine a suitable copula function on my data.
The multivariate computed from the original data is similar to the Plackett.
The GOF test on my data with size 8000, reject all ...
0
votes
0answers
18 views
Posterior Predictive Check for Hierarchical Logistic Regression Model
I need to apply Posterior Predictive Check (PPC) on Hierarchical Logistic Regression Model (so I have binary outcome) to validate my model (to see goodness of fit of my model). I know that I need to ...
2
votes
1answer
76 views
Limitations of Kolmogorov–Smirnov test
In this article, it is said that the test statistic tends to be more sensitive near the center of the distribution than at the tails. Can some please explain this what it means? (with a simple example ...
2
votes
0answers
29 views
Why do I get different results of goodness of fit test using AD tests from different R libraries?
I am trying to find the best method for goodness of fit while testing my data with different models of distributions. I need to use AD test. So I have compared results using two different ...
1
vote
0answers
39 views
p-value fitting a model to data
I have performed a physical experiment with few (~20) measurements, which have different uncertainties on the y-axis. The measurements were fitted to an exponential model $f(x)=ae^{-bx}+c$ using scipy....
0
votes
1answer
30 views
How to derive the formula for coefficient (slope) of a simple linear regression line? [duplicate]
There is a formula for calculating slope (Regression coefficient), b1, for the following regression line:
y= b0 + b1 xi + ei (alternatively y'(predicted)=b0 + b1 * x); which is
b1=(∑(xi-Ẋ) * (yi-...
0
votes
0answers
22 views
IRT degrees of freedom of item fit statistics
I'm fitting Rasch models with the mirt package. Upon measuring item-wise $Q_1$ using mirt::itemfit(model, fit_stats="X2"), I ...
1
vote
0answers
15 views
ugarchfit diagnostic tests: goodness of fit
I modeled a couple of financial return time series with different kinds of GARCH-models and different distribution termns and now whant to compare them. I very often face the problem that all ...
0
votes
0answers
41 views
Why Lack of fit test is significant?
I fitted a dose-response curve and part of my data is significant for Lack-of-Fit test, but I don't understand why? If visually the model has a good adjustment.
I know the Lack-of-Fit test is not a ...
-2
votes
2answers
46 views
How to confirm more people selected “Agree” than is possible by chance [closed]
In the table below, how do I prove statistically that more people are in the group: "Sum of Agree", than is possible by chance?
I do not know which test to use here, (although I am using the Extended ...
1
vote
1answer
25 views
I would like to find Goodness-of-fit between my differential equation simulation result and observed field data. How to find it?
I modelled a ecological system using ordinary differential equation. Simulated the model for 12 months. I also have 12 months of field data. Now I want to find the goodness of fit between my model ...
0
votes
0answers
10 views
Including probabilistic information in the model fit assessment of multivariate nominal regression
I have a ground truth nominal variable $Y$ and a continuous predictor variable $X$. I have split these up into a training dataset, $Y_1$ and $X_1$, and a testing dataset, $Y_2$ and $X_2$. I want to ...
1
vote
1answer
69 views
Difference between QQ plot and KS test
While solving some exercises for HW, I came up to use QQ plot and KS test for the sake of testing goodness of fitting, where I also came up with the following doubts:
Precisely, let $X, Y \sim \text{...
0
votes
0answers
17 views
Using the sum of the absolute residuals divided by the sample sum as a normalized goodness-of-fit estimator
I am trying to make an automatic method to do a peak fitting to some spectra, using multiple least squares. I would like to be able to compare the quality of the fit between several different cases. ...
2
votes
0answers
64 views
Random forest [R]: why is my OOB RMSE so much smaller than test RMSE?
I'm doing the kaggle challenge on timetravel predictions where the task is to predict the duration (Y) of a uber trip given some information about the start and end coordinates and the time the trip ...
0
votes
0answers
8 views
Generalization performance of a model (RStudio)
I'm kind of confused on the definition of generalization. E.g. I've created a model in RStudio, for which I've plotted the training history results:
Now I'm wondering, is the generalization of my ...
2
votes
0answers
73 views
How to check that quantile regression forests provide valid predictions?
I am using quantile regression forests to predict the distribution of a measure of performance in a medical context. I am using the ranger R package for that purpose.
I would like to have advices ...
3
votes
1answer
54 views
What is a lay explanation for the numerator of W in the Shapiro-Wilk test?
I am trying to understand in basic terms what the Shapiro-Wilk test is doing, but the math in the original 1965 paper is beyond me. (For starters, why is the covariance matrix n x n? What do the $v_{...
0
votes
0answers
23 views
Data generation under Pitman alternatives
I would like to numerically simulate the power function of the goodness-of-fit test $T_n$ in https://projecteuclid.org/download/pdf_1/euclid.aos/1176348133 for a set of certain probability ...
1
vote
2answers
50 views
Is it possible to define an optimal fit?
Let assume that we have n pairs of real numbers:
(x_1, y_1), (x_2, y_2), ..., (x_n, y_n)
Let as also assume that ...
2
votes
1answer
95 views
chi squared goodness of fit test to check for normality
I have the following data:
I would like to use the chi-squared goodness of fit test to test whether it comes from a normal distribution. How shall I go about it?
In particular,
I would like to know ...
0
votes
0answers
16 views
What is considered as a good fit using the fraction deviance?
I am using the fraction deviance given by Hastie et. al (2015),
$\mathcal{D}_{\lambda}^{2}=\frac{\mathcal{D}\mathrm{ev}_{\mathrm{null}}-\mathcal{D}\mathrm{ev}_{\lambda}}{\mathcal{D}\mathrm{ev}_{\...
0
votes
0answers
8 views
Questions Tags Users Unanswered Connection of the critical value for KS distance and model complexity
I asked a similar question is math exchange, but here it seems to fit better.
I wonder about a relationship of the critical value for the KS distance (with Lilliefor correction) and model complexity? ...
0
votes
0answers
34 views
MAPE vs R Squared, and their Interpretation in Regressions
So, I'm somewhat familiar with R squared and MAPE, and their usage to qualify the goodness of fit for a regression model. My question is not about when to use one or another (like in this post) but ...
0
votes
0answers
52 views
How to validate a simulation model
Apologies in advance if this question is more on the conceptual side, but hopefully it's still appropriate for here:
For context, there's a tradition in political science of building simulation ...
0
votes
1answer
32 views
Chi-square Goodness of fit with specific expected values
Let's say I have the following summary data/observed counts.
...
1
vote
1answer
28 views
Comparing survey data to passively measured data
Basic Situation:
I have quite a large sample of around 10k people.
In that sample I have two values for each person: a passively measured value of an amount of time, as well as their estimation of ...
0
votes
0answers
31 views
Measures of goodness of fit, to compare different fitting models and routines
I realise that there is no catch-all method of assessing and quantifying goodness of fit -- otherwise we'd all be using and it and it would be well known.
In my undergraduate studies I was always ...
3
votes
0answers
28 views
A goodness of fit test for two discrete distributions with unequal variance?
I have a computer simulation in which a virtual agent end up in different areas of a layout based on several factors. There are 18 conditions, so the data (you can find the csv file for a toy dataset ...
1
vote
1answer
57 views
Screening candidate models before AIC comparison?
I am interested in identifying the best of 3 physiologically reasonable models that fits my continuous data. Data is some measure derived from neurons recorded from 3 adjacent regions of brain tissue (...
0
votes
0answers
13 views
Fitting nonlinear numerical model
I am reading a scientific paper that fits a numerical model to a timeseries (observations). The numerical model has a few free parameters, some of them linear and some are not, which the authors wish ...
1
vote
1answer
16 views
How to control for a numeric variable when doing a chi square goodness of fit test? [closed]
I'm looking at race differences in the number of adverse events that happen at a hospital. At this hospital there are a lot more white people in general so I've been using a chi square goodness of fit ...
0
votes
0answers
23 views
How do I determine goodness of fit for a test of n binary decisions?
Let's say I have $n$ pairs of items. For each pair I select one item over another due to some perceived trait. This could be anything, for example, identifying which image is brighter than the other ...
0
votes
0answers
10 views
How to do model fit if the Hosmer Lemeshow GOF is less than 0.05
I have a data set that the response variable is count data, with other predictive variables all categorical.
I used the stepwise logistic regression to try to find the best model fit.
I used the code ...
1
vote
2answers
52 views
Creation of contigency table and calculation of effect size (Chi2, Cramer's V)
I am a bit unsure in my approach of applying the Chi2 GOF test to my data and calculating the correct effect size.
The situation:
I am studying the occurrence of specific biological phenomena in ...
3
votes
1answer
81 views
$R^2$ can be negative, but can it be -100% or lower? [closed]
I saw an $R^2$ statistic being reported as $-115\%$. I thought $R^2$ must be between $[-1, 1]$ so can it be negative to beyond $-1$?
