Questions tagged [goodness-of-fit]
Goodness of fit tests indicate whether or not it is reasonable to assume that a random sample comes from a specific distribution.
1,077
questions
1
vote
1
answer
13
views
Comparing goodness of fit across parametric and semi-parametric survival models
I've been learning about time-to-event analysis and playing with open datasets + fitting various Cox and parametric models for practice.
Other than by visually inspecting the estimated survival curves ...
- 13
2
votes
1
answer
27
views
Validating random variable generation from inverse transform sampling
I'm building a simulator and I have to implement some probability distributions. What is the best (formal) way of validating this implementation? I took a look at KS-tests but it seems to me they are ...
1
vote
0
answers
22
views
How to compare the goodness of fit between linear and logit? Why linear deviance is less than logit?
How can I evaluate which model - between linear and logit - determine the best fit to the data?
The models use the same input variables and I thought that comparing the deviances was the proper choice ...
- 11
0
votes
0
answers
39
views
Chi square of 0.00, but acceptable (other) fit indices?
I am running a path analysis for a sample of 75 participants using RStudio. I am comparing two models and have to decide which one to use (I am in favor of Model A, but because of the Chi-square: 0, I ...
0
votes
1
answer
18
views
Chi square fit, fitting data to a straight line: Incomplete gamma function as goodness-of-fit estimation
On page 661 of the book "Numerical Recipes in C:The art of scientific computing", we fit a set of N data points
$(x_{i}, y_i)$
to a straight line model
$$y(x)=a+bx.$$
We do this by ...
- 11
0
votes
0
answers
29
views
Does $R^2$ measure goodness of fit or not? [duplicate]
I have been warned against $R^2$ as goodness of fit before, but am unsure why. What is wrong with using it to characterize how well a line fits data points? I have looked at a few sources to try and ...
- 13
0
votes
0
answers
8
views
Goodness of fit for multinomials with small np
I've got a model with continuous explanatory variable $x_i$ and some positive integer $m_i$ for each datapoint, and then a multinomial response - i.e. $p$ categories, and the sum of counts across the $...
- 1
0
votes
0
answers
18
views
Measure of goodness-of-fit in errors-in-variable regression
I have two observed time series $x_i$ and $y_i$ and I want to test if $x_i$ is a good predictor of of $y_i$. So I would usually run a simple linear regression Y ~ X and use $R^2$ as a measure of ...
- 1
0
votes
0
answers
19
views
Smoothing time series with Adjusted R2-weighted averages
I have two parameters (a,b) resulting from an exponential estimation of a curve. I have estimated this curve every hour for one month. In other words, I have a total of 720 parameters a and b, and I ...
0
votes
1
answer
101
views
Extension of Pearson's chi-squared test to sequences of multinomial random variables
Background
Suppose we observe $n$ IID Bernoulli variables and our null hypothesis is that their common probability is $p$.
For denote by $\mathbb{1}_{\{i\}}$ the outcome of observation $i$.
Then by ...
- 29
0
votes
0
answers
19
views
Weighting the R-squared as a measure of goodness-of-fit in Linear Regression [duplicate]
I have two observed time series $x_i$ and $y_i$ and I want to test if $x_i$ is a good predictor of of $y_i$. So I run a simple linear regression Y ~ X and use $R^2$ as a measure of goodness of fit.
...
- 1
0
votes
0
answers
7
views
Evaluate the goodness of a fitted continous distribution in the case of ties (Used later in DES simulations)
I have continous data which shows the time in minutes, a process step takes.
Unfortunatelly, it is not possible to get data without rounding so I have many ties.
As I thought, a random process should ...
- 139
1
vote
0
answers
28
views
Why are my P-Values for the scipy kstest on a uniform distribution so high?
I'm using the Scipy implementation of the Kolmogorov–Smirnov test to check whether collections of random values are likely to have been drawn from a uniform distribution.
From what I understand, the ...
- 111
0
votes
0
answers
15
views
ERGM with missing tie data: Interpretation of the mcmc.diagnostics regarding autocorrelation
I am running an ergm model on network data that contains some missing tie variables. I am running this model to later use it to impute the missing data. I am currently investigating the mcmc....
0
votes
0
answers
43
views
How to compare distributions with errors on the data points?
Here's a mock set-up of my problem:
I have two non-normal probability density distributions (PDFs), $A$ and $B$.
Distribution $A$ has error measurements for each data point while distribution $B$ ...
0
votes
0
answers
9
views
Does the Bollen-Stine procedure in AMOS affect fit indices?
My colleagues and I are working on a cross-cultural study and we attempt to obtain evidences of factorial validity by doing a measurement invariance analysis. Since our data failed to meet the ...
0
votes
0
answers
7
views
Different tests for model fit
I saw that in linear regression models one often uses a hypothesis z- or t-test for $R^2$ or for effect sizes. A z-Test is only useful if the standardized $R^2$-values are standard normally ...
- 11
0
votes
1
answer
25
views
How to correctly perform a goodness-of-fit test for a contingency table (two-way, three-way, or more), in situations other than independence testing?
Let's say I have the following table from a sample of 462 people:
Gender
Happy
Meh
Sad
Men
70
32
120
Women
100
30
110
I don't want to test it against the hypothesis of independence, but against ...
- 127
0
votes
1
answer
14
views
Which model fit indices do you report from lavaan cfa for scale development?
For RMSEA there seems to be a regular value and two robust values. Are there any recommendations on which of the three values to report? Would you report 1, 2 or 3 from lavaan?
- 1
0
votes
0
answers
18
views
Time Series with Possible Discontinuity - Is the discontinuity significant?
I am just beginning to learn about statistics and statistical tests, so please forgive me if I'm using entirely the wrong terminology or I've missed something obvious.
I'm trying to solve a problem ...
- 1
1
vote
1
answer
51
views
How to compare results of a discrete (binary) model to a probabilistic model?
Let's suppose I have two models that both indicate the presence of some phenomenon:
Model A: Only binary results, i.e., the phenomenon is present or not,
Model B: Outputs class probabilities.
Of ...
- 93
0
votes
0
answers
21
views
Generating random data points following a Poisson point process from observed data points [duplicate]
I am a bit new to the domain of Spatial Statistics. So I am trying to generate complete spatial randomness(CSR) data with summary statistics similar to that of the observed data(data points in 2D). ...
0
votes
0
answers
37
views
Fit indices showing "na" in lavaan path analysis using DWLS
I am doing a path analysis using lavaan in R. I have two models of interest:
Model 1.
Model 2.
As one of the variables in the model is ordinal, I specified the ordered="Y2" command when ...
- 55
1
vote
2
answers
88
views
Distribution comparison by AIC
I'd like to compare several distributions fitted to one dataset (of i.i.d. random variables) by AIC. Do there exist some specific rules of thumb for such a situation?
It seems that most of such rules ...
- 201
2
votes
1
answer
68
views
Testing how well a sequence of observed word game solutions corresponds to expected word frequencies
Say we have a word game where each round involves finding a unique 5-letter word solution. (Wordle would be an example, for those familiar). For example, we may have a round where the word "magic&...
- 73
0
votes
1
answer
40
views
Regression curve with lowest combined AIC and BIC is a poor predictor
For fun, I was trying to make a predictor for how long it would take for George R. R. Martin's The Winds of Winter to be released. My "best" model is the one that had the lowest combined AIC ...
- 121
9
votes
3
answers
808
views
Is it reasonable to use the Kolmogorov-Smirnov test to assess the normality of a random variable?
Context
I am confused by the following post where the accepted answer states that :
You can't really even compare the two since the Kolmogorov-Smirnov is
for a completely specified distribution (so ...
- 357
1
vote
1
answer
95
views
Goodness of fit for Poisson regression with aggregated data
This a folow-up of this question Independence in Poisson regression when used for rates estimation
I have a set of thousands of observations. Each relates to an individual, and for eachI have the date ...
0
votes
0
answers
10
views
How to evaluate a GAM that predicts temperature over a certain extent with actual measured data?
I have a GAM that predicts the temperature on a relative scale (0 = coldest, 1= hottest value) of a raster extent. Secondly I have measured data from the same spatial extent that I can transform to ...
- 1
1
vote
2
answers
56
views
Goodness-of-fit tests for discrete distributions
I have data where only values at large x should fit to a particular distribution whose parameters I wish to determine. I want to do a goodness-of-fit test to find the value of x where the data fit to ...
- 121
0
votes
1
answer
29
views
model selection in linear mixed models
I have a question about finding the fittest model (lmer). What I've learned is to remove the non-significant terms from the full model until none of the terms can be removed. Then consider the high-...
- 35
0
votes
0
answers
32
views
Comparing fit of time series to true process
I know that this question have been asked many times in this forum, but I am having troubles in understanding the correct approach to my aim.
I have several time series representing a growth process ...
1
vote
0
answers
37
views
Negative F-test value comparing two nested models
I am comparing two non-linear nested models - let me call them model A and model B. Model B has one parameter more than model A, i.e. model A can be obtained as a special case of model B. These two ...
- 11
2
votes
1
answer
96
views
Is this a reasonable way to determine the reliability of a fit?
Background
I have measurements of a trajectory that is parameterized by time. The data consists of points with two spacial coordinates $(\tilde{x}_i, \tilde{y}_i)$ and a time stamp $(t_i)$. I'm using <...
- 105
0
votes
0
answers
65
views
R: Assumptions for GLM model (binomial family, logit link)
I want to explain the absence or the presence of a task by the condition of my participants
Response variable: "Task" = 0 or 1
I use a GLM mixed model ( because of repeated measures):
...
- 457
0
votes
0
answers
17
views
How to measure the goodness of fit for a multivariable nonlinear math model
How measure the goodness of fit of my model?
c1, k1...kn, a1, o1,o2,o3 are the coefficients, while t and x1...xn are the parameters. I may change the model though.
0
votes
0
answers
43
views
What is the formula to calculate Ryan-Joiner test statistic
I would like to know the formula to calculate the Ryan joiner test statistic and then on basis of that how to calculate the corresponding p-value. I am trying to replicate the result of Minitab in ...
1
vote
0
answers
31
views
How to combine test statistics and p-values from the same (stochastic) test repeated on a single dataset?
I would like to assess a generative model $M$'s goodness-of-fit using a simulation-based approach: I simulate some data from $M$, then compute a test statistic $T$ that measures how similar the ...
- 11
3
votes
0
answers
84
views
Question about statement in Oster (2019): variation in a control
In Oster (2019), she discusses how authors typically include controls and examine coefficient stability as a way to test for presence of confounding, and points out that researchers should consider ...
- 385
1
vote
2
answers
33
views
Why can we draw a more precise conclusion when we choose a lower accepted-risk in this hypothesis-testing setting, which seems contradictory?
We want to know if 100 integer values (in a vector X) are following a Poisson $P(\lambda=2)$ distribution, which is our $H_0$ hypothesis.
Let's say the observed ...
- 285
0
votes
0
answers
22
views
0
votes
1
answer
108
views
Can AIC be used to select the best model with multiply imputed data (MICE)?
I have used the mice() package in R to impute some missing values and create a pooled linear regression model. I have also created another version but this time ...
Jameshbj9
0
votes
0
answers
22
views
How can I determine best fit for a time series decomposition? [duplicate]
I’m new to time series decomposition and have been doing Seasonal and Trend decomposition using Loess (STL) in R.
My data is reported monthly and it appears that seasonality is annual. From what I’m ...
- 131
1
vote
1
answer
76
views
Goodness of fit of a distribution obtained by minimizing a log-loss function
I am trying to fit a log-normal distribution to a time-to-failure data of a product, but the data to which I want to fit the distribution is not regular data. In the data, every row $i$ has two pieces ...
1
vote
0
answers
46
views
Goodness of fit of two models
My two models that I would like to compare are Weibull distribution and Quantile regression (in the case of Weibull distribution I have estimated the quantile curves). I would like to know if there is ...
- 161
0
votes
0
answers
16
views
Cramér–von Mises criterion vs Goodness of Fit
I had the following question about the Cramér–von Mises (CVM) criterion vs Goodness of Fit
As described above, it seems like the CVM test measures how close the empirical probability distribution ...
- 5,688
0
votes
0
answers
16
views
GOF test for logistic regression with only categorical predictor/grouped data
I am working on logistic regression with only categorical (grouped data) variables. My sample size is quite small (N = 60) with N=40 cases (with disease) vs N=20 (non-disease). I have two questions ...
- 311
3
votes
2
answers
175
views
Spline-transform regression - concept clarification
I am learning spline transformation and am confused about several concepts. Any guidance is all appreciated!
Am I understanding this correctly: I should only spline-transform my continuous predictors ...
- 311
0
votes
0
answers
52
views
Nonlinear goodness of fit for data with uncertainties in both x and y
I have a set of samples with an age given with some uncertainty (x $\pm$ dx) and that have some measured property with some given uncertainty (y $\pm$ dy). I would like to fit a curve that begins at $(...
- 49
0
votes
2
answers
186
views
Goodness of fit for Linear Probability Model (LPM)
I'm running a linear probability model (LPM), i.e. my outcome is binary and I have predictors that are categorical and continuous (I'm aware of some of the pros and cons of using LPM for a binary ...
- 37