Questions tagged [goodness-of-fit]
Goodness of fit tests indicate whether or not it is reasonable to assume that a random sample comes from a specific distribution.
1,098
questions
2
votes
1
answer
31
views
shepard diagram for high dimension data
I went through this website https://www.displayr.com/goodness-of-fit-in-mds-and-t-sne-with-shepard-diagrams/ for shepard diagram.
Right now I have a 20 dimensional data with 85 samples, I used some ...
- 21
1
vote
0
answers
17
views
Classifying multivariate data as gaussian, non-gaussian, or a mixture of gaussians?
It's for the following use case:
I have a set of "detects" of a group of individual targets (mostly stationary) that I wish to cluster or segment such that each cluster (group of detects) ...
- 11
0
votes
0
answers
29
views
Bayesian or Frequentist goodnes of fit: depends on the data?
I'm working with the data of the real masses of exoplanets published in the catalogues (NASA, exoplanet.eu). Those catalogues update almost everyday by adding new exoplanet data or correcting some of ...
- 11
5
votes
2
answers
75
views
Goodness of fit for exponential model in lmfit
I'm having a dataset where I measured the growth of a given molecule over 7 timepoints.
The growth can be described by the formula:
$(P_{0} - plateau) * e^{-k t} + plateau$ , where k is the growth ...
- 51
0
votes
0
answers
21
views
Test whether a sample (which is a 1D array of values) comes from a population (2D array of values)
I'm taking samples in such a way that I record multiple discrete values for each sample. Given an instance of a sample, is there any way to test if this sample appears to "fit" the ...
- 123
0
votes
0
answers
12
views
Shuffle data inside learning sample in order independet transformer model
Does it make sense to create new samples with shuffled items "tokens" inside a learning sample for the order independent (no positional encoding) transformer model to improve model accuracy?
- 33
7
votes
2
answers
328
views
How can I fit distribution for data which "almost fits"?
I have a sample for events occurring at certain continuous distances (kilometers), let's suppose emergency calls to hospitals. I have 200k observations, coming from 500 hospitals for an entire month. ...
- 173
0
votes
0
answers
22
views
Chi squared min / max (as in best / worst possible fit) values
Naturally the answer to the former would be 0 in case of a perfect fit. But how would one go about finding out the value of the worst possible fit?
Background: Doing a simple experiment to crack ...
1
vote
1
answer
29
views
What is the right distribution to model the duration of (short-term) sick-leave in days?
I would like to model the distribution of the duration of (short-term) sick-leave: if some one calls in sick, how long can you expect the sickness to last? Short-term would typically mean between 1 ...
- 2,052
1
vote
0
answers
32
views
McKelvey and Zavoina's pseudo-$R^2$ score range
I am trying to build a probit regression model. Looking for some proper goodness of fit indicator for my model I read that McKelvey and Zavoina's pseudo-$R^2$ could be the best index for this purpose.
...
- 11
0
votes
0
answers
22
views
Which fit indices in lavaan are appropriate for WLSMV estimator?
I'm conducting a few CFAs with Likert-type data (7 and 5 categories), and hoping to compare my results with the MLR (and MLM) estimators to those with WLSMV in lavaan. I haven't been able to find any ...
- 1
1
vote
1
answer
43
views
Are MAE and RMSE possible ways of choosing between ols and types of glm?
I have a longitudinal data set and have fit a number of different regression models [ols, poisson, binomial....and more]. I want to justify the final model selection and assumed one method might be to ...
- 31
2
votes
2
answers
107
views
Goodness of fit for exponential distribution and large sample
I'm new to statistics and statistics are not my area of research so maybe my question is simple and answer is on the surface. In my research the empirical CDF for the data looks like exponential ...
- 21
0
votes
0
answers
28
views
comparison of standardised residuals from chi-square goodness of fit test
I am analysing the observed distribution of certain amino acid sequence motifs across the superkingdoms against that predicted based on theory using chi-square goodness-of-fit test. In order to find ...
- 11
0
votes
0
answers
7
views
Moderated non-linear factor analysis (MNLFA): Information on fit indicees
I am planing to test measurement invariance using the MNLFA approach for continuous and categorical backround variables.
I was wondering if there are any fit criteria that can be calculated to ...
- 1
0
votes
0
answers
23
views
Why is Total Variation Distance not used as a statistic for categorical goodness-of-fit?
The test statistic is proposed in this paper, although as they mention, it is a fairly obvious thing to try. While I'm skeptical of the paper's claims of optimality (since they show optimality only ...
- 121
2
votes
0
answers
33
views
Find the likelihood threshold for a Goodness-of-Fit test for multinomial data
Given a sample size $n \in \mathbb{N}$, a null hypothesis $H_0 = \langle p_1, p_2, \dots p_k\rangle$ which is an element of the $k$-dimensional probability simplex, and a significance threshold $\...
- 121
0
votes
0
answers
15
views
why Goodness of fit tests for PDF gives 0 p-value for large data sets? [duplicate]
I am working on traffic data, consisitng of data points in ten of thousands.I run goodness of fit tests (K-S,Anderson-Darling,Chi-squared) to check which Probabilty distribution model is best fit for ...
0
votes
0
answers
65
views
Metrics to measure the goodness of fit in Python statsmodels OLS
I am using Python’s statsmodels.api.OLS(Y, X).fit() for multivariable data fit. Model summary showed high R^2 and F-statistic. So I did a smoke test. My target ...
- 111
1
vote
1
answer
27
views
Goodness of fit test for a transition density function of a Markov process
Suppose that you have one realization $x = \{x_n\}_{n = 1}^{N}$ of the stochastic process $X = \{X_n\}_{n = 1}^{N}$ with state space $\mathbb{R}$. Assume that the process is Markovian, time-...
- 11
2
votes
1
answer
34
views
Can the confidence interval assess the goodness of fit of fitted regression line?
I understand that the confidence interval for the mean is completely different from the confidence interval band for the regression model. I also understand that the confidence interval bad represents ...
- 507
0
votes
0
answers
16
views
Are there any goodness of fit tests jointly for continuous and binary data?
Let's say that some users have filled out an online questionnaire, where some questions were simple Yes/No questions, and for the other questions the users had a slider which they could adjust to ...
- 243
0
votes
1
answer
73
views
AIC and BIC in GEE
Can I apply Akaike’s information criterion (AIC) and Quasi-likelihood under the independence model information criterion (QIC) in GEE?
0
votes
0
answers
24
views
Degrees of freedom in an photon counting experiment for chi squared test
I am attempting to apply a chi-squared test to determine the goodness of fit between modeled and experimental data of a photon counting experiment. My data sets are two histograms, each with 75 bins. ...
- 1
2
votes
0
answers
103
views
Independence Testing for Discrete Random Variable
Suppose $X$ and $Y$ are two discrete random variable take values on $\mathcal{X}$ and $\mathcal{Y}$ with their iid observation $\{ X_i\}_{i = 1}^n$ and $\{ Y_i\}_{i = 1}^n$. If both $\mathcal{X}$ and $...
- 31
1
vote
2
answers
75
views
Why use normality tests if we have goodness-of-fit tests?
What are the reason/s to use a nonparametric normality test (e.gr., Shapiro-Wilk, Jarque-Bera) instead of generic, parametric goodness-of-fit tests (good for any distribution including but not limited ...
- 203
0
votes
0
answers
79
views
chi-square goodness of fit test on count data
I am performing chi-square goodness-of-fit test on a predicted and observed count data. Some observed counts are very low or zero.
I have a two part question,
- Does r's chisq.test function exclude ...
- 11
0
votes
0
answers
16
views
goodness of fit test for model
I did GLMM with negative binomial family. After checking goodness of fit using the equation
"1-pchisq(summary(model4)$deviance,summary(model4)$df.residual)" it gives the value zero. what it ...
- 1
1
vote
1
answer
63
views
Comparing goodness of fit across parametric and semi-parametric survival models
I've been learning about time-to-event analysis and playing with open datasets + fitting various Cox and parametric models for practice.
Other than by visually inspecting the estimated survival curves ...
- 37
2
votes
1
answer
31
views
Validating random variable generation from inverse transform sampling
I'm building a simulator and I have to implement some probability distributions. What is the best (formal) way of validating this implementation? I took a look at KS-tests but it seems to me they are ...
1
vote
0
answers
33
views
How to compare the goodness of fit between linear and logit? Why linear deviance is less than logit?
How can I evaluate which model - between linear and logit - determine the best fit to the data?
The models use the same input variables and I thought that comparing the deviances was the proper choice ...
- 11
0
votes
0
answers
54
views
Chi square of 0.00, but acceptable (other) fit indices?
I am running a path analysis for a sample of 75 participants using RStudio. I am comparing two models and have to decide which one to use (I am in favor of Model A, but because of the Chi-square: 0, I ...
0
votes
1
answer
42
views
Chi square fit, fitting data to a straight line: Incomplete gamma function as goodness-of-fit estimation
On page 661 of the book "Numerical Recipes in C:The art of scientific computing", we fit a set of N data points
$(x_{i}, y_i)$
to a straight line model
$$y(x)=a+bx.$$
We do this by ...
- 11
0
votes
0
answers
36
views
Does $R^2$ measure goodness of fit or not? [duplicate]
I have been warned against $R^2$ as goodness of fit before, but am unsure why. What is wrong with using it to characterize how well a line fits data points? I have looked at a few sources to try and ...
- 13
0
votes
0
answers
10
views
Goodness of fit for multinomials with small np
I've got a model with continuous explanatory variable $x_i$ and some positive integer $m_i$ for each datapoint, and then a multinomial response - i.e. $p$ categories, and the sum of counts across the $...
- 51
0
votes
0
answers
30
views
Measure of goodness-of-fit in errors-in-variable regression
I have two observed time series $x_i$ and $y_i$ and I want to test if $x_i$ is a good predictor of of $y_i$. So I would usually run a simple linear regression Y ~ X and use $R^2$ as a measure of ...
- 1
0
votes
0
answers
25
views
Smoothing time series with Adjusted R2-weighted averages
I have two parameters (a,b) resulting from an exponential estimation of a curve. I have estimated this curve every hour for one month. In other words, I have a total of 720 parameters a and b, and I ...
- 11
0
votes
1
answer
123
views
Extension of Pearson's chi-squared test to sequences of multinomial random variables
Background
Suppose we observe $n$ IID Bernoulli variables and our null hypothesis is that their common probability is $p$.
For denote by $\mathbb{1}_{\{i\}}$ the outcome of observation $i$.
Then by ...
- 29
0
votes
0
answers
19
views
Weighting the R-squared as a measure of goodness-of-fit in Linear Regression [duplicate]
I have two observed time series $x_i$ and $y_i$ and I want to test if $x_i$ is a good predictor of of $y_i$. So I run a simple linear regression Y ~ X and use $R^2$ as a measure of goodness of fit.
...
- 1
0
votes
0
answers
10
views
Evaluate the goodness of a fitted continous distribution in the case of ties (Used later in DES simulations)
I have continous data which shows the time in minutes, a process step takes.
Unfortunatelly, it is not possible to get data without rounding so I have many ties.
As I thought, a random process should ...
- 139
1
vote
0
answers
143
views
Why are my P-Values for the scipy kstest on a uniform distribution so high?
I'm using the Scipy implementation of the Kolmogorov–Smirnov test to check whether collections of random values are likely to have been drawn from a uniform distribution.
From what I understand, the ...
- 111
0
votes
0
answers
27
views
ERGM with missing tie data: Interpretation of the mcmc.diagnostics regarding autocorrelation
I am running an ergm model on network data that contains some missing tie variables. I am running this model to later use it to impute the missing data. I am currently investigating the mcmc....
0
votes
0
answers
48
views
How to compare distributions with errors on the data points?
Here's a mock set-up of my problem:
I have two non-normal probability density distributions (PDFs), $A$ and $B$.
Distribution $A$ has error measurements for each data point while distribution $B$ ...
0
votes
0
answers
34
views
Does the Bollen-Stine procedure in AMOS affect fit indices?
My colleagues and I are working on a cross-cultural study and we attempt to obtain evidences of factorial validity by doing a measurement invariance analysis. Since our data failed to meet the ...
0
votes
0
answers
12
views
Different tests for model fit
I saw that in linear regression models one often uses a hypothesis z- or t-test for $R^2$ or for effect sizes. A z-Test is only useful if the standardized $R^2$-values are standard normally ...
- 11
0
votes
1
answer
45
views
How to correctly perform a goodness-of-fit test for a contingency table (two-way, three-way, or more), in situations other than independence testing?
Let's say I have the following table from a sample of 462 people:
Gender
Happy
Meh
Sad
Men
70
32
120
Women
100
30
110
I don't want to test it against the hypothesis of independence, but against ...
- 128
0
votes
1
answer
31
views
Which model fit indices do you report from lavaan cfa for scale development?
For RMSEA there seems to be a regular value and two robust values. Are there any recommendations on which of the three values to report? Would you report 1, 2 or 3 from lavaan?
- 11
0
votes
0
answers
20
views
Time Series with Possible Discontinuity - Is the discontinuity significant?
I am just beginning to learn about statistics and statistical tests, so please forgive me if I'm using entirely the wrong terminology or I've missed something obvious.
I'm trying to solve a problem ...
- 1
1
vote
1
answer
59
views
How to compare results of a discrete (binary) model to a probabilistic model?
Let's suppose I have two models that both indicate the presence of some phenomenon:
Model A: Only binary results, i.e., the phenomenon is present or not,
Model B: Outputs class probabilities.
Of ...
- 93
0
votes
0
answers
21
views
Generating random data points following a Poisson point process from observed data points [duplicate]
I am a bit new to the domain of Spatial Statistics. So I am trying to generate complete spatial randomness(CSR) data with summary statistics similar to that of the observed data(data points in 2D). ...
- 11