Questions tagged [goodness-of-fit]

Goodness of fit tests indicate whether or not it is reasonable to assume that a random sample comes from a specific distribution.

Filter by
Sorted by
Tagged with
2 votes
1 answer
31 views

shepard diagram for high dimension data

I went through this website https://www.displayr.com/goodness-of-fit-in-mds-and-t-sne-with-shepard-diagrams/ for shepard diagram. Right now I have a 20 dimensional data with 85 samples, I used some ...
  • 21
1 vote
0 answers
17 views

Classifying multivariate data as gaussian, non-gaussian, or a mixture of gaussians?

It's for the following use case: I have a set of "detects" of a group of individual targets (mostly stationary) that I wish to cluster or segment such that each cluster (group of detects) ...
  • 11
0 votes
0 answers
29 views

Bayesian or Frequentist goodnes of fit: depends on the data?

I'm working with the data of the real masses of exoplanets published in the catalogues (NASA, exoplanet.eu). Those catalogues update almost everyday by adding new exoplanet data or correcting some of ...
  • 11
5 votes
2 answers
75 views

Goodness of fit for exponential model in lmfit

I'm having a dataset where I measured the growth of a given molecule over 7 timepoints. The growth can be described by the formula: $(P_{0} - plateau) * e^{-k t} + plateau$ , where k is the growth ...
  • 51
0 votes
0 answers
21 views

Test whether a sample (which is a 1D array of values) comes from a population (2D array of values)

I'm taking samples in such a way that I record multiple discrete values for each sample. Given an instance of a sample, is there any way to test if this sample appears to "fit" the ...
0 votes
0 answers
12 views

Shuffle data inside learning sample in order independet transformer model

Does it make sense to create new samples with shuffled items "tokens" inside a learning sample for the order independent (no positional encoding) transformer model to improve model accuracy?
7 votes
2 answers
328 views

How can I fit distribution for data which "almost fits"?

I have a sample for events occurring at certain continuous distances (kilometers), let's suppose emergency calls to hospitals. I have 200k observations, coming from 500 hospitals for an entire month. ...
0 votes
0 answers
22 views

Chi squared min / max (as in best / worst possible fit) values

Naturally the answer to the former would be 0 in case of a perfect fit. But how would one go about finding out the value of the worst possible fit? Background: Doing a simple experiment to crack ...
1 vote
1 answer
29 views

What is the right distribution to model the duration of (short-term) sick-leave in days?

I would like to model the distribution of the duration of (short-term) sick-leave: if some one calls in sick, how long can you expect the sickness to last? Short-term would typically mean between 1 ...
  • 2,052
1 vote
0 answers
32 views

McKelvey and Zavoina's pseudo-$R^2$ score range

I am trying to build a probit regression model. Looking for some proper goodness of fit indicator for my model I read that McKelvey and Zavoina's pseudo-$R^2$ could be the best index for this purpose. ...
  • 11
0 votes
0 answers
22 views

Which fit indices in lavaan are appropriate for WLSMV estimator?

I'm conducting a few CFAs with Likert-type data (7 and 5 categories), and hoping to compare my results with the MLR (and MLM) estimators to those with WLSMV in lavaan. I haven't been able to find any ...
  • 1
1 vote
1 answer
43 views

Are MAE and RMSE possible ways of choosing between ols and types of glm?

I have a longitudinal data set and have fit a number of different regression models [ols, poisson, binomial....and more]. I want to justify the final model selection and assumed one method might be to ...
2 votes
2 answers
107 views

Goodness of fit for exponential distribution and large sample

I'm new to statistics and statistics are not my area of research so maybe my question is simple and answer is on the surface. In my research the empirical CDF for the data looks like exponential ...
0 votes
0 answers
28 views

comparison of standardised residuals from chi-square goodness of fit test

I am analysing the observed distribution of certain amino acid sequence motifs across the superkingdoms against that predicted based on theory using chi-square goodness-of-fit test. In order to find ...
  • 11
0 votes
0 answers
7 views

Moderated non-linear factor analysis (MNLFA): Information on fit indicees

I am planing to test measurement invariance using the MNLFA approach for continuous and categorical backround variables. I was wondering if there are any fit criteria that can be calculated to ...
  • 1
0 votes
0 answers
23 views

Why is Total Variation Distance not used as a statistic for categorical goodness-of-fit?

The test statistic is proposed in this paper, although as they mention, it is a fairly obvious thing to try. While I'm skeptical of the paper's claims of optimality (since they show optimality only ...
2 votes
0 answers
33 views

Find the likelihood threshold for a Goodness-of-Fit test for multinomial data

Given a sample size $n \in \mathbb{N}$, a null hypothesis $H_0 = \langle p_1, p_2, \dots p_k\rangle$ which is an element of the $k$-dimensional probability simplex, and a significance threshold $\...
0 votes
0 answers
15 views

why Goodness of fit tests for PDF gives 0 p-value for large data sets? [duplicate]

I am working on traffic data, consisitng of data points in ten of thousands.I run goodness of fit tests (K-S,Anderson-Darling,Chi-squared) to check which Probabilty distribution model is best fit for ...
0 votes
0 answers
65 views

Metrics to measure the goodness of fit in Python statsmodels OLS

I am using Python’s statsmodels.api.OLS(Y, X).fit() for multivariable data fit. Model summary showed high R^2 and F-statistic. So I did a smoke test. My target ...
1 vote
1 answer
27 views

Goodness of fit test for a transition density function of a Markov process

Suppose that you have one realization $x = \{x_n\}_{n = 1}^{N}$ of the stochastic process $X = \{X_n\}_{n = 1}^{N}$ with state space $\mathbb{R}$. Assume that the process is Markovian, time-...
  • 11
2 votes
1 answer
34 views

Can the confidence interval assess the goodness of fit of fitted regression line?

I understand that the confidence interval for the mean is completely different from the confidence interval band for the regression model. I also understand that the confidence interval bad represents ...
  • 507
0 votes
0 answers
16 views

Are there any goodness of fit tests jointly for continuous and binary data?

Let's say that some users have filled out an online questionnaire, where some questions were simple Yes/No questions, and for the other questions the users had a slider which they could adjust to ...
  • 243
0 votes
1 answer
73 views

AIC and BIC in GEE

Can I apply Akaike’s information criterion (AIC) and Quasi-likelihood under the independence model information criterion (QIC) in GEE?
0 votes
0 answers
24 views

Degrees of freedom in an photon counting experiment for chi squared test

I am attempting to apply a chi-squared test to determine the goodness of fit between modeled and experimental data of a photon counting experiment. My data sets are two histograms, each with 75 bins. ...
  • 1
2 votes
0 answers
103 views

Independence Testing for Discrete Random Variable

Suppose $X$ and $Y$ are two discrete random variable take values on $\mathcal{X}$ and $\mathcal{Y}$ with their iid observation $\{ X_i\}_{i = 1}^n$ and $\{ Y_i\}_{i = 1}^n$. If both $\mathcal{X}$ and $...
1 vote
2 answers
75 views

Why use normality tests if we have goodness-of-fit tests?

What are the reason/s to use a nonparametric normality test (e.gr., Shapiro-Wilk, Jarque-Bera) instead of generic, parametric goodness-of-fit tests (good for any distribution including but not limited ...
  • 203
0 votes
0 answers
79 views

chi-square goodness of fit test on count data

I am performing chi-square goodness-of-fit test on a predicted and observed count data. Some observed counts are very low or zero. I have a two part question, - Does r's chisq.test function exclude ...
  • 11
0 votes
0 answers
16 views

goodness of fit test for model

I did GLMM with negative binomial family. After checking goodness of fit using the equation "1-pchisq(summary(model4)$deviance,summary(model4)$df.residual)" it gives the value zero. what it ...
1 vote
1 answer
63 views

Comparing goodness of fit across parametric and semi-parametric survival models

I've been learning about time-to-event analysis and playing with open datasets + fitting various Cox and parametric models for practice. Other than by visually inspecting the estimated survival curves ...
  • 37
2 votes
1 answer
31 views

Validating random variable generation from inverse transform sampling

I'm building a simulator and I have to implement some probability distributions. What is the best (formal) way of validating this implementation? I took a look at KS-tests but it seems to me they are ...
1 vote
0 answers
33 views

How to compare the goodness of fit between linear and logit? Why linear deviance is less than logit?

How can I evaluate which model - between linear and logit - determine the best fit to the data? The models use the same input variables and I thought that comparing the deviances was the proper choice ...
0 votes
0 answers
54 views

Chi square of 0.00, but acceptable (other) fit indices?

I am running a path analysis for a sample of 75 participants using RStudio. I am comparing two models and have to decide which one to use (I am in favor of Model A, but because of the Chi-square: 0, I ...
0 votes
1 answer
42 views

Chi square fit, fitting data to a straight line: Incomplete gamma function as goodness-of-fit estimation

On page 661 of the book "Numerical Recipes in C:The art of scientific computing", we fit a set of N data points $(x_{i}, y_i)$ to a straight line model $$y(x)=a+bx.$$ We do this by ...
0 votes
0 answers
36 views

Does $R^2$ measure goodness of fit or not? [duplicate]

I have been warned against $R^2$ as goodness of fit before, but am unsure why. What is wrong with using it to characterize how well a line fits data points? I have looked at a few sources to try and ...
  • 13
0 votes
0 answers
10 views

Goodness of fit for multinomials with small np

I've got a model with continuous explanatory variable $x_i$ and some positive integer $m_i$ for each datapoint, and then a multinomial response - i.e. $p$ categories, and the sum of counts across the $...
  • 51
0 votes
0 answers
30 views

Measure of goodness-of-fit in errors-in-variable regression

I have two observed time series $x_i$ and $y_i$ and I want to test if $x_i$ is a good predictor of of $y_i$. So I would usually run a simple linear regression Y ~ X and use $R^2$ as a measure of ...
  • 1
0 votes
0 answers
25 views

Smoothing time series with Adjusted R2-weighted averages

I have two parameters (a,b) resulting from an exponential estimation of a curve. I have estimated this curve every hour for one month. In other words, I have a total of 720 parameters a and b, and I ...
0 votes
1 answer
123 views

Extension of Pearson's chi-squared test to sequences of multinomial random variables

Background Suppose we observe $n$ IID Bernoulli variables and our null hypothesis is that their common probability is $p$. For denote by $\mathbb{1}_{\{i\}}$ the outcome of observation $i$. Then by ...
0 votes
0 answers
19 views

Weighting the R-squared as a measure of goodness-of-fit in Linear Regression [duplicate]

I have two observed time series $x_i$ and $y_i$ and I want to test if $x_i$ is a good predictor of of $y_i$. So I run a simple linear regression Y ~ X and use $R^2$ as a measure of goodness of fit. ...
  • 1
0 votes
0 answers
10 views

Evaluate the goodness of a fitted continous distribution in the case of ties (Used later in DES simulations)

I have continous data which shows the time in minutes, a process step takes. Unfortunatelly, it is not possible to get data without rounding so I have many ties. As I thought, a random process should ...
  • 139
1 vote
0 answers
143 views

Why are my P-Values for the scipy kstest on a uniform distribution so high?

I'm using the Scipy implementation of the Kolmogorov–Smirnov test to check whether collections of random values are likely to have been drawn from a uniform distribution. From what I understand, the ...
0 votes
0 answers
27 views

ERGM with missing tie data: Interpretation of the mcmc.diagnostics regarding autocorrelation

I am running an ergm model on network data that contains some missing tie variables. I am running this model to later use it to impute the missing data. I am currently investigating the mcmc....
0 votes
0 answers
48 views

How to compare distributions with errors on the data points?

Here's a mock set-up of my problem: I have two non-normal probability density distributions (PDFs), $A$ and $B$. Distribution $A$ has error measurements for each data point while distribution $B$ ...
0 votes
0 answers
34 views

Does the Bollen-Stine procedure in AMOS affect fit indices?

My colleagues and I are working on a cross-cultural study and we attempt to obtain evidences of factorial validity by doing a measurement invariance analysis. Since our data failed to meet the ...
0 votes
0 answers
12 views

Different tests for model fit

I saw that in linear regression models one often uses a hypothesis z- or t-test for $R^2$ or for effect sizes. A z-Test is only useful if the standardized $R^2$-values are standard normally ...
0 votes
1 answer
45 views

How to correctly perform a goodness-of-fit test for a contingency table (two-way, three-way, or more), in situations other than independence testing?

Let's say I have the following table from a sample of 462 people: Gender Happy Meh Sad Men 70 32 120 Women 100 30 110 I don't want to test it against the hypothesis of independence, but against ...
  • 128
0 votes
1 answer
31 views

Which model fit indices do you report from lavaan cfa for scale development?

For RMSEA there seems to be a regular value and two robust values. Are there any recommendations on which of the three values to report? Would you report 1, 2 or 3 from lavaan?
  • 11
0 votes
0 answers
20 views

Time Series with Possible Discontinuity - Is the discontinuity significant?

I am just beginning to learn about statistics and statistical tests, so please forgive me if I'm using entirely the wrong terminology or I've missed something obvious. I'm trying to solve a problem ...
1 vote
1 answer
59 views

How to compare results of a discrete (binary) model to a probabilistic model?

Let's suppose I have two models that both indicate the presence of some phenomenon: Model A: Only binary results, i.e., the phenomenon is present or not, Model B: Outputs class probabilities. Of ...
0 votes
0 answers
21 views

Generating random data points following a Poisson point process from observed data points [duplicate]

I am a bit new to the domain of Spatial Statistics. So I am trying to generate complete spatial randomness(CSR) data with summary statistics similar to that of the observed data(data points in 2D). ...

1
2 3 4 5
22