Questions tagged [goodness-of-fit]

Goodness of fit tests indicate whether or not it is reasonable to assume that a random sample comes from a specific distribution.

Filter by
Sorted by
Tagged with
0
votes
1answer
17 views

Are most fit indices inappropriate for EFA (not CFA)?

Is it the case that many of the available fit indices generated by package "psych" in R are not appropriate for EFA, only for CFA (even though generated by package 'psych' in R for EFA) not useful ...
0
votes
0answers
6 views

Goodness of fit for multivariate Gaussian with identity covariance

My assumptions: I have $X_1, \dots, X_n, T_1, \dots, T_n \overset{i.i.d.}{\sim} \mathcal{D} , X_1 \in \mathbb{R}^n$. I made some assumptions and have fitted an $f : \mathbb{R}^n \to \mathbb{R}^n$ ...
0
votes
0answers
25 views

How to do K-S GOF test in R? [closed]

I have $n = 256$ of $t$ data. Some of the data are $t=77.0,62.1,59.6,56.3,56.8,...$. I want to know if the data fit the Weibull distribution using the K-S GOF test. I fitted the data ...
0
votes
0answers
12 views

Panel Data (Longitudinal) Analysis with SEM with continuous variables

I am estimating a SEM MIMIC model with 2 indicators, and 5 causes of a latent variable. I have a panel of around 170 countries for the period 1960-2018. However, my chi-sq and general fit stats are ...
0
votes
0answers
12 views

Goodness of fit measure for GMM similar to the likelihood function

Suppose we have two trained models for a specific color in an image: 1-Multivariate Gaussian Model (MVG), 2-Gaussian Mixture (GMM). I need to identify the most probable region which then be used for ...
0
votes
2answers
40 views

What fitness test to use if the actual data and expected data has a lot of zeros?

I am testing the agreement of two sets of data with each other, albit testing for the goodness-of-fit. However, both experimental data and model data has actually a lot of zeros in it so chi-squared ...
0
votes
0answers
16 views

Hurdle model “goodness of fit” statistics?

I apologise in advance if this sounds like a stupid question. I am used to GLM with continuous data. I've run a hurdle model in R (pscl:hurdle) with a negative binomial distribution for the "count" ...
0
votes
1answer
30 views

How to improve the fit for a GLM model?

I am fitting a glm model to examine associations between some predictors and a 3-levels outcome variable (see data below): ...
0
votes
0answers
10 views

Assessing goodness of fit for stochastic models

I'm working with rainfall data and trying to construct a stochastic model that models the data well (e.g. seasonality, dry and wet periods, duration of storms, etc.). Specifically, I'm looking at ...
0
votes
0answers
21 views

What is the figure of merite of a *likelihood* fit?

There are two main types of fit to obtain some floated parameters of a model (functional form) on a given dataset : (i) the $\chi^2$ fit, which minimize the $\chi^2$ ii) the maximum likelihood fit, ...
0
votes
1answer
24 views

SEM: High SRMR in model with binary predictor. Other fit indices OK [closed]

In this post I may display severe lack of understanding in SEM and the math behind it so please be kind. I am testing the following structural model with lavaan using MLM (due to non-normality) with ...
2
votes
1answer
67 views

How to choose and perform a goodness-of-fit test?

I notice there are a lot of questions on the topic (most are along the lines of: how to do a Chi-squared goodness-of-fit test), but mine is perhaps more general. Using an example I recently ...
0
votes
1answer
39 views

How do I get the sample mean and standard deviation? [duplicate]

I am trying to get the mean and standard deviation for the following sample means with the given sample frequencies for each mean. So in a histogram, the x-axis will show the mean range and sample ...
1
vote
0answers
22 views

Degree of freedom of different distributions: Chi-Square Goodness of Fit Test

I try to understand why the number of constraints differ for the distributions in the table below, for the Chi-square goodness of fit test. I found the content in the following link. https://www....
3
votes
0answers
45 views

Over “specification” of a statistics model?

For a simple example, I am fitting data with a likelihood function generated by a normal distribution. The first model is the normal distribution with two-parameters. The second competing model is the ...
0
votes
0answers
17 views

Anderson-Darling test Critical Values for Log Logistic (asymptotically using Monte Carlo Sim)

I am trying to to compute the AD test critical values for a Log Logistic distribution (not a normality test, but an EDF) . I am trying to do a Monte Carlo simulation in R. The steps are the following:...
1
vote
0answers
35 views

Goodness of fit ranking for several models

I have some data that I am trying to fit with different models: exponential, sum of two exponentials, hyperbolic secant, etc. I would like to have a criteria to be able to rank the different models ...
1
vote
1answer
38 views

Goodness of fit for Copula

I was wondering if the graphical methods can be used instead of formal tests such as Cramer-Von Mises test as the GOF for copulas? The scatter plot of pseudo-observations is as follows: The Q-Q ...
2
votes
0answers
14 views

Goodness of Fit Test vs Testing RMSE

I have a bit of a broad question. It seems to be that there are two different approaches (Borrowing a bit of Breiman's Two Culture's paper) when it comes to testing if our data looks like our model. ...
3
votes
2answers
147 views

Non-significant result in CFA model

I've fitted a one-factor model to data originating from a unidimensional 8-item scale; sample size is 400. Because the scale uses a Likert response format, and data does not follow a multivariate ...
1
vote
0answers
16 views

ERGM with larger log-likelihood but worse fit than reference model?

I'm using the R package statnet to fit some ERGMs to the Faux Dixon High simulated network data provided with the package. The first model I fit is almost identical ...
0
votes
1answer
53 views

How to assess the quality and goodness of fit of a lasso model?

So far I evaluate the lasso model by its out-of-sample predictions and by the $R^2$ for the goodness of fit. What other measures should I look at? Particularly, do I need to examine the residuals for ...
2
votes
0answers
51 views

Different loss functions for estimation and cross-validation

Assume that the goal is to estimate some parameter $\theta$ (finite or infinite dimensional) based on some data available. Also, assume that there are other nuisance parameters are present in the ...
1
vote
0answers
35 views

Fish counts and Poisson [closed]

I am having troubles with fitting a Poisson distribution to my data, let me explain: I have fish counts of different species from a closed list, at 5 sites, 2 different depths, and across 10 years. ...
1
vote
0answers
16 views

Does the large number of sample is the reason for low p-value in GOF test of Copula? (Cramer_Von mises)

I would like to determine a suitable copula function on my data. The multivariate computed from the original data is similar to the Plackett. The GOF test on my data with size 8000, reject all ...
0
votes
0answers
23 views

Posterior Predictive Check for Hierarchical Logistic Regression Model

I need to apply Posterior Predictive Check (PPC) on Hierarchical Logistic Regression Model (so I have binary outcome) to validate my model (to see goodness of fit of my model). I know that I need to ...
2
votes
1answer
102 views

Limitations of Kolmogorov–Smirnov test

In this article, it is said that the test statistic tends to be more sensitive near the center of the distribution than at the tails. Can some please explain this what it means? (with a simple example ...
2
votes
0answers
31 views

Why do I get different results of goodness of fit test using AD tests from different R libraries?

I am trying to find the best method for goodness of fit while testing my data with different models of distributions. I need to use AD test. So I have compared results using two different ...
1
vote
0answers
47 views

p-value fitting a model to data

I have performed a physical experiment with few (~20) measurements, which have different uncertainties on the y-axis. The measurements were fitted to an exponential model $f(x)=ae^{-bx}+c$ using scipy....
0
votes
1answer
31 views

How to derive the formula for coefficient (slope) of a simple linear regression line? [duplicate]

There is a formula for calculating slope (Regression coefficient), b1, for the following regression line: y= b0 + b1 xi + ei (alternatively y'(predicted)=b0 + b1 * x); which is b1=(∑(xi-Ẋ) * (yi-...
0
votes
0answers
24 views

IRT degrees of freedom of item fit statistics

I'm fitting Rasch models with the mirt package. Upon measuring item-wise $Q_1$ using mirt::itemfit(model, fit_stats="X2"), I ...
1
vote
0answers
17 views

ugarchfit diagnostic tests: goodness of fit

I modeled a couple of financial return time series with different kinds of GARCH-models and different distribution termns and now whant to compare them. I very often face the problem that all ...
0
votes
0answers
41 views

Why Lack of fit test is significant?

I fitted a dose-response curve and part of my data is significant for Lack-of-Fit test, but I don't understand why? If visually the model has a good adjustment. I know the Lack-of-Fit test is not a ...
-2
votes
2answers
47 views

How to confirm more people selected “Agree” than is possible by chance [closed]

In the table below, how do I prove statistically that more people are in the group: "Sum of Agree", than is possible by chance? I do not know which test to use here, (although I am using the Extended ...
1
vote
1answer
27 views

I would like to find Goodness-of-fit between my differential equation simulation result and observed field data. How to find it?

I modelled a ecological system using ordinary differential equation. Simulated the model for 12 months. I also have 12 months of field data. Now I want to find the goodness of fit between my model ...
0
votes
0answers
10 views

Including probabilistic information in the model fit assessment of multivariate nominal regression

I have a ground truth nominal variable $Y$ and a continuous predictor variable $X$. I have split these up into a training dataset, $Y_1$ and $X_1$, and a testing dataset, $Y_2$ and $X_2$. I want to ...
1
vote
1answer
76 views

Difference between QQ plot and KS test

While solving some exercises for HW, I came up to use QQ plot and KS test for the sake of testing goodness of fitting, where I also came up with the following doubts: Precisely, let $X, Y \sim \text{...
0
votes
0answers
17 views

Using the sum of the absolute residuals divided by the sample sum as a normalized goodness-of-fit estimator

I am trying to make an automatic method to do a peak fitting to some spectra, using multiple least squares. I would like to be able to compare the quality of the fit between several different cases. ...
2
votes
0answers
84 views

Random forest [R]: why is my OOB RMSE so much smaller than test RMSE?

I'm doing the kaggle challenge on timetravel predictions where the task is to predict the duration (Y) of a uber trip given some information about the start and end coordinates and the time the trip ...
0
votes
0answers
9 views

Generalization performance of a model (RStudio)

I'm kind of confused on the definition of generalization. E.g. I've created a model in RStudio, for which I've plotted the training history results: Now I'm wondering, is the generalization of my ...
2
votes
0answers
83 views

How to check that quantile regression forests provide valid predictions?

I am using quantile regression forests to predict the distribution of a measure of performance in a medical context. I am using the ranger R package for that purpose. I would like to have advices ...
3
votes
1answer
64 views

What is a lay explanation for the numerator of W in the Shapiro-Wilk test?

I am trying to understand in basic terms what the Shapiro-Wilk test is doing, but the math in the original 1965 paper is beyond me. (For starters, why is the covariance matrix n x n? What do the $v_{...
0
votes
0answers
24 views

Data generation under Pitman alternatives

I would like to numerically simulate the power function of the goodness-of-fit test $T_n$ in https://projecteuclid.org/download/pdf_1/euclid.aos/1176348133 for a set of certain probability ...
1
vote
2answers
53 views

Is it possible to define an optimal fit?

Let assume that we have n pairs of real numbers: (x_1, y_1), (x_2, y_2), ..., (x_n, y_n) Let as also assume that ...
2
votes
1answer
103 views

chi squared goodness of fit test to check for normality

I have the following data: I would like to use the chi-squared goodness of fit test to test whether it comes from a normal distribution. How shall I go about it? In particular, I would like to know ...
0
votes
0answers
17 views

What is considered as a good fit using the fraction deviance?

I am using the fraction deviance given by Hastie et. al (2015), $\mathcal{D}_{\lambda}^{2}=\frac{\mathcal{D}\mathrm{ev}_{\mathrm{null}}-\mathcal{D}\mathrm{ev}_{\lambda}}{\mathcal{D}\mathrm{ev}_{\...
0
votes
0answers
8 views

Questions Tags Users Unanswered Connection of the critical value for KS distance and model complexity

I asked a similar question is math exchange, but here it seems to fit better. I wonder about a relationship of the critical value for the KS distance (with Lilliefor correction) and model complexity? ...
0
votes
0answers
46 views

MAPE vs R Squared, and their Interpretation in Regressions

So, I'm somewhat familiar with R squared and MAPE, and their usage to qualify the goodness of fit for a regression model. My question is not about when to use one or another (like in this post) but ...
0
votes
0answers
58 views

How to validate a simulation model

Apologies in advance if this question is more on the conceptual side, but hopefully it's still appropriate for here: For context, there's a tradition in political science of building simulation ...
0
votes
1answer
32 views

Chi-square Goodness of fit with specific expected values

Let's say I have the following summary data/observed counts. ...

1 2 3 4 5 18