Questions tagged [goodness-of-fit]
Goodness of fit tests indicate whether or not it is reasonable to assume that a random sample comes from a specific distribution.
901
questions
0
votes
0answers
4 views
Comparing return distributions
Taking the log and/or normalizing (subtracting the mean and dividing by the standard deviation) stock returns still does not eliminate the dependence or heterogeneity. E.g. I cannot assume that each ...
1
vote
1answer
13 views
goodness of fit statistic independant of the fixed effects
I am using glm model to fit data.
I want to check if my fit is good. My model is usually the classical model with only fixed effects. In order to do so, I use goodness of fit tools like residual ...
1
vote
0answers
8 views
Can you compare fitness of a data set to a predetermined/expected line?
I have a experimental data of counting the timing of events which, if random, should follow a Poisson distribution. To evaluate whether this is true I plotted the standard deviation of the data ...
0
votes
0answers
19 views
Goodness of fit using characteristic function
When assessing goodness of fit of a model, one can use, for example, a Q-Q plot of empirical vs theoretical distributions. But how does one perform a GoF assessment when there is no closed form ...
0
votes
0answers
16 views
p value for the Cramér–von Mises two sample test in python [closed]
Where can I find a package that computes the p-value for the Cramér–von Mises 2 sample test in Python.
I was able to find a package that computes the Cramér–von Mises 2 sample test itself :
https://...
3
votes
1answer
88 views
Goodness of Fit ot Least Squares with known measurement uncertainty
We want to estimate $\beta$ for
$$
y = x\beta + \epsilon
$$
where $y$ and $x$ are $n\times 1$ vector and $\epsilon$ is not i.i.d, but $\epsilon \sim N(0, \sigma^2\Omega)$, where $\Omega$ and $W$ are $...
0
votes
0answers
13 views
How to interpret Hosmer and Lemeshow test in R?
I'm reading some researches about how to use Hosmer-Lemeshow Goodness of Fit (GOF) Test in R. The results are quite clear and reasonable: X-squared, Degree of Freedom, p-value,.. However when I take a ...
1
vote
1answer
79 views
$R^2$ of regression of unity on $X$ (i.e. $Y$ is a vector of ones)
Assume we have a vector of ones and some (one or more) variables $X$, and we run a linear regression of unity on $X$ (i.e. if we only assume one RHS variable, the implied scatter plot would be a flat ...
0
votes
0answers
13 views
RMSE normalization. Number of bins
I am using RMSE (Root mean squared error) as a measure of goodness of fit.
I am fitting a formula to binned data. The number of bins is not fixed: if there are less than 5 data values in a certain bin,...
1
vote
0answers
57 views
Koenker & Machado (1999) goodness-of-fit criterion (R1)
Has anyone come across a journal paper or a book providing a rule of thumb regarding what R1 is appropriate in research that focuses on the impact of macroeconomic or bank-specific variables on bank ...
3
votes
4answers
209 views
Applying bootstrapping to test if the data followed a certain distribution
I have a dataset with a large sample size (around 80,000). I would like to test if the data followed a certain distribution. I can fit a distribution function, such as log-normal or gamma, to the ...
0
votes
1answer
13 views
Comparing two discrete paired datasets
The problem: I am studying the estimated number of cases of malaria in regions of the world and found that both the WHO and IHME have their own estimates. I want to find if the difference between the ...
1
vote
1answer
11 views
How to compute $R^2$ of test set in leave-one-out CV?
In leave-one-out cross validation, at each iteration, my test set is composed by only one data point - precisely the "left out", to be compared with the predicted one, using the estimated coefficients ...
1
vote
0answers
44 views
Using Cross Validation for Goodness of Fit for Competing Inferential Models
I have a project where I need to 1) perform inference: understand the role of predictors on the response through models (with my data I need to choose a model where I can reasonably assess how changes ...
0
votes
1answer
30 views
How to test that a subset of data is distributed no differently than original (discrete, not normal)
In the research I am doing we have initially had ~600 samples which had a certain distribution (not normal, although close). The characteristic of these samples that I am interested in is discrete (...
3
votes
1answer
58 views
Why McFadden's pseudo-R^2?
Why can't we simply use ordinary $R^2$ in logistic regression, as we do in linear regression? Domencich and McFadden seem to imply that heteroskedasticity is an issue:
but I don't understand why. In ...
1
vote
0answers
18 views
Highly fluctuating value of test statistic in Anderson Darling test
I am trying to verify whether my sampled data is from a population with a Gamma density function. My plan is to do this by means of the Anderson-Darling test. Before doing so, I thought it would be ...
0
votes
0answers
13 views
Sum of pure errors
We know that in a simple linear regression model that the sum of all the residuals is 0 but why is it that the sum of all the pure errors is also 0? Is there a relation between them?
0
votes
0answers
20 views
What is a right definition for “nested models” in Zero Inflated Poisson?
Suppose I have 2 models:
...
0
votes
0answers
39 views
Ridge regression and low R-squared
I am currently working with a strongly balanced data set. I have 15 countries over the period 1990-2017. My dependent variable is CO2 emissions. My independent variables are GDP per capita, total ...
0
votes
0answers
13 views
What is a good metric for a binary classifier when we are more interested in Precision than Recall but care about confidence scores?
I have an heavily imbalanced dataset and am training a binary classifier (which produces scores in the range 0 to 1) and need a single summary metric to tune hyperparameters with.
For my problem, ...
0
votes
0answers
22 views
How to compare AIC and goodness of fit tests?
My data is categorical. I have fitted two models, say, model 1 and model 2. Model 1 is larger than model 2 (in terms of the number of estimated parameters) and model 2 is nested within model 1.
AIC ...
0
votes
1answer
18 views
Height of fitted distribution much higher than real data
For what reason can the height of a fitted distribution be much has higher than the real data it was fitted on? Does it just mean that the distribution does not fit the data or that I have too few ...
2
votes
1answer
47 views
Statistical test to detect regional excess of observations compared to background observations
I have data on the protein position of genetic variants. I want to determine whether there is a region of the proteins with a significant excess of variants relative to controls.
Consider this ...
1
vote
0answers
23 views
Zero inflated generalized poisson: How can I compare 2 models?
I'm using Statsmodels to fit count data by Zero Inflated Poisson Regression.
Suppose that I have 2 models:
...
0
votes
0answers
27 views
Goodness of fit with observed and estimated data
I am trying to fit a model to some data. It represents how density decreases with radius. The model or profile is exponential.
I wanted to apply some goodness of fit test. I tried with the chi-square ...
1
vote
1answer
30 views
Can we apply the Probability Integral Transform to Dependent Random Variables?
Let's suppose we deal with a non-homogeneous Poisson process having intensity function λ(t), t ≥ 0. The event times X1, X2, … of ...
0
votes
0answers
18 views
How to decide between goodness of fit tests
I have a question on BIC and AIC.
I have a data set and I need to test this data set if it fits various distributions (for example, Gamma or Poisson, etc.) I need to use AIC and BIC statistics for ...
0
votes
0answers
9 views
What is going on when deviation is significant but then not significant for outlier test using DHARMa?
I've used the DHARMa package to determine if my model fit is acceptable. I have run the following model:
Number of types of bird food ~ number of feeders, age, education, bird feeding years.
I've ...
0
votes
0answers
15 views
P-value for goodness of fit test
I have a test statistic of 0.533208 for a chi-square distribution with degrees of freedom =2.When I calculate the P-value, it automatically gives me p-value for right tail(0.766)and for left tail(0....
0
votes
3answers
34 views
OLS regression fit
I have 5 data sets fitting exactly the same ols model y = 5 + 0.4x. What is the best way to assess model's goodness of fit for each of the dataset and determine whether to use that model ?
1
vote
0answers
14 views
Convergence rate of test-statistic to chi-square distribution
I know that the to test whether $\Sigma=\Sigma_0$ against $\Sigma\ne\Sigma_0$ for an $n\times p$ data matrix, the test statistic is $np(a-1-\log g)$ where $a$ and $g$ are the AM and GM of the eigen ...
0
votes
1answer
56 views
Can the Chi-Square Value be Used to Quantitatively Discuss Goodness of Fit for any Data?
To quantitatively discuss the relationship between experimentally measured results and a fitted curve, is obtaining the Chi-Square value for my data relevant? By that I mean if I obtain the Chi-square ...
4
votes
1answer
74 views
In a linear regression, trying to show $R^2 = r_{xy}^2$ using projections / geometric intuition
In a linear regression
$$
Y = X\beta + \varepsilon,
$$
I define two (standard) projection matrices. The projection matrix into subspace spanned by columns of the design matrix $X$:
$$
H := X(X^\top ...
0
votes
0answers
26 views
Chi Square test gives bad results for good gaussian fit
I have a set of data, that I wanted to fit with the sum of 3 Gaussians. The data can be fit well:
However, as can be seen, my p-value is 0.00. Can someone explain me, why?
The code, that I used for ...
1
vote
2answers
60 views
What does goodness of fit tell us with skewed data?
"The goodness of fit test is a statistical hypothesis test to see how well sample data fit a distribution from a population with a normal distribution. Put differently, this test shows if your sample ...
1
vote
0answers
15 views
Multivariable residual analysis and goodness of fit
I am reading the paper Statistical Modeling: The Two Cultures (2001) by Leo Breiman.
In section 5.2 he claims:
Residual analysis is similarly unreliable. In a discussion after a presentation of ...
1
vote
0answers
16 views
How to combine/pool the standard error of fitting parameters from multiple fits of the same model?
Let me first explain the context of the problem:
I have a time series of the (z-)positions of a particle relative to a surface. For 5 independent subsamples of this time series, I calculate the ...
0
votes
0answers
14 views
Interpreting LR test of model fit when additional variables are insignificant?
Suppose we estimate the following (made up) model:
$$\tag{1}
pizza = \beta_0 + \beta_1 price + \epsilon
$$
where
pizza is quantity of pizza purchased
price is the price of a pizza
Now suppose we ...
1
vote
1answer
94 views
How can I derive goodness-of-fit measures (R-squared and chi-square values) from multinom (multinomial logistic regression) in R?
I am running multinomial logistic regression using multinom command in R. However, I could not figure out how to derive R-squared and chi-squared values from it. I somewhat approached my own way to ...
1
vote
0answers
20 views
Deviance as a measure of fit
I have a general question about using deviance as a measure of fit in generalized linear models (i.e. multinomial, poisson etc.). I think I've gotten lost in all the equations and have totally missed ...
0
votes
1answer
26 views
Are most fit indices inappropriate for EFA (not CFA)?
Is it the case that many of the available fit indices generated by package "psych" in R are
not appropriate for EFA, only for CFA (even though generated by package 'psych' in R for EFA)
not useful ...
0
votes
0answers
10 views
Goodness of fit for multivariate Gaussian with identity covariance
My assumptions:
I have $X_1, \dots, X_n, T_1, \dots, T_n \overset{i.i.d.}{\sim} \mathcal{D} , X_1 \in \mathbb{R}^n$. I made some assumptions and have fitted an $f : \mathbb{R}^n \to \mathbb{R}^n$ ...
0
votes
0answers
16 views
Panel Data (Longitudinal) Analysis with SEM with continuous variables
I am estimating a SEM MIMIC model with 2 indicators, and 5 causes of a latent variable. I have a panel of around 170 countries for the period 1960-2018.
However, my chi-sq and general fit stats are ...
0
votes
0answers
38 views
Goodness of fit measure for GMM similar to the likelihood function
Suppose we have two trained models for a specific color in an image: 1-Multivariate Gaussian Model (MVG), 2-Gaussian Mixture (GMM).
I need to identify the most probable region which then be used for ...
0
votes
2answers
40 views
What fitness test to use if the actual data and expected data has a lot of zeros?
I am testing the agreement of two sets of data with each other, albit testing for the goodness-of-fit. However, both experimental data and model data has actually a lot of zeros in it so chi-squared ...
0
votes
0answers
39 views
Hurdle model “goodness of fit” statistics?
I apologise in advance if this sounds like a stupid question. I am used to GLM with continuous data.
I've run a hurdle model in R (pscl:hurdle) with a negative binomial distribution for the "count" ...
0
votes
1answer
37 views
How to improve the fit for a GLM model?
I am fitting a glm model to examine associations between some predictors and a 3-levels outcome variable (see data below):
...
0
votes
0answers
13 views
Assessing goodness of fit for stochastic models
I'm working with rainfall data and trying to construct a stochastic model that models the data well (e.g. seasonality, dry and wet periods, duration of storms, etc.). Specifically, I'm looking at ...
0
votes
0answers
28 views
What is the figure of merite of a *likelihood* fit?
There are two main types of fit to obtain some floated parameters of a model (functional form) on a given dataset :
(i) the $\chi^2$ fit, which minimize the $\chi^2$
ii) the maximum likelihood fit, ...
