All Questions
Tagged with hypothesis-testing goodness-of-fit
136 questions
0
votes
0
answers
22
views
Not computed tests in gofstat in R
I am working on a survival analysis project. For this project, I use this dataset:
https://archive.ics.uci.edu/dataset/519/heart+failure+clinical+records
I began by importing these libraries :
...
- 1
0
votes
0
answers
48
views
Model comparison on data with Cauchy distribution
I am interested in determining the dependence of a state's position in density space, n, on magnetic field. With the magnetic field kept fixed, the presence of the state is measured by an increase in ...
10
votes
3
answers
37k
views
Chi-square test: difference between goodness-of-fit test and test of independence
Concerning the Pearson chi-square test there seems to be a subtle difference between the goodness-of-fit test and the test of independence.
What is confusing is that both tests seem to be calculated ...
- 290k
0
votes
0
answers
12
views
Power analysis for a goodness-of-fit test over multivariate categorical distribution
I have some data where each datapoint is described by N categorical variables. I do not know a priori whether these variables are independent from each other (but to an extreme, I may assume it), e.g.,...
- 161
5
votes
1
answer
209
views
Why in ordinary linear regression is no global test for lack of model fit unless there are replicate observations at various settings of X?
I read this quote from Regression Modelling Strategies.
In ordinary linear regression there is no global test for lack of model fit unless there are replicate observations at various settings of X. ...
- 98.4k
2
votes
1
answer
155
views
Deriving Sample version of Anderson Darling test statistic from the theoretical version
In literature, I have seen two types of Anderson-Darling test statistic. One is expressed as
$A_T^2 = n\int_{-\infty}^{\infty}\frac{(F_n(x)-F(x))^2}{F(x)(1-F(x))}dF(x)$ and the other is given by $A_s^...
2
votes
0
answers
70
views
How well does my model fit? Specifying a null-model in non-linear mixed models
I want to fit a model y ~ b * exp(-exp(a) * x), but including a random effect, with this data:
...
- 33
3
votes
1
answer
284
views
Interpretation of 'incorrect' results of chi square test
I use the chisq.test() function for the goodness of fit test. I run the test through a range of variables, and in some cases get the following message:
...
- 82.8k
0
votes
0
answers
13
views
external internal validation and chi square
The internal test set, created from 20 percent of the train data set, consists of 'a' and 'b' labels. I take the 'a' labels from the internal test and combine them with the 'c' group of another ...
- 1
1
vote
0
answers
18
views
Poor RMSEA/Fit for Simple Poisson Regression
I am running a simple Poisson regression. $X$ = time, $Y$ = count data. This is a huge dataset with many years. There is significance between $X$ and $Y$. But model shows poor fit via high RMSEA value....
- 18.4k
3
votes
2
answers
67
views
Appropriate test to use when each sample is a permutation from $S_k$
Suppose I have a dataset $x_1, \ldots, x_n$ where each $x_i$ is a permutation of $\{1, \ldots, k\}$.
[For example, if $k=4$ the data might be $x_1 = (2, 1, 3, 4)$, $x_2 = (3, 2, 4, 1)$, $x_3 = (4, 3, ...
- 2,338
33
votes
6
answers
9k
views
How can I test the fairness of a d20?
How can I test the fairness of a twenty sided die (d20)? Obviously I would be comparing the distribution of values against a uniform distribution. I vaguely remember using a Chi-square test in ...
- 290k
30
votes
7
answers
3k
views
Distribution hypothesis testing - what is the point of doing it if you can't "accept" your null hypothesis?
Various hypothesis tests, such as the $\chi^{2}$ GOF test, Kolmogorov-Smirnov, Anderson-Darling, etc., follow this basic format:
$H_0$: The data follow the given distribution.
$H_1$: The data do not ...
- 290k
2
votes
1
answer
56
views
Validating random variable generation from inverse transform sampling
I'm building a simulator and I have to implement some probability distributions. What is the best (formal) way of validating this implementation? I took a look at KS-tests but it seems to me they are ...
- 108k
6
votes
1
answer
2k
views
Statistical test for uniform distribution
I have a sample of 5 numbers from known interval [0, 10].
Are 5 numbers enough to make some conclusions about whether these numbers are drawn from uniform distribution or not?
- 10.3k
1
vote
0
answers
63
views
Interpretation of Anderson–Darling test
Assume that I am using Anderson–Darling test to evaluate whether a given sample of data is drawn from a normal distribution with some mean and standard deviation.
If you accept the null hypothesis in ...
- 363
5
votes
1
answer
3k
views
Limitations of Kolmogorov–Smirnov test
In this article, it is said that the test statistic tends to be more sensitive near the center of the distribution than at the tails. Can some please explain this what it means? (with a simple example ...
- 290k
53
votes
8
answers
75k
views
How can I test if given samples are taken from a Poisson distribution?
I know of normality tests, but how do I test for "Poisson-ness"?
I have sample of ~1000 non-negative integers, which I suspect are taken from a Poisson distribution, and I would like to test that.
- 10.3k
0
votes
0
answers
60
views
Recent goodness-of-fit tests related to proper scoring rules, CRPS and work of Gneiting
In a conference, I overheard a casual discussion about testing goodness of fit (GOF). Kolmogorov-Smirnov, Cramer-von Mises and Anderson-Darling were mentioned as some established GOF tests, but then ...
- 69.5k
25
votes
4
answers
125k
views
Whats the relationship between $R^2$ and F-Test?
I was wondering if there is a relationship between $R^2$ and a F-Test.
Usually $$R^2=\frac {\sum (\hat Y_t - \bar Y)^2 / T-1} {\sum( Y_t - \bar Y)^2 / T-1}$$ and it measures the strength of the ...
2
votes
1
answer
557
views
Compare chi-squared goodness of fit results for Poisson and Negative binomial
Steps followed for testing :
Derive parameter estimates using fitdistr() function in MASS package for the dataset, dt ...
- 12.1k
0
votes
0
answers
405
views
Testing whether a set of points on the unit sphere is uniformly distributed
The canonical way to do the test is to perform the spherical harmonic transform of the empirical distribution and then check that the power spectrum decays, but this is presumably fairly expensive. Is ...
- 82.8k
5
votes
1
answer
644
views
Can Anderson-Darling Test be performed on a very large sample of N=6362620?
Up to what sample size does Anderson-Darling Test gives reliable results on p-value?
As well as I have come across this statement for Anderson-Darling Test:
Small samples sizes tend to “fail to reject”...
- 59.4k
5
votes
3
answers
806
views
Finding the best-fitting parametric distribution for empirical data
I have an empirical dataset of samples that I'm trying to fit a (continuous) parametric distribution to from a list of possible distributions. I'm not trying to fit the parameters of a specific ...
- 2,318
0
votes
1
answer
1k
views
What does Spiegelhalter's normality test do?
What's exactly the null-hypothesis of this test?
Does it test for having a normal distribution with specified parameters or just generally?
I've searched it on the internet, but I haven't found ...
- 290k
19
votes
1
answer
12k
views
Does it make sense to perform a one-tailed Kolmogorov-Smirnov test?
Is it meaningful and possible to perform a one-tailed KS test? What would the null hypothesis of such a test be? Or is the KS test inherently a two-tailed test?
I would benefit from an answer that ...
- 290k
2
votes
0
answers
393
views
Converting a chi2 to a sigma value
I am working in astronomy. I fit a theoretical model to some data. The model takes input parameters (like the mass and age) and produces some outputs that I compare with the data.
I measured the ...
- 3,132
0
votes
0
answers
166
views
Zeros in contingency table for fisher's exact test for data drift detection
Background
I'm trying to use fisher's exact test for to measure data-drift detection between two features. The usecase is , I can retrain my model if the data has drifted. I'm trying to use fisher's ...
- 1
1
vote
0
answers
49
views
Derivation of Henze/Baringhaus Goodness-of-fit statistic for Multivariate Normality
In the paper A Consistent Test for Multivariate Normality Based on the Empirical Characteristic Function, Baringhaus and Henze define a test statistic,
$$T_n = n \int_{R^d} |\frac{1}{n} \sum_{i=1}^n ...
- 82.8k
0
votes
0
answers
42
views
Test whether a sample (which is a 1D array of values) comes from a population (2D array of values)
I'm taking samples in such a way that I record multiple discrete values for each sample. Given an instance of a sample, is there any way to test if this sample appears to "fit" the ...
- 133
1
vote
1
answer
48
views
Goodness of fit test for a transition density function of a Markov process
Suppose that you have one realization $x = \{x_n\}_{n = 1}^{N}$ of the stochastic process $X = \{X_n\}_{n = 1}^{N}$ with state space $\mathbb{R}$. Assume that the process is Markovian, time-...
- 11
0
votes
1
answer
738
views
How to do normality test for high dimension data?
I have samples from a $d$ dimensional distribution $p$. The distribution of $p$ is unknown. I want to use the samples to judge whether or not the $p$ is close to a standard unit Gaussian distribution. ...
- 82.8k
2
votes
2
answers
269
views
Why use normality tests if we have goodness-of-fit tests?
What are the reason/s to use a nonparametric normality test (e.gr., Shapiro-Wilk, Jarque-Bera) instead of generic, parametric goodness-of-fit tests (good for any distribution including but not limited ...
- 213
2
votes
0
answers
391
views
Independence Testing for Discrete Random Variable
Suppose $X$ and $Y$ are two discrete random variable take values on $\mathcal{X}$ and $\mathcal{Y}$ with their iid observation $\{ X_i\}_{i = 1}^n$ and $\{ Y_i\}_{i = 1}^n$. If both $\mathcal{X}$ and $...
- 203
0
votes
1
answer
159
views
Extension of Pearson's chi-squared test to sequences of multinomial random variables
Background
Suppose we observe $n$ IID Bernoulli variables and our null hypothesis is that their common probability is $p$.
For denote by $\mathbb{1}_{\{i\}}$ the outcome of observation $i$.
Then by ...
- 59
1
vote
1
answer
220
views
How to correctly perform a goodness-of-fit test for a contingency table (two-way, three-way, or more), in situations other than independence testing?
Let's say I have the following table from a sample of 462 people:
Gender
Happy
Meh
Sad
Men
70
32
120
Women
100
30
110
I don't want to test it against the hypothesis of independence, but against ...
- 1,771
3
votes
1
answer
83
views
Testing how well a sequence of observed word game solutions corresponds to expected word frequencies
Say we have a word game where each round involves finding a unique 5-letter word solution. (Wordle would be an example, for those familiar). For example, we may have a round where the word "magic&...
- 83
0
votes
0
answers
52
views
Comparing fit of time series to true process
I know that this question have been asked many times in this forum, but I am having troubles in understanding the correct approach to my aim.
I have several time series representing a growth process ...
1
vote
0
answers
103
views
Negative F-test value comparing two nested models
I am comparing two non-linear nested models - let me call them model A and model B. Model B has one parameter more than model A, i.e. model A can be obtained as a special case of model B. These two ...
- 11
0
votes
0
answers
26
views
2
votes
2
answers
42
views
Why can we draw a more precise conclusion when we choose a lower accepted-risk in this hypothesis-testing setting, which seems contradictory?
We want to know if 100 integer values (in a vector X) are following a Poisson $P(\lambda=2)$ distribution, which is our $H_0$ hypothesis.
Let's say the observed ...
- 133k
1
vote
0
answers
55
views
D number in Kuiper test
I want to use Kuiper test. My question is about the D number in this test. The documentation (here), mentions this:
Returns (D, fpp), where D is the Kuiper D number and fpp is the probability that a ...
- 5,114
3
votes
1
answer
679
views
Why is the deviance defined with a factor 2 (or likelihood ratio squared)?
The deviance statistic is defined as
$$D(\mathbf{y}, \hat{\mu}) = 2 \Big( \log p(\mathbf{y} | \hat{\theta}_s) - \log p(\mathbf{y} | \hat{\theta}_0) \Big),$$
where $\hat{\mu} = \mathbb{E}(Y|\hat{\theta}...
- 133k
1
vote
0
answers
213
views
How to test the goodness of fit for histograms/
There is an histogram, $h$, with user-defined $k = 5$ bins and probabilities $[1/2, 1/3, 1/30, 1/30, 1/10]$ for the each bin.
Then $1000$ histograms were simulated. It is required to establish that ...
- 856
1
vote
1
answer
54
views
P-value of alternative hypothesis - Can you simply take the difference of the chi2 goodness-of-fit tests?
I guess this is the kind of question extremely hard to Google if you're lacking just the right word.
My situation is: I have some experimental data points, and I have two models: the "simple ...
- 378
0
votes
0
answers
307
views
Goodness-of-fit test for poisson distribution
Consider the following sample of times in seconds:
5.8, 7.3, 8.9, 7.1, 8.8, 6.4, 7.2, 5.2, 10.1, 8.6, 9.0, 9.3, 6.4, 7, 9.9, 6.8
Test the hypothesis that the data is from a Poisson distribution. What ...
- 11
3
votes
1
answer
109
views
Chi Square Goodness of Fit vs. Igloo
I am very new to Chi square goodness of fit tests but have done a fair bit of research.
Basically, I have the following 12 data points:
7392,
7656,
7241,
6164,
4984,
4664,
15262,
17053,
...
- 82.8k
2
votes
0
answers
192
views
Measuring "uniqueness of fit" or "unique goodness of fit" of a model to some data
Given some parametric model, we can determine the best-fit parameters to some data set. Once we have quantified goodness of fit for each parameter set, we can easily generalize from this to measure ...
- 475
0
votes
0
answers
23
views
How can I determine statistically how well my estimator predicts actual results?
I have a business in which I contract with my clients to perform work for them for an indefinite amount of time. To forecast my profitability, I need to forecast when my existing contracts will end. ...
- 1
0
votes
0
answers
14
views
A very big JB value in Jarque-Bera test [duplicate]
I have run the JB test for my data using two different commands. I am quite clueless about what conclusion I can make from the second picture which shows results of 0. This tells me that there is no ...
- 41