All Questions
Tagged with hypothesis-testing goodness-of-fit
136 questions
0
votes
0
answers
22
views
Not computed tests in gofstat in R
I am working on a survival analysis project. For this project, I use this dataset:
https://archive.ics.uci.edu/dataset/519/heart+failure+clinical+records
I began by importing these libraries :
...
- 1
0
votes
0
answers
48
views
Model comparison on data with Cauchy distribution
I am interested in determining the dependence of a state's position in density space, n, on magnetic field. With the magnetic field kept fixed, the presence of the state is measured by an increase in ...
0
votes
0
answers
12
views
Power analysis for a goodness-of-fit test over multivariate categorical distribution
I have some data where each datapoint is described by N categorical variables. I do not know a priori whether these variables are independent from each other (but to an extreme, I may assume it), e.g.,...
- 161
5
votes
1
answer
209
views
Why in ordinary linear regression is no global test for lack of model fit unless there are replicate observations at various settings of X?
I read this quote from Regression Modelling Strategies.
In ordinary linear regression there is no global test for lack of model fit unless there are replicate observations at various settings of X. ...
- 761
0
votes
0
answers
13
views
external internal validation and chi square
The internal test set, created from 20 percent of the train data set, consists of 'a' and 'b' labels. I take the 'a' labels from the internal test and combine them with the 'c' group of another ...
- 1
1
vote
0
answers
18
views
Poor RMSEA/Fit for Simple Poisson Regression
I am running a simple Poisson regression. $X$ = time, $Y$ = count data. This is a huge dataset with many years. There is significance between $X$ and $Y$. But model shows poor fit via high RMSEA value....
- 11
2
votes
0
answers
70
views
How well does my model fit? Specifying a null-model in non-linear mixed models
I want to fit a model y ~ b * exp(-exp(a) * x), but including a random effect, with this data:
...
- 33
2
votes
1
answer
155
views
Deriving Sample version of Anderson Darling test statistic from the theoretical version
In literature, I have seen two types of Anderson-Darling test statistic. One is expressed as
$A_T^2 = n\int_{-\infty}^{\infty}\frac{(F_n(x)-F(x))^2}{F(x)(1-F(x))}dF(x)$ and the other is given by $A_s^...
- 305
3
votes
2
answers
67
views
Appropriate test to use when each sample is a permutation from $S_k$
Suppose I have a dataset $x_1, \ldots, x_n$ where each $x_i$ is a permutation of $\{1, \ldots, k\}$.
[For example, if $k=4$ the data might be $x_1 = (2, 1, 3, 4)$, $x_2 = (3, 2, 4, 1)$, $x_3 = (4, 3, ...
- 2,338
1
vote
0
answers
63
views
Interpretation of Anderson–Darling test
Assume that I am using Anderson–Darling test to evaluate whether a given sample of data is drawn from a normal distribution with some mean and standard deviation.
If you accept the null hypothesis in ...
- 363
0
votes
0
answers
60
views
Recent goodness-of-fit tests related to proper scoring rules, CRPS and work of Gneiting
In a conference, I overheard a casual discussion about testing goodness of fit (GOF). Kolmogorov-Smirnov, Cramer-von Mises and Anderson-Darling were mentioned as some established GOF tests, but then ...
- 69.5k
2
votes
1
answer
557
views
Compare chi-squared goodness of fit results for Poisson and Negative binomial
Steps followed for testing :
Derive parameter estimates using fitdistr() function in MASS package for the dataset, dt ...
- 261
0
votes
0
answers
405
views
Testing whether a set of points on the unit sphere is uniformly distributed
The canonical way to do the test is to perform the spherical harmonic transform of the empirical distribution and then check that the power spectrum decays, but this is presumably fairly expensive. Is ...
- 337
5
votes
1
answer
644
views
Can Anderson-Darling Test be performed on a very large sample of N=6362620?
Up to what sample size does Anderson-Darling Test gives reliable results on p-value?
As well as I have come across this statement for Anderson-Darling Test:
Small samples sizes tend to “fail to reject”...
- 51
2
votes
0
answers
393
views
Converting a chi2 to a sigma value
I am working in astronomy. I fit a theoretical model to some data. The model takes input parameters (like the mass and age) and produces some outputs that I compare with the data.
I measured the ...
- 3,132
0
votes
0
answers
166
views
Zeros in contingency table for fisher's exact test for data drift detection
Background
I'm trying to use fisher's exact test for to measure data-drift detection between two features. The usecase is , I can retrain my model if the data has drifted. I'm trying to use fisher's ...
- 1
0
votes
0
answers
42
views
Test whether a sample (which is a 1D array of values) comes from a population (2D array of values)
I'm taking samples in such a way that I record multiple discrete values for each sample. Given an instance of a sample, is there any way to test if this sample appears to "fit" the ...
- 133
1
vote
1
answer
48
views
Goodness of fit test for a transition density function of a Markov process
Suppose that you have one realization $x = \{x_n\}_{n = 1}^{N}$ of the stochastic process $X = \{X_n\}_{n = 1}^{N}$ with state space $\mathbb{R}$. Assume that the process is Markovian, time-...
- 11
2
votes
0
answers
391
views
Independence Testing for Discrete Random Variable
Suppose $X$ and $Y$ are two discrete random variable take values on $\mathcal{X}$ and $\mathcal{Y}$ with their iid observation $\{ X_i\}_{i = 1}^n$ and $\{ Y_i\}_{i = 1}^n$. If both $\mathcal{X}$ and $...
- 203
2
votes
2
answers
269
views
Why use normality tests if we have goodness-of-fit tests?
What are the reason/s to use a nonparametric normality test (e.gr., Shapiro-Wilk, Jarque-Bera) instead of generic, parametric goodness-of-fit tests (good for any distribution including but not limited ...
- 213
2
votes
1
answer
56
views
Validating random variable generation from inverse transform sampling
I'm building a simulator and I have to implement some probability distributions. What is the best (formal) way of validating this implementation? I took a look at KS-tests but it seems to me they are ...
0
votes
1
answer
159
views
Extension of Pearson's chi-squared test to sequences of multinomial random variables
Background
Suppose we observe $n$ IID Bernoulli variables and our null hypothesis is that their common probability is $p$.
For denote by $\mathbb{1}_{\{i\}}$ the outcome of observation $i$.
Then by ...
- 59
1
vote
1
answer
220
views
How to correctly perform a goodness-of-fit test for a contingency table (two-way, three-way, or more), in situations other than independence testing?
Let's say I have the following table from a sample of 462 people:
Gender
Happy
Meh
Sad
Men
70
32
120
Women
100
30
110
I don't want to test it against the hypothesis of independence, but against ...
- 5,848
3
votes
1
answer
83
views
Testing how well a sequence of observed word game solutions corresponds to expected word frequencies
Say we have a word game where each round involves finding a unique 5-letter word solution. (Wordle would be an example, for those familiar). For example, we may have a round where the word "magic&...
- 83
0
votes
0
answers
52
views
Comparing fit of time series to true process
I know that this question have been asked many times in this forum, but I am having troubles in understanding the correct approach to my aim.
I have several time series representing a growth process ...
1
vote
0
answers
103
views
Negative F-test value comparing two nested models
I am comparing two non-linear nested models - let me call them model A and model B. Model B has one parameter more than model A, i.e. model A can be obtained as a special case of model B. These two ...
- 11
2
votes
2
answers
42
views
Why can we draw a more precise conclusion when we choose a lower accepted-risk in this hypothesis-testing setting, which seems contradictory?
We want to know if 100 integer values (in a vector X) are following a Poisson $P(\lambda=2)$ distribution, which is our $H_0$ hypothesis.
Let's say the observed ...
- 622
0
votes
0
answers
26
views
1
vote
0
answers
55
views
D number in Kuiper test
I want to use Kuiper test. My question is about the D number in this test. The documentation (here), mentions this:
Returns (D, fpp), where D is the Kuiper D number and fpp is the probability that a ...
- 171
3
votes
1
answer
283
views
Interpretation of 'incorrect' results of chi square test
I use the chisq.test() function for the goodness of fit test. I run the test through a range of variables, and in some cases get the following message:
...
- 97
1
vote
0
answers
213
views
How to test the goodness of fit for histograms/
There is an histogram, $h$, with user-defined $k = 5$ bins and probabilities $[1/2, 1/3, 1/30, 1/30, 1/10]$ for the each bin.
Then $1000$ histograms were simulated. It is required to establish that ...
- 856
1
vote
1
answer
54
views
P-value of alternative hypothesis - Can you simply take the difference of the chi2 goodness-of-fit tests?
I guess this is the kind of question extremely hard to Google if you're lacking just the right word.
My situation is: I have some experimental data points, and I have two models: the "simple ...
- 13
0
votes
0
answers
307
views
Goodness-of-fit test for poisson distribution
Consider the following sample of times in seconds:
5.8, 7.3, 8.9, 7.1, 8.8, 6.4, 7.2, 5.2, 10.1, 8.6, 9.0, 9.3, 6.4, 7, 9.9, 6.8
Test the hypothesis that the data is from a Poisson distribution. What ...
- 11
2
votes
0
answers
192
views
Measuring "uniqueness of fit" or "unique goodness of fit" of a model to some data
Given some parametric model, we can determine the best-fit parameters to some data set. Once we have quantified goodness of fit for each parameter set, we can easily generalize from this to measure ...
- 475
0
votes
0
answers
23
views
How can I determine statistically how well my estimator predicts actual results?
I have a business in which I contract with my clients to perform work for them for an indefinite amount of time. To forecast my profitability, I need to forecast when my existing contracts will end. ...
- 1
0
votes
1
answer
738
views
How to do normality test for high dimension data?
I have samples from a $d$ dimensional distribution $p$. The distribution of $p$ is unknown. I want to use the samples to judge whether or not the $p$ is close to a standard unit Gaussian distribution. ...
- 215
0
votes
0
answers
14
views
A very big JB value in Jarque-Bera test [duplicate]
I have run the JB test for my data using two different commands. I am quite clueless about what conclusion I can make from the second picture which shows results of 0. This tells me that there is no ...
- 41
2
votes
0
answers
85
views
goodness of fit for psychometric data (perceptual threshold)
I'm running an experiment on perceptual thresholds in audio. I'll try not to bog you down with too many details:
The experiment is about vibrato speed; specifically, when can you tell the difference ...
- 21
4
votes
1
answer
236
views
Is there a G-test equivalent for continuous variables?
The G-test is similar to the chi-square test for goodness of fit. It is proportional to the kl-divergence.
I am wondering if there is a similar test that is applicable to continuous variables. Since ...
- 21.9k
0
votes
0
answers
19
views
Determining trend statistic for two time series
I have two time series: one with reported scatter plot points, and a solid line that represents a fit to the data based on other variables. Here:
https://docs.google.com/document/d/1gizOAV8ZjaATLm-...
0
votes
0
answers
253
views
Chi-squared test, Poisson distribution, type I error overestimated - well-suited test for discrete distributions?
UPDATE I edited my original question to make it as clear as possible.
My goal is to find a reliable goodness-of-fit test for Poisson-distributed samples. There are a few discussions here related to ...
- 126
1
vote
1
answer
159
views
goodness-of-fit and bootstrap
Assume one has two data samples: $X = \{ x_{1}, \dots, x_{n} \}$ and $Y = \{y_{1}, \dots, y_{m}\}$. Next, we aim to check if the data $Y$ was generated by the same data generating process (DGP) as $X$ ...
- 668
0
votes
1
answer
29
views
Is it possible to calculate a p-value from a modified test?
I was asked to fit a distribution to some data and calculate the goodness-of-fit and a corresponding p-value. I've been using Pearsons chi-squared test (chi2gof in matlab) to do this but my advisor ...
0
votes
1
answer
53
views
Wrong answer in basic goodness-of-fit test
I am following my lecture notes on this test:
However, when I calculate the expression $2\log \Lambda$ (Python script attached below), I get $21.8$ instead of $44.9$, which is quite far off. The ...
- 101
0
votes
0
answers
218
views
Goodness of fit that puts high weight towards the tail of the distributions
I have two distributions A and B and I am looking for a goodness of fit test that measures how much the tail of A matches (or fail to match) the tail of B. Alternatively, I am looking for a test that ...
- 385
1
vote
1
answer
126
views
Comparing two discrete paired datasets
The problem: I am studying the estimated number of cases of malaria in regions of the world and found that both the WHO and IHME have their own estimates. I want to find if the difference between the ...
- 11
2
votes
1
answer
55
views
Statistical test to detect regional excess of observations compared to background observations
I have data on the protein position of genetic variants. I want to determine whether there is a region of the proteins with a significant excess of variants relative to controls.
Consider this ...
- 139
0
votes
0
answers
31
views
How to decide between goodness of fit tests
I have a question on BIC and AIC.
I have a data set and I need to test this data set if it fits various distributions (for example, Gamma or Poisson, etc.) I need to use AIC and BIC statistics for ...
- 1
1
vote
0
answers
137
views
Convergence rate of test-statistic to chi-square distribution
I know that the to test whether $\Sigma=\Sigma_0$ against $\Sigma\ne\Sigma_0$ for an $n\times p$ data matrix, the test statistic is $np(a-1-\log g)$ where $a$ and $g$ are the AM and GM of the eigen ...
- 545
1
vote
2
answers
399
views
What does goodness of fit tell us with skewed data?
"The goodness of fit test is a statistical hypothesis test to see how well sample data fit a distribution from a population with a normal distribution. Put differently, this test shows if your sample ...
- 25