Questions tagged [goodness-of-fit]
Goodness of fit tests indicate whether or not it is reasonable to assume that a random sample comes from a specific distribution.
0
votes
0answers
8 views
Interpreting Pearson and Deviance residual graphs
Using a generalised linear model and predicted probabilities, I have been able to plot the Pearson residuals and Deviance residuals. I did this in order to have goodness of fit measures for the model.
...
0
votes
0answers
25 views
Goodness-of-fit glm: Pearson's residuals or deviance residuals?
I want to evaluate the goodness-of-fit (or badness-of-fit) of a negative binomial glm. However, even here within CV, I've seen multiple different approaches for doing so.
Some use the the residual ...
1
vote
0answers
31 views
Can I test for inequality in H0 using chi square test?
Let's say I want to test whether an $n$-sided dice is not too unfair. In the standard chi-square test we test the zero-hypothesis
$$
H_0\colon (p_1,\dots p_n) = (1/n,\dots,1/n)
,\quad\text{i.e.,}\quad
...
0
votes
0answers
9 views
Expected counts of Pearson's Chi-squared test for goodness of fit
Assume that I want to test whether two groups have the same proportion of success. Hence, my null hypothesis is $H_0: p_1 = p_2$.
I collect data from $10$ individuals per group and I observe $1$ ...
0
votes
0answers
21 views
Time-Series Test Statistics
We have observed a time-series of number of occurrences of a process over time (discrete time steps). We assume a hidden Markov model ($k$ states) governing the rate of occurrence of points in each ...
1
vote
0answers
16 views
Goodness Of Fit Testing When Multiple Parameters Are Estimated
I am working with independent data points $X_1,...,X_N$ each of which, under my model assumptions, is a sum of exponential waiting times with different rates, and so should come from gamma ...
1
vote
0answers
39 views
How to understand Chi-squared goodness-of-fit for linear regression?
I am trying to understand a claim from a paper*:
The slopes were calculated using least-squares linear regression to fit the
level versus X functions.
The goodness of fit of these linear ...
0
votes
1answer
41 views
A statistical test to check whether a set of data are Pareto distributed
I need to construct a statistical test which outputs the p value for the hypothesis, H0: Data are Pareto distributed Vs H1: Data are not Pareto distributed.
I found a test in stack exchange, but it's ...
0
votes
0answers
19 views
How to use RSSI (Received Signal Strength) data to find out a wireless channel model?
I am trying to obtain the distribution of RSSI measurements collected from 802.15.4 devices like Zolertia Re-Motes. These devices provide RSSI values in dBm units with +/-1 dBm error in measurements. ...
3
votes
1answer
117 views
Why do two implementations of the Anderson-Darling test produce such different p-values?
I want to check the normality assumption of my series
I am trying to figure out the way is computed the p-value of the Anderson Darling in R packages, goftest and <...
0
votes
1answer
81 views
Using chi square on unequal groups with unequal repeated trials
My dataset includes four groups (A, B, C, D) of unequal numbers of individuals. Each individual engages in an activity that results in one of three categorical outcomes (X, Y, Z). The individuals ...
8
votes
2answers
156 views
How can I compare models without fitting?
Regression and machine learning are used in the natural sciences to test hypotheses, estimate parameters, and make predictions by fitting models to data. However, when I have an a priori model, I don'...
0
votes
1answer
17 views
Fitting a (maybe Poisson) regression to my data in R
I'm trying to understand the probability distribution for my data. Each point represents an individual patient, the 0 or 1 label indicates whether they have a particular disease, and the number of ...
0
votes
0answers
21 views
Confidence interval of fit parameter
First of all: I am quite new to statistics, so please don't tear me apart. ;-)
I am currently trying to calculate a confidence interval for a fit of a curve to measured data points. I have done this ...
0
votes
0answers
25 views
Goodness of Fit Tests for Logistic Regression [duplicate]
I am trying to build a logistic regression model from a data set with several different features, and ~500 entries. Some of these features are discrete, some are continuous, and some are binary.
What ...
0
votes
0answers
13 views
Why the ratio of (distance from expectation)^2 / expectation in a goodness fit test follow a chi-square distribution?
I know that the sum of square of normal random variables follow a chi-square distribution. But when I learn how to do a goodness-fit test I don't know why the ratio of (O-E)^2/E follows a chi-square. ...
4
votes
1answer
60 views
Why there's never a good reason to use the Jarque-Bera test
This question was inspired by a comment made by @whuber in response to this question regarding the use of distributional tests.
The comment stated there is never any reason to use [the Jarque-Bera ...
0
votes
1answer
26 views
Is there an alternative to R squared to compare goodness of fits of different datasets? Slope makes them incomparable
I'm fitting the degradation of a signal. Some instruments degrade faster than others, so the slope varies a bit. This makes it difficult to compare how good the fits are relative to eachother.
See ...
1
vote
0answers
26 views
Goodness-of-fit test when the sample space is monotonically increasing [closed]
I came across this paper which develops a GoF test on data drawn from a circular sample space (i.e. it has cyclical support).
I am now wondering if there is a parallel to monotonic supports (or if ...
0
votes
0answers
12 views
Goodness-of-fit test applied to tuples in sequences/samples, instead of raw data
High level problem:
I have two sequences of data. The first one is the reference, the truth, the second is one which contains missed values (due to measurement errors). I want a test which can ...
0
votes
0answers
56 views
Evaluating goodness of fit for Bernoulli glm
I am trying to fit a model estimating the success probability of the Bernoulli distributed random variable with the logistic link function.
However, I am stuck with testing the goodness of fit of my ...
0
votes
1answer
27 views
How much bias am I risking by doing model goodness of fit comparison without accounting for clustering?
I am interested in testing whether an interaction term is statistically significant or not in a logistic regression. Data is large and observations are clustered by family and suffer from sparsity for ...
4
votes
2answers
74 views
Likelihood of rejecting a fair coin (repeated significance testing)
Suppose I have a fair coin and I flip it numerous times, testing after every time using Pearson's $\chi^2$ test of fit to fairness. What is the likelihood that I will, at some point, reject that the ...
0
votes
0answers
115 views
Goodness of fit by Hosmer-Lemeshow test and ROC Curve for Logistic Regression not accompanying results conclusions
I am trying to perform Logistic regression on the sample data set. After its modeling, I tried to check its goodness of fit using the Hosmer Lemeshow test and found the p-value < 0.05, which tells ...
0
votes
0answers
44 views
How do I interpret the p-value from a Shapiro-Francia Test?
I have a situation where I have more than 50 samples in a given set of inputs and I cannot use the Shapiro-Wilk test as I don't have the numbers for the pyramid for $n>50$. I was then asked to use ...
0
votes
0answers
17 views
Measure goodness of fit between a model that has error and data with error
Is there a way to express how well a model matches data where both have uncertainties? I looked for other examples that may capture this, but was unsuccessful in finding one. If you happen to know of ...
1
vote
1answer
89 views
Failing to implement Bayesian Chi2 goodness of fit test
I am trying to implement one of the methods described in Valen Johnson's A Bayesian Chi-Squared Test for Goodness of Fit. It presents a couple of variants depending on whether the random variable of ...
0
votes
0answers
16 views
Is there a relationship between the L2-norm and the Pearson chi-square test?
Suppose I have two datasets, $\mathbf{a}$ and $\mathbf{b}$, with some data weighting term (or error term), $\mathbf{e}$. I can compute the L2-norm for these two sets by the following:
$$(\mathbf{a}-\...
1
vote
2answers
53 views
Probability that a sample came from a known distribution
I'm looking for a general solution to what I assume must be a common problem because it comes up in every Bayesian calculation, but doesn't seem to be directly answered anywhere. I have an extremely ...
0
votes
0answers
15 views
Compare overall proportions between 2 samples
I want to compare multiple proportions between two samples. There are 60 brands of toys (T1, T2, T3...T60). I would like to know if overall, the proportion of respondents who own each brand of toy is ...
4
votes
1answer
108 views
Why is the deviance defined with a factor 2 (or likelihood ratio squared)?
Deviance is defined as
I see the motivation in why we would define the deviance as a difference of logLikelihoods or just the log(Likelihood Ratio), but why the factor 2? Why square the ratio? Does ...
0
votes
1answer
62 views
How to Assess Goodness of Fit of Multinomial Logit Model for Ungrouped Data?
I have some data which starts like:
...
0
votes
0answers
36 views
log transform fixed PH in Cox model - how?
I have survival data to which I am fitting a Cox model with a continuous predictor. The cumulative martingale residual method (supremum test) of Lin, Wei and Ying
suggested that both proportional ...
2
votes
1answer
27 views
What goodness-of-fit tests avoids the ambiguity in chi-square statistic?
The chi-square statistic is a fairly standard goodness-of-fit test defined as:
$$\chi^2 = \sum_k^N \left ( \frac{d_k-m_k}{\Delta d_k}\right )^2$$
where $\mathbf{d}$ is a vector of data with error $\...
0
votes
0answers
101 views
Trying to understand the Kolmogorov-Smirnov test in MATLAB
I have two sets of data, $\mathbf{d_1}$ and $\mathbf{d_2}$ which I want to test to see if they are drawn from the same distribution using the two-sample KS test in MATLAB.
My understanding of the KS ...
1
vote
1answer
389 views
Ramsey's reset test (i.e. Omitted variable bias test) result interpretation
I have the Ramsey's reset test result to find whether my regression has any omitted variable bias. I have the following result and shall I say I do or do not have omitted variable bias and why?
...
3
votes
0answers
96 views
Determining the type of data distribution
I am looking on expanding my introductory knowledge in stat, and I have come up with this challenge.
I have an application that is designed to analyze and index news articles.
The goal of the ...
0
votes
0answers
39 views
chi-square goodness of fit from the logistic regression
I am trying to get the chi-square goodness of fit from my logistic regression where the results are as below.
...
2
votes
0answers
36 views
Log-likelihood calculation on separate test set
I'm looking for a "hack" in R that would allow me to calculate the log-likelihood of a GLM fit on a separate test set easily regardless of the distribution. For instance for a Gamma GLM, this is how ...
1
vote
0answers
77 views
How to fit a distribution to binned values that come from administrative data?
Fitting a distribution to data (e.g. with maximum likelihood), or testing goodness of fit (e.g. with Kolmogorov-Smirnov) assumes that the data are randomly drawn from a population. But what if the ...
1
vote
1answer
31 views
How do I best use a fit statistic like chi-squared fit for a model that predicts two independent sets of measurements?
I have a model $M(\vec{x})$ for a vector of model parameters $\vec{x}$ that predicts two sets of measurements that I have taken - $v(h)$ and $L(h)$. The two independent data sets each have their own ...
0
votes
0answers
17 views
is there a measure for the roughness of a contour plot
There has to be a measure for the difference between "instantaneous" change of "energy" along a line in a space compared to averaged changed of energy along a line.
I could take a smooth surface in ...
1
vote
0answers
54 views
What statistics can I use to compare OLS to an ordered probit
I am trying to justify to the use of an ordered probit, my dependent variable is a survey response on a likert scale so is likely ordinal, but I wanted to provide a goodness of fit stat to back up my ...
1
vote
0answers
42 views
Why always AIC and BIC are used in mixture model than Vuong test
I am working with mixture models. I fitted more than one model to the data and then try to select the most appropriate model using different selection criteria, for example, AIC. My supervisors asked ...
0
votes
0answers
129 views
Goodness of fit in logistic regression using pchisq
I am using a logistic regression from the glm function in R.
I get the following result but my main concern is the goodness of fit where I am trying to use "pchisq" function to check this.
I have ...
1
vote
0answers
23 views
Estimation and testing of unconditional distribution in time series models
In time series models, such as ARMA-GARCH, is it possible to estimate what the unconditional distribution is? Given that the time series is auto-correlated / persistent, how many observations would be ...
1
vote
1answer
65 views
A powerful test for any distribution
Simulations by Razali et al (2011) showed that the Shapiro-Wilk test of normality provided the most power (at a fixed significance level) when compared to Anderson-Darling, Kolmogorov-Smirnov, etc. My ...
1
vote
0answers
18 views
Measure of goodness of fit of ellipsoid model to data
Consider a measurement $m$ of a physical property, the value of which depends on the direction investigated. If the property derives from the application of a second-rank symmetric tensor, the ...
2
votes
0answers
69 views
How to compare model fit of OLS and Poisson regression?
I have built two regression models to predict sales of different products based on a number of explanatory variables, with an offset term for the number of days each product was on sale. One is a ...
0
votes
0answers
29 views
Unbiased estimator for $L^2$ probability distance norm
I am trying to find an unbiased estimator for (what looks like) the $L^2$ Wasserstein distance between two probability measures. I'm pretty sure that by bickel-lehmann, there is an unbiased estimator. ...
