Questions tagged [goodness-of-fit]
Goodness of fit tests indicate whether or not it is reasonable to assume that a random sample comes from a specific distribution.
3
votes
0answers
17 views
Interpreting contradictory goodness of fit
I am studying the effect of categorical indepdent variables on a binary outcome. To do this, I have fitted a glm as following :
...
0
votes
1answer
26 views
Goodness of fit test for a normal distribuition
I have the following exercise that shows $n=6$ numbers:
$$ 1.40, 1.55, 1.35, 1.50, 1.29, 1.64 $$
Is data normally distributed at the 5% significance level?
Surely $\overline{x} = 1.455$, $s=0....
0
votes
0answers
23 views
R: How can I calculate the F-statistic of a logistic model in R?
I am running a logistic regression in R and I noticed that the output does not include the F-statistic which shows the overall significance of the model.
In another post, the formula for the F-...
0
votes
0answers
34 views
Goodness of Fit in linear regression - to me this is not a [duplicate]
The link that is supposed to answer my question neither mentions $R^{2}$ nor GoF. At least I do not see it. What is the name and the formula to calculate the GoF in that link? Both should exist for a ...
2
votes
2answers
50 views
Adjusted R2 for model with only one independent variable?
Adjusted R2 is said to be more unbiased than ordinary R2 as it takes the number of explanatory variables into account.
Can adjusted R2 be used in a model with only an intercept and one independent ...
1
vote
1answer
25 views
Goodness of fit measurement in Python
I have a (pretty noisy) 1D array of data in Python that I fit a linear function to -- see plot below. The x-axis is a wavelength scale.
I would like to measure the "goodness of fit" of this new line ...
2
votes
1answer
28 views
Is there a goodness of fit metric that can be computed online with $O(1)$ memory?
Say I have two random streams of two dimensional data. I want to measure how closely their underlying PDF's match. My current method is to estimate the PDF's by accumulating the samples online in a ...
0
votes
0answers
14 views
Evaluating model fit for glmmTMB with beta distribution
I am looking for a way to evaluate model fit (e.g. R2, %deviance explained) for the following model:
m1<-glmmTMB(AUC_indep~woody+dispersal+log_densitys+log_seedwts+I(log_seedwts^2)+(1|modeltype), ...
0
votes
0answers
5 views
Goodness of fit test (probit data)
So I have several probit based models (not derived from my own data), which I would like to use my data on to see whether these models actually perform well on my data or not. I obviously have data to ...
0
votes
0answers
20 views
goodness-of-fit for logistic regression with a ratio dependent variable
My dependent variable is number of days in a week a certain activity occurs, so I figured I would express it as a percentage out of 7 (days) and model it using logistic regression.
I would like to ...
0
votes
0answers
16 views
Evaluate fit of distribution to separate data sets
I know there are several approaches to evaluate how well a particular distribution fits a given data set, but how could one evaluate which family of distributions to separately fit to multiple data ...
0
votes
2answers
13 views
Compare model fit (-2 log likelihood), is my model significantly better?
I have 6 models with -2 log likelihood as an indication of model fit. The fit increases from model 1 to 6, with model 1 having the worst fit (lowest -2 log likelihood) and 6 the best fit. How do I ...
3
votes
1answer
47 views
Kolmogorov Smirnov test vs. Anderson Darling test
I learned that Kolmogorov Smirnov loses sensitivity (power) in the tails, thus it is not adequate for testing goodness-of-fit of fat tailed distributions. However, Anderson Darling test is more ...
1
vote
0answers
54 views
GLM Frequency and Severity Models. How Do I improve from here? (R code) [closed]
Background:
I've been tasked with creating a rating model by Peril using GLMs. It's commercial lines property, so the data is pretty sparse. The carriers have been asking for Premiums by peril, so we'...
0
votes
3answers
54 views
Standardised mean absolute error (SMAE) and how to calculate it?
I am using the mean absolute error
mean(abs(obs - pred))
as one of the measures assessing the fit of my model. I would also like to have a standardised measure ...
0
votes
0answers
4 views
How will p-values behave when fitting normal/poisson to binomial?
I know p-values behave uniformly. Now as p(np) is fixed and n goes to infinity, binomial converges to normal(poisson). Now suppose I take random binomial samplings and fir normal(poisson) to it, for ...
1
vote
0answers
66 views
coefficient of determination (R2): why is SSR not equal to 1- SSE/SST?
I feel like this should be straight forward but I'm not sure what I'm doing wrong. My understanding is
1 = (SSR/SST) + (SSE/SST)
SST = total sum of squares
SSR = regression sum of squares
SSE = ...
0
votes
0answers
36 views
Meaning of centered vs uncentered coefficient of determination
The coefficient of determination, $R^2$, can be expressed as $$R^2=\frac{ESS}{TSS}=\frac{\hat y'M_{[1]}\hat y}{y'M_{[1]}y},$$ where $M_{[1]}$ is the residual maker for the unity vector, ESS is ...
0
votes
0answers
15 views
Are degrees-of-freedom corrections necessary or possible for the exact multinomial goodness-of-fit test?
As the title:
I have a discrete distribution with a small number of categories. I want to test if it is compatible with a parameterized distribution. If I were to do a chi-squared or G-test, I ...
2
votes
1answer
99 views
Bootstrap for goodness of fit
Suppose I want to test whether a die is fair. I toss 60 times with outcome $X = (40,20,0,0,0,0)$ where $i$th coordinate denotes times of $i$th face occurs.Consider Hypothest test($\alpha = 0.05$):
$$...
1
vote
0answers
37 views
Interpreting Pearson and Deviance residual graphs
Using a generalised linear model and predicted probabilities, I have been able to plot the Pearson residuals and Deviance residuals. I did this in order to have goodness of fit measures for the model.
...
0
votes
0answers
92 views
Goodness-of-fit glm: Pearson's residuals or deviance residuals?
I want to evaluate the goodness-of-fit (or badness-of-fit) of a negative binomial glm. However, even here within CV, I've seen multiple different approaches for doing so.
Some use the the residual ...
1
vote
0answers
34 views
Can I test for inequality in H0 using chi square test?
Let's say I want to test whether an $n$-sided dice is not too unfair. In the standard chi-square test we test the zero-hypothesis
$$
H_0\colon (p_1,\dots p_n) = (1/n,\dots,1/n)
,\quad\text{i.e.,}\quad
...
0
votes
0answers
17 views
Expected counts of Pearson's Chi-squared test for goodness of fit
Assume that I want to test whether two groups have the same proportion of success. Hence, my null hypothesis is $H_0: p_1 = p_2$.
I collect data from $10$ individuals per group and I observe $1$ ...
0
votes
0answers
27 views
Time-Series Test Statistics
We have observed a time-series of number of occurrences of a process over time (discrete time steps). We assume a hidden Markov model ($k$ states) governing the rate of occurrence of points in each ...
1
vote
0answers
25 views
Goodness Of Fit Testing When Multiple Parameters Are Estimated
I am working with independent data points $X_1,...,X_N$ each of which, under my model assumptions, is a sum of exponential waiting times with different rates, and so should come from gamma ...
1
vote
0answers
113 views
How to understand Chi-squared goodness-of-fit for linear regression?
I am trying to understand a claim from a paper*:
The slopes were calculated using least-squares linear regression to fit the
level versus X functions.
The goodness of fit of these linear ...
0
votes
1answer
55 views
A statistical test to check whether a set of data are Pareto distributed
I need to construct a statistical test which outputs the p value for the hypothesis, H0: Data are Pareto distributed Vs H1: Data are not Pareto distributed.
I found a test in stack exchange, but it's ...
0
votes
0answers
20 views
How to use RSSI (Received Signal Strength) data to find out a wireless channel model?
I am trying to obtain the distribution of RSSI measurements collected from 802.15.4 devices like Zolertia Re-Motes. These devices provide RSSI values in dBm units with +/-1 dBm error in measurements. ...
4
votes
1answer
195 views
Why do two implementations of the Anderson-Darling test produce such different p-values?
I want to check the normality assumption of my series
I am trying to figure out the way is computed the p-value of the Anderson Darling in R packages, goftest and <...
0
votes
1answer
94 views
Using chi square on unequal groups with unequal repeated trials
My dataset includes four groups (A, B, C, D) of unequal numbers of individuals. Each individual engages in an activity that results in one of three categorical outcomes (X, Y, Z). The individuals ...
8
votes
2answers
159 views
How can I compare models without fitting?
Regression and machine learning are used in the natural sciences to test hypotheses, estimate parameters, and make predictions by fitting models to data. However, when I have an a priori model, I don'...
0
votes
1answer
18 views
Fitting a (maybe Poisson) regression to my data in R
I'm trying to understand the probability distribution for my data. Each point represents an individual patient, the 0 or 1 label indicates whether they have a particular disease, and the number of ...
0
votes
0answers
33 views
Confidence interval of fit parameter
First of all: I am quite new to statistics, so please don't tear me apart. ;-)
I am currently trying to calculate a confidence interval for a fit of a curve to measured data points. I have done this ...
0
votes
0answers
35 views
Goodness of Fit Tests for Logistic Regression [duplicate]
I am trying to build a logistic regression model from a data set with several different features, and ~500 entries. Some of these features are discrete, some are continuous, and some are binary.
What ...
0
votes
0answers
16 views
Why the ratio of (distance from expectation)^2 / expectation in a goodness fit test follow a chi-square distribution?
I know that the sum of square of normal random variables follow a chi-square distribution. But when I learn how to do a goodness-fit test I don't know why the ratio of (O-E)^2/E follows a chi-square. ...
4
votes
1answer
94 views
Why there's never a good reason to use the Jarque-Bera test
This question was inspired by a comment made by @whuber in response to this question regarding the use of distributional tests.
The comment stated there is never any reason to use [the Jarque-Bera ...
0
votes
1answer
34 views
Is there an alternative to R squared to compare goodness of fits of different datasets? Slope makes them incomparable
I'm fitting the degradation of a signal. Some instruments degrade faster than others, so the slope varies a bit. This makes it difficult to compare how good the fits are relative to eachother.
See ...
1
vote
0answers
27 views
Goodness-of-fit test when the sample space is monotonically increasing [closed]
I came across this paper which develops a GoF test on data drawn from a circular sample space (i.e. it has cyclical support).
I am now wondering if there is a parallel to monotonic supports (or if ...
0
votes
0answers
12 views
Goodness-of-fit test applied to tuples in sequences/samples, instead of raw data
High level problem:
I have two sequences of data. The first one is the reference, the truth, the second is one which contains missed values (due to measurement errors). I want a test which can ...
0
votes
0answers
125 views
Evaluating goodness of fit for Bernoulli glm [duplicate]
I am trying to fit a model estimating the success probability of the Bernoulli distributed random variable with the logistic link function.
However, I am stuck with testing the goodness of fit of my ...
0
votes
1answer
27 views
How much bias am I risking by doing model goodness of fit comparison without accounting for clustering?
I am interested in testing whether an interaction term is statistically significant or not in a logistic regression. Data is large and observations are clustered by family and suffer from sparsity for ...
4
votes
2answers
75 views
Likelihood of rejecting a fair coin (repeated significance testing)
Suppose I have a fair coin and I flip it numerous times, testing after every time using Pearson's $\chi^2$ test of fit to fairness. What is the likelihood that I will, at some point, reject that the ...
0
votes
0answers
330 views
Goodness of fit by Hosmer-Lemeshow test and ROC Curve for Logistic Regression not accompanying results conclusions
I am trying to perform Logistic regression on the sample data set. After its modeling, I tried to check its goodness of fit using the Hosmer Lemeshow test and found the p-value < 0.05, which tells ...
0
votes
0answers
46 views
How do I interpret the p-value from a Shapiro-Francia Test?
I have a situation where I have more than 50 samples in a given set of inputs and I cannot use the Shapiro-Wilk test as I don't have the numbers for the pyramid for $n>50$. I was then asked to use ...
0
votes
0answers
20 views
Measure goodness of fit between a model that has error and data with error
Is there a way to express how well a model matches data where both have uncertainties? I looked for other examples that may capture this, but was unsuccessful in finding one. If you happen to know of ...
1
vote
1answer
99 views
Failing to implement Bayesian Chi2 goodness of fit test
I am trying to implement one of the methods described in Valen Johnson's A Bayesian Chi-Squared Test for Goodness of Fit. It presents a couple of variants depending on whether the random variable of ...
0
votes
0answers
27 views
Is there a relationship between the L2-norm and the Pearson chi-square test?
Suppose I have two datasets, $\mathbf{a}$ and $\mathbf{b}$, with some data weighting term (or error term), $\mathbf{e}$. I can compute the L2-norm for these two sets by the following:
$$(\mathbf{a}-\...
1
vote
2answers
59 views
Probability that a sample came from a known distribution
I'm looking for a general solution to what I assume must be a common problem because it comes up in every Bayesian calculation, but doesn't seem to be directly answered anywhere. I have an extremely ...
0
votes
0answers
15 views
Compare overall proportions between 2 samples
I want to compare multiple proportions between two samples. There are 60 brands of toys (T1, T2, T3...T60). I would like to know if overall, the proportion of respondents who own each brand of toy is ...
