Questions tagged [goodness-of-fit]

Goodness of fit tests indicate whether or not it is reasonable to assume that a random sample comes from a specific distribution.

Filter by
Sorted by
Tagged with
19
votes
1answer
6k views

Goodness of fit and which model to choose linear regression or Poisson

I need some advice regarding two main dilemmas in my research, which is a case study of 3 big pharmaceuticals and innovation. Number of patents per year is the dependent variable. My questions are ...
12
votes
3answers
2k views

Is my model any good, based on the diagnostic metric ($R^2$/ AUC/ accuracy/ RMSE etc.) value?

I've fitted my model and am trying to understand whether it's any good. I've calculated the recommended metrics to assess it ($R^2$/ AUC / accuracy / prediction error / etc) but do not know how to ...
58
votes
7answers
70k views

Which pseudo-$R^2$ measure is the one to report for logistic regression (Cox & Snell or Nagelkerke)?

I have SPSS output for a logistic regression model. The output reports two measures for the model fit, Cox & Snell and ...
141
votes
2answers
190k views

How to determine which distribution fits my data best?

I have a dataset and would like to figure out which distribution fits my data best. I used the fitdistr() function to estimate the necessary parameters to ...
39
votes
2answers
53k views

PP-plots vs. QQ-plots

What is the difference between probability plots, PP-plots and QQ-plots when trying to analyse a fitted distribution to data?
7
votes
2answers
9k views

Testing normality

I have a large dataset (500000 data, V1 column include all the data). x <- read.csv("mydata.csv", header=F) hist(x) Which gives: Looking at the data, I ...
22
votes
6answers
16k views

Appropriate normality tests for small samples

So far, I've been using the Shapiro-Wilk statistic in order to test normality assumptions in small samples. Could you please recommend another technique?
26
votes
7answers
2k views

Distribution hypothesis testing - what is the point of doing it if you can't “accept” your null hypothesis?

Various hypothesis tests, such as the $\chi^{2}$ GOF test, Kolmogorov-Smirnov, Anderson-Darling, etc., follow this basic format: $H_0$: The data follow the given distribution. $H_1$: The data do not ...
28
votes
6answers
7k views

How can I test the fairness of a d20?

How can I test the fairness of a twenty sided die (d20)? Obviously I would be comparing the distribution of values against a uniform distribution. I vaguely remember using a Chi-square test in ...
7
votes
1answer
4k views

Goodness of fit test for a mixture in R

I just estimated the parameters for a mixture of two gaussians with different means and different sigmas, I would like to test if the data adjusts well to the explicit form of the mixture, do I ...
41
votes
8answers
53k views

How can I test if given samples are taken from a Poisson distribution?

I know of normality tests, but how do I test for "Poisson-ness"? I have sample of ~1000 non-negative integers, which I suspect are taken from a Poisson distribution, and I would like to test that.
17
votes
4answers
67k views

Whats the relationship between $R^2$ and F-Test?

I was wondering if there is a relationship between $R^2$ and a F-Test. Usually $$R^2=\frac {\sum (\hat Y_t - \bar Y)^2 / T-1} {\sum( Y_t - \bar Y)^2 / T-1}$$ and it measures the strength of the ...
1
vote
0answers
340 views

Why is R2 not reported for GLMs based on last iteration of IRLS weighted least square regression with which it is fit

Given that GLMs are generally fit using iteratively reweighted least squares (based on a Fisher scoring algorithm to maximize the max likelihood objective, which is a variant of Newton-Raphson, see ...
24
votes
3answers
33k views

Evaluating logistic regression and interpretation of Hosmer-Lemeshow Goodness of Fit

As we all know, there are 2 methods to evaluate the logistic regression model and they are testing very different things Predictive power: Get a statistic that measures how well you can predict the ...
29
votes
6answers
165k views

Interpretation of Shapiro-Wilk test

I'm pretty new to statistics and I need your help. I have a small sample, as follows: H4U 0.269 0.357 0.2 0.221 0.275 0.277 0.253 0.127 0.246 ...
3
votes
2answers
4k views

Is my data gamma distributed?

I have some data which looks like this when I plot a normalized histogram. The full data set is available here and here (the second link is pastebin). It is 20,000 lines long. My guess is that it ...
4
votes
2answers
2k views

Regression: Scatterplot with low R squared and high p-values

Based on three datasets, I have produced the scatterplot below in Python: I am trying to fit a line on each dataset, but when I check the metrics this is what I get: Set 1 (red): $R^2$=0.002, p-...
19
votes
1answer
5k views

Goodness of fit for 2D histograms

I have two sets of data representing stars parameters: an observed one and a modeled one. With these sets I create what is called a two-color-diagram (TCD). A sample can be seen here: A being the ...
12
votes
1answer
11k views

Logistic Regression with regression splines in R

I have been developing a logistic regression model based on retrospective data from a national trauma database of head injury in the UK. The key outcome is 30 day mortality (denoted as "Survive" ...
33
votes
2answers
3k views

Degrees of freedom of $\chi^2$ in Hosmer-Lemeshow test

The test statistic for the Hosmer-Lemeshow test (HLT) for goodness of fit (GOF) of a logistic regression model is defined as follows: The sample is then split into $d=10$ deciles, $D_1, D_2, \dots ...
18
votes
2answers
3k views

Impact of data-based bin boundaries on a chi-square goodness of fit test?

Leaving aside the obvious issue of the low power of the chi-square in this sort of circumstance, imagine doing a chi-square goodness of test for some density with unspecified parameters, by binning ...
17
votes
3answers
34k views

What does negative R-squared mean?

Let's say I have some data, and then I fit the data with a model (a non-linear regression). Then I calculate the R-squared ($R^2$). When R-squared is negative, what does that mean? Does that mean my ...
13
votes
2answers
18k views

How to test whether a sample of data fits the family of Gamma distribution?

I have a sample of data which was generated from a continuous random variable X. And from the histogram I draw using R, I guess that maybe the distribution of X obeys a certain Gamma distribution. But ...
15
votes
1answer
5k views

Does it make sense to perform a one-tailed Kolmogorov-Smirnov test?

Is it meaningful and possible to perform a one-tailed KS test? What would the null hypothesis of such a test be? Or is the KS test inherently a two-tailed test? I would benefit from an answer that ...
22
votes
3answers
65k views

How do I check if my data fits an exponential distribution?

How could I check if my data e.g. salary is from a continuous exponential distribution in R? Here is histogram of my sample: . Any help will be greatly appreciated!
24
votes
1answer
11k views

Kolmogorov-Smirnov with discrete data: What is proper use of dgof::ks.test in R?

Beginner questions: I want to test whether two discrete data sets come from the same distribution. A Kolmogorov-Smirnov test was suggested to me. Conover (Practical Nonparametric Statistics, 3d) ...
16
votes
5answers
4k views

Checking ANOVA assumptions

A few months ago I posted a question about homoscedasticity tests in R on SO, and Ian Fellows answered that (I'll paraphrase his answer very loosely): Homoscedasticity tests are not a good tool ...
20
votes
2answers
11k views

Is a weighted $R^2$ in robust linear model meaningful for goodness of fit analysis?

I estimated a robust linear model in R with MM weights using the rlm() in the MASS package. `R`` does not provide an $R^2$ value ...
8
votes
2answers
5k views

How can $R^2$ have two different values for the same regression (without an intercept) [duplicate]

My question is multifaceted. So I'll start by asking my question and then explain what has caused me to ask this question. How can I calculate the coefficient of determination for a linear ...
5
votes
1answer
4k views

ARMA-GARCH model selection / fit evaluation

I'm trying to fit an ARMA-GARCH model to a data set of FTSE 100 log returns (which I've uploaded here). However, I'm not able to find a well-fitting model. Below are the ACF and PACF of the log return ...
16
votes
2answers
11k views

Post-hoc test for chi-square goodness-of-fit test

I'm conducting a chi-square goodness-of-fit (GOF) test with three categories and specifically want to test the null that the population proportions in each category are equal (i.e., the proportion is ...
8
votes
1answer
1k views

Goodness of fit for discrete data: best approach

The data: For the purposes of this question/communication we can assume the data looks like rnbinom(1000,size=0.1,prob=0.01) in R, which generates a random sample ...
5
votes
1answer
5k views

Goodness of fit in a GLM with scaled deviance

On this page, I am interested in the section “goodness of fit”, which is near the bottom of the page and contains the table of deviance functions. The author states that the scaled deviance, i.e. $D^...
8
votes
1answer
2k views

Multi-dimensional goodness of fit

I have two (soon to be three) dimensional spatial data and I what to compare the goodness of fit between my model and reality. As the data is continuous and not categorial I can't really use the Chi-...
6
votes
3answers
5k views

Zero-inflated Poisson regression Vuong test: Raw, AIC- or BIC-corrected results

I'm analyzing count data for a set of ten species and found that for the five species with highest detection rate, the zero-inflated poisson (ZIP) regression fits the data significantly better than ...
4
votes
2answers
2k views

Understanding lack of fit in logistic regression

How does one interpret the fact that a dataset has a poor fit / lack of fit with respect to a logistic regression model? I can make sense, for example, of a lack of fit in the case of a linear ...
3
votes
2answers
4k views

Computing chi-square for large tables with some expected cell counts equal to zero

What should I do if the expected value in a Chi-square goodness-of-fit test is zero? I know there's Fisher's test but I have a very large table!
2
votes
1answer
1k views

Validation of logistic regression - goodness of fit (pearson)

I have developed a scoring system using logistic regression. The score ranges between 0 and 6 (using integers) and predicts death. It does not use a conventional regression formula and thus I am not ...
1
vote
1answer
2k views

Comparing proportions in two samples

(x-posted to Statalist) I have two (probably) independent samples taken from the same population under two different surveys conducted a few years apart. I'd like to check whether the frequencies for ...
32
votes
2answers
26k views

Raw residuals versus standardised residuals versus studentised residuals - what to use when?

This looks like a similar question and didn't get many responses. Omitting tests such as Cook's D, and just looking at residuals as a group, I am interested in how others use residuals when assessing ...
5
votes
2answers
10k views

How to choose the most appropriate distribution for a given vector in R?

When creating a Monte Carlo simulation model for a variable, a critical step is to choose the distribution that best fits the variable's probability density. I generally do this by looking at the ...
13
votes
1answer
9k views

How to test whether a distribution follows a power law?

I have data on how many users post how many questions. For example, [UserCount, QuestionCount] [2, 100] [9, 10] [3, 80] ... ... This means that 2 users ...
10
votes
1answer
29k views

How to draw fitted graph and actual graph of gamma distribution in one plot?

Load the package needed. library(ggplot2) library(MASS) Generate 10,000 numbers fitted to gamma distribution. ...
21
votes
2answers
9k views

How to measure/argue the goodness of fit of a trendline to a power law?

I have some data to which I am trying to fit a trendline. I believe the data to follow a power law, and so have plotted the data on log-log axes looking for a straight line. This has resulted in an (...
7
votes
3answers
5k views

Fitting mixture distributions and computing goodness-of-fit?

This question is a follow-up from a previous question of mine here. Thanks to @Glen_b, @gung and @rbatt for teaching me so many new things yesterday. It was mentioned in passing that mixture ...
13
votes
2answers
7k views

Assessing logistic regression models

This question arises from my actual confusion about how to decide if a logistic model is good enough. I have models that use the state of pairs individual-project two years after they are formed as a ...
6
votes
2answers
6k views

What is a “good fit” Brier score and Harrell's C Index

This is a question I originally posted on r-help but it is more suited here. I will post the question and the answer I received from Dr. Winsemius and would be most grateful for any additional answers ...
4
votes
1answer
10k views

Does my data come from a gamma or beta distribution? [closed]

I have data and I want to ascertain whether it is beta or gamma distribution. Once I know what the distribution is, how do I find out what the parameters are?
6
votes
2answers
375 views

How to Assess the Fit of Thousands of Distributions?

I have thousands of subjects, for each of whom I have a fitted gamma distribution, with the parameters estimated from each subject's data. It is easy to look at the distribution for one subject (say ...
4
votes
0answers
2k views

chi-squared goodness-of-fit: effect size and power

How can effect size and power of the chi-square goodness-of-fit test be calculated such that: Effect size is linear and standardized (given effect size $E_{x^2}$, $0 \le E_{x^2} \le 1$) so that ...