Questions tagged [goodness-of-fit]
Goodness of fit tests indicate whether or not it is reasonable to assume that a random sample comes from a specific distribution.
0
votes
1answer
24 views
Is there an alternative to R squared to compare goodness of fits of different datasets? Slope makes them incomparable
I'm fitting the degradation of a signal. Some instruments degrade faster than others, so the slope varies a bit. This makes it difficult to compare how good the fits are relative to eachother.
See ...
0
votes
0answers
11 views
Goodness-of-fit test when the sample space is monotonically increasing
I came across this paper which develops a GoF test on data drawn from a circular sample space (i.e. it has cyclical support).
I am now wondering if there is a parallel to monotonic supports (or if ...
0
votes
0answers
12 views
Goodness-of-fit test applied to tuples in sequences/samples, instead of raw data
High level problem:
I have two sequences of data. The first one is the reference, the truth, the second is one which contains missed values (due to measurement errors). I want a test which can ...
0
votes
0answers
11 views
R: How can I make a graph showing the distribution of my model vs the actual distribution by decile? [closed]
At my company, we currently use a program that spits out the below graph for model fit. and I'm trying to replicate it in R. The graph splits it into deciles, then shows a line of the actual ...
0
votes
0answers
25 views
Evaluating goodness of fit for Bernoulli glm
I am trying to fit a model estimating the success probability of the Bernoulli distributed random variable with the logistic link function.
However, I am stuck with testing the goodness of fit of my ...
0
votes
0answers
19 views
Kolmogorov Smirnov test for multivariate Generalized Hyperbolic Distribution in R [closed]
I am trying to make a goodness of fit for multivariate generalized hyperbolic distribution in R. My data set is the returns for 2 stocks of s&p500.
First, I fit a GHD and find the coeficients, ...
0
votes
0answers
8 views
Changing group number in Hosmer Lemeshow Test in SPSS [closed]
I am using SPSS to run a Binary Logistic regression. I have the option of choosing Hosmer-Lemeshow goodness of fit test but see that SPSS chooses the default group size of 10 for the test. I want to ...
0
votes
1answer
26 views
How much bias am I risking by doing model goodness of fit comparison without accounting for clustering?
I am interested in testing whether an interaction term is statistically significant or not in a logistic regression. Data is large and observations are clustered by family and suffer from sparsity for ...
4
votes
2answers
73 views
Likelihood of rejecting a fair coin (repeated significance testing)
Suppose I have a fair coin and I flip it numerous times, testing after every time using Pearson's $\chi^2$ test of fit to fairness. What is the likelihood that I will, at some point, reject that the ...
0
votes
0answers
39 views
Goodness of fit by Hosmer-Lemeshow test and ROC Curve for Logistic Regression not accompanying results conclusions
I am trying to perform Logistic regression on the sample data set. After its modeling, I tried to check its goodness of fit using the Hosmer Lemeshow test and found the p-value < 0.05, which tells ...
0
votes
0answers
42 views
How do I interpret the p-value from a Shapiro-Francia Test?
I have a situation where I have more than 50 samples in a given set of inputs and I cannot use the Shapiro-Wilk test as I don't have the numbers for the pyramid for $n>50$. I was then asked to use ...
0
votes
0answers
15 views
Measure goodness of fit between a model that has error and data with error
Is there a way to express how well a model matches data where both have uncertainties? I looked for other examples that may capture this, but was unsuccessful in finding one. If you happen to know of ...
1
vote
1answer
84 views
Failing to implement Bayesian Chi2 goodness of fit test
I am trying to implement one of the methods described in Valen Johnson's A Bayesian Chi-Squared Test for Goodness of Fit. It presents a couple of variants depending on whether the random variable of ...
0
votes
0answers
11 views
Is there a relationship between the L2-norm and the Pearson chi-square test?
Suppose I have two datasets, $\mathbf{a}$ and $\mathbf{b}$, with some data weighting term (or error term), $\mathbf{e}$. I can compute the L2-norm for these two sets by the following:
$$(\mathbf{a}-\...
1
vote
2answers
46 views
Probability that a sample came from a known distribution
I'm looking for a general solution to what I assume must be a common problem because it comes up in every Bayesian calculation, but doesn't seem to be directly answered anywhere. I have an extremely ...
0
votes
0answers
15 views
Compare overall proportions between 2 samples
I want to compare multiple proportions between two samples. There are 60 brands of toys (T1, T2, T3...T60). I would like to know if overall, the proportion of respondents who own each brand of toy is ...
4
votes
1answer
48 views
Why is the deviance defined with a factor 2 (or likelihood ratio squared)?
Deviance is defined as
I see the motivation in why we would define the deviance as a difference of logLikelihoods or just the log(Likelihood Ratio), but why the factor 2? Why square the ratio? Does ...
0
votes
1answer
37 views
How to Assess Goodness of Fit of Multinomial Logit Model for Ungrouped Data?
I have some data which starts like:
...
0
votes
0answers
27 views
log transform fixed PH in Cox model - how?
I have survival data to which I am fitting a Cox model with a continuous predictor. The cumulative martingale residual method (supremum test) of Lin, Wei and Ying
suggested that both proportional ...
1
vote
1answer
24 views
What goodness-of-fit tests avoids the ambiguity in chi-square statistic?
The chi-square statistic is a fairly standard goodness-of-fit test defined as:
$$\chi^2 = \sum_k^N \left ( \frac{d_k-m_k}{\Delta d_k}\right )^2$$
where $\mathbf{d}$ is a vector of data with error $\...
0
votes
0answers
45 views
Trying to understand the Kolmogorov-Smirnov test in MATLAB
I have two sets of data, $\mathbf{d_1}$ and $\mathbf{d_2}$ which I want to test to see if they are drawn from the same distribution using the two-sample KS test in MATLAB.
My understanding of the KS ...
-1
votes
0answers
17 views
Fitting data on a time demanding stochastic Model
I have a multi-parameter (8 at least) model which is very time consuming. It's not an analytical function but instead is a model which integrates many differential equation and some times the result ...
1
vote
1answer
232 views
Ramsey's reset test (i.e. Omitted variable bias test) result interpretation
I have the Ramsey's reset test result to find whether my regression has any omitted variable bias. I have the following result and shall I say I do or do not have omitted variable bias and why?
...
3
votes
0answers
80 views
Determining the type of data distribution
I am looking on expanding my introductory knowledge in stat, and I have come up with this challenge.
I have an application that is designed to analyze and index news articles.
The goal of the ...
0
votes
0answers
28 views
chi-square goodness of fit from the logistic regression
I am trying to get the chi-square goodness of fit from my logistic regression where the results are as below.
...
2
votes
0answers
28 views
Log-likelihood calculation on separate test set
I'm looking for a "hack" in R that would allow me to calculate the log-likelihood of a GLM fit on a separate test set easily regardless of the distribution. For instance for a Gamma GLM, this is how ...
1
vote
0answers
47 views
How to fit a distribution to binned values that come from administrative data?
Fitting a distribution to data (e.g. with maximum likelihood), or testing goodness of fit (e.g. with Kolmogorov-Smirnov) assumes that the data are randomly drawn from a population. But what if the ...
1
vote
1answer
25 views
How do I best use a fit statistic like chi-squared fit for a model that predicts two independent sets of measurements?
I have a model $M(\vec{x})$ for a vector of model parameters $\vec{x}$ that predicts two sets of measurements that I have taken - $v(h)$ and $L(h)$. The two independent data sets each have their own ...
0
votes
0answers
15 views
is there a measure for the roughness of a contour plot
There has to be a measure for the difference between "instantaneous" change of "energy" along a line in a space compared to averaged changed of energy along a line.
I could take a smooth surface in ...
1
vote
0answers
47 views
What statistics can I use to compare OLS to an ordered probit
I am trying to justify to the use of an ordered probit, my dependent variable is a survey response on a likert scale so is likely ordinal, but I wanted to provide a goodness of fit stat to back up my ...
1
vote
0answers
36 views
Why always AIC and BIC are used in mixture model than Vuong test
I am working with mixture models. I fitted more than one model to the data and then try to select the most appropriate model using different selection criteria, for example, AIC. My supervisors asked ...
0
votes
0answers
52 views
Goodness of fit in logistic regression using pchisq
I am using a logistic regression from the glm function in R.
I get the following result but my main concern is the goodness of fit where I am trying to use "pchisq" function to check this.
I have ...
1
vote
0answers
21 views
Estimation and testing of unconditional distribution in time series models
In time series models, such as ARMA-GARCH, is it possible to estimate what the unconditional distribution is? Given that the time series is auto-correlated / persistent, how many observations would be ...
1
vote
1answer
64 views
A powerful test for any distribution
Simulations by Razali et al (2011) showed that the Shapiro-Wilk test of normality provided the most power (at a fixed significance level) when compared to Anderson-Darling, Kolmogorov-Smirnov, etc. My ...
1
vote
0answers
16 views
Measure of goodness of fit of ellipsoid model to data
Consider a measurement $m$ of a physical property, the value of which depends on the direction investigated. If the property derives from the application of a second-rank symmetric tensor, the ...
2
votes
0answers
56 views
How to compare model fit of OLS and Poisson regression?
I have built two regression models to predict sales of different products based on a number of explanatory variables, with an offset term for the number of days each product was on sale. One is a ...
0
votes
0answers
13 views
model calibration in complex data
I am working with a complex dataset (national inpatient sample) which has weight, cluster and stratum variables. My aim is to look for predictors of pediatric post-operative respiratory failure. I ...
0
votes
0answers
28 views
Unbiased estimator for $L^2$ probability distance norm
I am trying to find an unbiased estimator for (what looks like) the $L^2$ Wasserstein distance between two probability measures. I'm pretty sure that by bickel-lehmann, there is an unbiased estimator. ...
6
votes
2answers
143 views
Significance vs. goodness-of-fit in regression
Assume that I am interested in analyzing the following linear regression model:
$$
Y = \beta_0 +\beta_1 x_1 +\beta_2 x_2+e
$$
Please explain the difference between testing the p-value for each ...
0
votes
0answers
24 views
Poisson model fitting data
I am having trouble understanding why the below would not be a good fit for the Poisson model. The first graph shows the frequencies and the second graph shows the expected value in comparison to the ...
2
votes
0answers
29 views
What is the difference between Shapiro-Francia and Shapiro-Wilk tests of normality?
I apologize for this easy/basic question. Please forgive me if this is a duplicate question.
I've been researching the Shapiro-Wilk test (1965) of normality for a project. In doing so, I found an ...
1
vote
0answers
70 views
Is the covariance matrix a diagonal matrix with variances on the diagonals?
I am a geophysicist learning about geophysical inverse problems. In many papers, the authors discuss the "covariance matrix" as it applies to the inverse problem.
In most geophysical applications, ...
0
votes
0answers
28 views
Goodness of fit test for my Data
We have one Original Set and one set of the same data after smoothing.
The smoothed data set has been generated through natural cubic spline regression over the original data, which is a non-...
2
votes
0answers
28 views
Finding a distribution that fits different observations of data
I am observing session lengths on a network. I want to fit a distribution to the data that I have collected. I have data from two different observations (about a month apart). The plot below shows ...
1
vote
0answers
18 views
Testing a Multiple Logistic regression Model Goodness of Fit and accuracy
I am trying to create a predictive model that will determine the likelihood that a lead will take a specific action. I used a Logistic regression because either the Lead does (1) or does not (0) take ...
0
votes
1answer
180 views
Fit Hawkes process to 1d data using python package TICK
How can one fit the 1-dimensional Hawkes process with exponential kernel to the experimental 1d dataset (t1,t2,t3...tn) and check the goodness-of-fit
via tick python3 package?
I found on official ...
1
vote
0answers
32 views
Evaluate Bayesian SEM goodness of fit blavaan
I'm currently trying to identify how to evaluate model fit with Bayesian SEM. I've been reading that the posterior predictive p-value can be used, with a p-value of approximately 0.5 indicating good ...
1
vote
0answers
29 views
Inference using model with low goodness of fit.
Assuming a model is correctly specified, would it be appropriate to draw inferences based on a model with a low Goodness of Fit (~0.15)?
Of course, using such a model to make predictions is likely ...
4
votes
4answers
889 views
Linear regression with negative estimated value for intercept
Does a negative value of intercept suggest that the regression line provides poor fit to the data? why? and why not?
3
votes
1answer
55 views
Goodness of fit test
What is the intuition behind pooling frequencies that are less than five in a Chi-squared goodness of fit test?
Even though I have been told that it is not used in industries today, the concept is ...
