Goodness of fit for exponential distribution and large sample

Question

I'm new to statistics and statistics are not my area of research so maybe my question is simple and answer is on the surface. In my research the empirical CDF for the data looks like exponential distribution CDF. Plus the values of the mean and the standard deviation are close. The data is time periods in days, so it would make sense that it is exponentially distributed. I've tried to perform goodness of fit tests such as Chi-square test and KS test, but the problem is that the sample is large (2000+) and those tests have rejected the distribution type hypothesis. I've read that they seem to work "too" well on large samples and reject even the smallest deviation. They seem to work correctly on most smaller random samples of the main sample (200+). What would be the best scientifically acceptable approach in this situation to:

1. Proof that the exponential distribution can be used for the data;
2. Find the most fitting distribution parameter (lambda) with 95% probability.

EDIT: I need to clarify what my research is, I should've stated that in the first place. Without going into details of my research, the values in the dataset, let's call them T, represent harm to a person's health (measured in days spent on sick leave due to influence of production factors). My goal is to find a way to measure probability of harm to health based on T (the bigger the T is, the higher the probability). In other words, I need to find a function of T and the result of that function would be the probability. The idea is to use the CDF of the data as that function. On the graph below blue line is the ECDF, red is exponential CDF.

Answer 1

I've read that they seem to work "too" well on large samples and reject even the smallest deviation.

This sort of (misplaced) sentiment is common, but it arises from people misusing the tests. They are misusing them at all sample sizes, it's just more obvious at large ones. It's particularly a problem with goodness of fit tests, but it's also common with many other kinds of tests.

Hypothesis tests are (almost always) deliberately designed to be consistent, that is, to reject all false nulls as $n$ goes to infinity; indeed it's what people demand of their tests.

The problem is that people choose to state an exact equality null (and use a test for that exact equality null) when that's not actually what they want to test for; otherwise they would have no complaint when the test quite reasonably rejects a small deviation from the model in a large sample. It's like complaining that a hammer is just a bad tool when you've been using it to bang screws in. The problem in that scenario is not with the hammer; it's often a useful tool when used for the task it has been designed to carry out.

Your actual problem is not "are these two distributions* exactly equal?" so why use a test that will try very hard to tell you whenever they aren't?

It seems like you want to know whether an exponential model would be adequate for some purpose ... and in that case you would need to consider the situation in relation to its impact on the properties of that purpose. Each such impact is more like a question of effect-size, not significance.

You have not stated the purpose that I can see so there's little more to be done, except to talk in generalities.

Proof that the exponential distribution can be used for the data

That depends on what you're using the data to do, on what properties of that thing you care about, and how sensitive you are to deviations in those property (e.g. if you were computing a CI, you might want to investigate the impact on coverage -- e.g. how much would a 95% interval only giving 92% coverage matter? Sometimes the deviation may be a minor issue, at other times you might care a lot. If you're making a forecast, you might wonder what might the effect on the MSE of a prediction be? .... or many other things beside those)

This is not a matter of proof, though, just investigation. It might involve bootstrapping the sample, or sampling a "fuzzed" version of your sample, or looking at simulation from models that generalize your simple model in ways that more nearly encompass your data, or a number of other possibilities.

Find the most fitting distribution parameter (lambda) with 95% probability.

We can optimize parameter estimates according to any number of criteria (find the 'best' for a long list of potential choices of what you regard as 'better' which criteria you have not stated), but the "with 95% probability" part doesn't seem to make sense; I'm not even sure what you might have been trying to express there, if anything specific; were you looking for an interval estimate as well as a point estimate?

What would be the best scientifically acceptable approach in this situation

I'm unclear what counts as scientifically acceptable. What are the criteria for this? Who is judging them?

---

* the one you have a sample from and the one you're using as a model.

Answer 2

You're basically asking "does the deviations from the fitted exponential distribution matter?" and the obvious follow-up question to that is "...matter for what?"

I can't tell you whether the deviations matter because I don't know how you will use the result!

For example, if the deviations lie in the tail of the distribution and your application is primarily influenced by the head, then no, the deviations don't matter. If your application is heavily dependent on the tail, then yeah, deviations are a problem.

I would try using both the fitted exponential distribution and your real data in the same way, and see if the difference in final outcome matters clinically for your application.

In the same way, you find the appropriate lambda by trying many different ones and seeing which one generates the smallest error in the real application compared to the real data set.

---

Personally, I'm biased toward using the observed, real data set as it is, rather than first fitting a theoretical distribution to it. Using the real data won't let you derive nice, clean theoretical results, but for the most part, you can easily simulate the same thing instead. And by drawing from the real data set, the results are definitely properly influenced by whatever hidden quirks you never thought to model with the theoretical distributions. Sometimes those quirks are where the money is.