Is it better to look at ECDFs than PDFs when exploring empirical sample distributions?

Question

I know that when plotting CDFs, ECDFs, or PDFs by using finite samples, we must be doing some form of interpolation.

As far as I guess, empirical cumulative density functions (ECDFs) perform linear or stair-case interpolation in most software that plot them. Which is a rough interpolation with sharp edges.

But my eyes (and the natural neural network in my head) can automatically interpolate smooth curves by looking at the rough ECDFs. Which is nice in my view.

Alternatively, I can represent that ECDF into a PDF. But this will require me to interpolate using a software that is different than my eyes (and the natural neural network in my head). Usually such software didn't have the chance to evolve for billions of years (which I had), and usually we end up with KDE where we need to pick a kernel and a bandwidth which affects how the smoothing of the interpolation. And I cannot use parametric density estimation methods because the goal here is to explore an empirically collected sample points. Assuming any known distribution will reduce my ability in identifying the true distribution.

The problem I face when I look at PDFs is that I always worry about how the smoothing is affecting the shape of the PDF to that extent that I end up asking my self: should I believe this PDF?

However, I don't face the PDFs problem above when I look at ECDFs as I know that what I look at is the result of very primitive interpolation (linear/stair-case) that cannot be too misguiding to my eyes.

So my questions are:

• Is the problem above really a problem? Or is it a side effect of simply me not knowing enough about statistical methods of visualizing the data?
• Is there anything in the tool-kit of statistics that can solve my PDF problem above?
• What are generally the preferred data exploring methods that are usually used as first-attempt methods?

Answer 1

I'd say, if you're interested in the CDF, then plot the CDF. If you're interested in the PDF, then plot the PDF (which you'll have to estimate). They're useful for different purposes. For example, the CDF makes it easy to identify quantiles of a distribution, which are not as obvious in the PDF. The PDF makes it easy to identify the modes and shape of a distribution, which is not as obvious in the CDF (that would require mentally differentiating your mentally smoothed PDF).

Histograms and kernel density estimators are the most popular ways to estimate a PDF nonparametrically, and they generally do a fine job. You're right not to blindly trust an estimate with arbitrary parameters. Some ways around this problem are: 1) If you're just eyeballing things, plot the PDF for multiple choices of the parameter (e.g. bin number, kernel bandwidth). Your mental assessment of the goodness of each estimate will be related to what level of roughness vs. structure looks right to you, based on prior knowledge/assumptions about the distribution. You can also plot confidence intervals to help you see whether you're undersmoothing. 2) Choose the parameter in a principled way (e.g. using cross validation or a formula based on asymptotic assumptions).

Looking at the empirical CDF won't give you the PDF for free. For example, try smoothing it (or fitting it with some nonparametric regression method) and then differentiating to obtain a PDF. This can be certainly done, but you'll find that the result is highly dependent on the smoothing/fitting parameters, which puts you in the exact same situation as having to choose the number of histogram bins or kernel bandwidth. You could say that, when you smooth/fit the empirical CDF, you can eyeball how good the match is. But, you're still left with the problem of deciding whether small wiggles are real or not, just like with a histogram or KDE.

Answer 2

(I'll try to avoid rehashing issues that are already well covered in the other answer; in particular I strongly agree with its first paragraph and much of the remainder, though I will expand slightly on one of its points along the way.)

If you're comfortable with some smoothing by eye, make your density estimates (like histograms or KDEs or whatever else) rougher than the supposed "optima" offered by many packages. They are generally aimed at giving minimum MSE estimates of the density under some set of assumptions, and while they work well for what they're designed for, they're not optimized for what they're actually used for.

For visual impression purposes -- where as you note, you already smooth by eye -- I think they oversmooth, often quite heavily.

I'll often use the default and half of the default as binwidth/bandwidth - and sometimes halve it again - to give several views of the data at different settings (the choice of kernel is generally less critical).