I wish to test some of my ideas that I think are better than anything that I have seen. I could be wrong but I'd like to test my ideas and vanquish my doubts by more certain observations.
What I have been thinking to do is the following:
- Analytically define a set of distributions. Some of these are easy ones like Gaussian, uniform, or Tophat. But some of these must be difficult and challenging such as the Simpsons distribution.
- Implement software based on those analytical distributions, and use them to generate some samples.
- Because the distributions are analytically defined, I already -by definition- know their true PDFs. This is great.
- Then I'll test the following PDF estimation methods against the samples above:
- Existing PDF estimation methods (like KDE with various kernels and bandwidths).
- My own idea that I think is worth trying.
- Then I will measure the error of the estimations against the true PDFs.
- Then I will better know which of the PDF estimation methods is good.
My questions are:
- Q1: Are there any improvements over my plan above?
- Q2: I find it difficult for me to analytically define many true PDFs. Is there already a comprehensive list of many analytically defined true PDFs with varying difficulties (including very difficult ones) that I can re-use here?