Different non-parametric methods for estimating the probability distribution of data I have some data and was trying to fit a smooth curve to it. However, I do not want to enforce too many prior beliefs or too strong pre-conceptions (except the ones implied by the rest of my question) on it, or any specific distributions. 
I just wanted to fit it with some smooth curve (or have a good estimate of the probability distribution that it might have come from). The only method that I know for doing this is kernel density estimation (KDE). I was wondering, if people knew of other methods for estimating such a thing. I just wanted a list of them and from that I can do my own research to find out which ones I want to use.
Giving any links or good references (or intuitions on which ones are good) are always welcome (and are encouraged)!
 A: You don't specify that you're talking about continuous random variables, but I'll assume, since you mention KDE, that you intend this.
Two other methods for fitting smooth densities:
1) log-spline density estimation. Here a spline curve is fitted to the log-density.
An example paper:
Kooperberg and Stone (1991),
"A study of logspline density estimation,"
Computational Statistics & Data Analysis, 12, 327-347
Kooperberg provides a link to a pdf of his paper here, under "1991".
If you use R, there's a package for this. An example of a fit generated by it is here. Below is a histogram of the logs of the data set there, and reproductions of the logspline and kernel density estimates from the answer:

$$  $$
Logspline density estimate:

Kernel density estimate:

2) Finite mixture models. Here some convenient family of distributions is chosen (in many cases, the normal), and the density is assumed to be a mixture of several different members of that family. Note that kernel density estimates can be seen as such a mixture (with a Gaussian kernel, they're a mixture of Gaussians). 
More generally these might be fitted via ML, or the EM algorithm, or in some cases via moment matching, though in particular circumstances other approaches may be feasible.
(There are a plethora of R packages that do various forms of mixture modelling.)
Added in edit:
3) Averaged shifted histograms
(which are not literally smooth, but perhaps smooth enough for your unstated criteria):
Imagine computing a sequence of histograms at some fixed binwidth ($b$), across a bin-origin that shifts by $b/k$ for some integer $k$ each time, and then averaged. 
This looks at first glance like a histogram done at binwidth $b/k$, but is much smoother.
E.g. compute 4 histograms each at binwidth 1, but offset by +0,+0.25,+0.5,+0.75 and then
average the heights at any given $x$. You end up with something like so:

Diagram taken from this answer. As I say there, if you go to that level of effort, you might as well do kernel density estimation.
A: Subject to the comments above about assumptions such as smoothness etc. You can do Bayesian nonparametric density estimation using mixture models with the Dirichlet process prior. 
The picture below shows the probability density contours recovered from MCMC estimation of a bivariate normal DP-mixture model for the 'old faithful' data. The points are coloured IIRC according to the clustering obtained on the last MCMC step.

Teh 2010 provides some good background.
A: A popular choice are random forest (see concretely chapter five of "Decision Forests: A Unified Framework for Classification, Regression, Density Estimation, Manifold Learning and Semi-Supervised Learning".
It describes in detail the algorithm and evaluates it against other popular choices like k-means, GMM and KDE. Random Forest are implemented in R and scikit-learn.
Random Forest are bagged decision trees in a clever way.
