Random values in fixed interval - how to assign probability distribution? Please excuse my lack of terminology. I am just a humble discrete optimizer
Assume we have a kind of "coin toss" where the result is not binary, but a rational number between -1 and 1. There is nothing known a priori about the probability distribution underlying this "coin". 
What can I do to estimate the probability distribution through experiments?
I read a bit about Bayesian and Maximum Entropy approaches, but I need somebody to guide me in the right direction. 
 A: If I understand what you mean, you don't have an a priori about the family  (normal, Beta...) the distribution belongs to. This is called a non parametric approach.
A simple method to estimate a distribution is kernel density estimator.
You can also use Maximum Entropy but then you implicitly assume a certain family. You have to:


*

*define a few properties of the distribution you estimate empirically (like expectation, expectation of the log, expectation of the square, variance...)

*Take the distribution matching these properties that has maximum entropy.


With this method, defining the properties you want to estimate is equivalent to choosing a distribution family. For example, if you choose mean and variance (assume your values range over $]-\infty;\infty[$), then the distribution will be a normal distribution.
Kernel density estimator is maybe closer to the idea of having no a priori. 
A: 
There is nothing known a priori about the probability distribution underlying this "coin". 

Do you have no information at all about the source that produces this information? If you know something about the nature of the process, you could choose a type of distribution and then fit the model by finding the maximum likelihood estimation for the parameters. 
If there is really no a priori information available about the type of distribution, the best estimate that you can get is an empirical histogram or kernel density estimate. Both the histogram and the kernel density estimate have parameters that determine how the estimate will look like. For the histogram this is the bin width, and for the kernel density estimate this is bandwidth parameter. Elements of Density Estimation by the Utah university provides a good overview of those methods. 
If the bandwidth or bin width is chosen too low, the estimate will be too coarse. If the bandwidth is too narrow, you might overfit the model if you have too little data, as can be seen in this picture from the KDE wikipedia page. In general, the more data is available, the narrower you can set the bandwith parameter. Chapter 3 of the Utah paper talks about optimal bandwidth selection.
How good your estimate will be all depends on the number of samples that you can get from your source, and from the distribution itself. For example, when estimating a histogram bin with an estimated probability of one in a hundred, you need well over a hundred samples to be certain that the probability is actually one in a hundred. A simple test that you could do is to divide your data up in to multiple sets, and then compute the histogram or KDE for each set. If the variance of the different estimates is within a reasonable boundary, you can use this of a proof that is a good estimate. Once again, the Utah paper provides some nice background about this problem. 
