Why is specifying the distribution of the data different from specifying the data itself?

Question

I am performing MCMC with pyMC on a nonlinear model, specified in Probabilistic modelling MCMC question with pyMC Imagine I have 2000 points of experimental data, normally distributed:

data = np.random.normal(.2, 1, 2000)

Imagine that, instead of having the raw data, the experimentalist gives me just one number, but assures me that it comes from 2000 measures, normally distributed with precision=sigma=1. I could model that with:

data = pm.Normal('data', .2, 1.)

I would expect these two data to bear the same information, however, when I perform MCMC with pyMC, the trace of the stochastic variable (phi) that depends on data is much lower in the first place.

In detail:

phi = pm.Uniform("phi", 0, 180., value=150)
tau = pm.Normal('tau', 5., .05)
# coupling is a deterministic function of phi.
obs = pm.Normal("obs", coupling, tau, value=data, observed=True)
model = pm.Model([obs, phi, tau])

Why is giving 2000 data points "better"? Isn't there a way I can give one experimental measure that is as informative as the 2000 measures? I am afraid there is some fundamental thing about pyMC that I do not grasp...

Answer 1

First, you didn't explicitly specify it's Python you're using. Not everyone's familiar with the language and its stat libraries here. For instance, it's not obvious that pm.Normal() is the probability distribution class, while np.random.normal() is random number generator unless you know Python.

Second, why would you think that specifying the distribution should be the same as giving the data from the distribution?

When you say "data" you really mean a sample from the distribution. So, the difference is that when I specify the distribution, i.e. give you pm.Normal, you get the population parameters. So, in this case you can query the class to learn that population mean is 0.2.

When I give you the data from the distribution - or statistically speaking a sample - you get the sample mean, not the population mean. So, if you call mean() on your data (sample), the number will be different from 0.2 (population mean). It'll be very close if the sample size is large, but it will not be the same.

This the data (sample) contains less information (more noise, uncertainty) about the population parameters.