Process vs Distribution and Generative Algorithms [closed]

Question

While trying to understand why normal distributions come up all the time, i found something like: means of sampling distributions always are normally distributed when N--->infinity.

But then I also found that there is some process that generated a distribution. Can we talk of distributions without talking about generating processes?

Also, generative algorithms like Naive Bayes, why are they called generative? Do they capture the mechanics of process generating the data i.e are they the model of the generating process itself?

Answer 1

There are two parts to your question.

1. Means of sampling distribution Vs. underlying process: Say you generate a sample of size 100 using Poisson distribution. Suppose you repeat this process k times (k is large). So you have ended up with k buckets each of size 100. If you take mean within each bucket and then look at distribution of the means (there'll be k such data points) it will look like normal. This is the reason why we can talk about distributions without talking about generating process. The normality claim is on mean of sampling distribution and not about the nature of underlying process.

2. Your understanding of generative algorithms is correct. We are trying to model the process which generated the data. But note that we make assumptions about structure of the process. For example, Gaussian mixture model will start with 'If I assume the data to be generated by Gaussians, what conclusions can I draw about their means' and so on.

Hope this helps.