# Relationship between distribution and data generating process

My question is: are the concepts of probability distribution, data generating process and population equivalent? If not, then what is the relationship they have. My question arises from the following excerpt from Hansen's book on econometrics, though it doesn't go too deep into it:

In econometric theory we refer to the underlying common distribution F as the population. Some authors prefer the label the data-generating-process (DGP). You can think of it as a theoretical concept or an infinitely-large potential population.

The probability distribution is the actual mathematical function $$P({\bf x}; \theta)$$ that can assign a probability to each possible vector $${\bf x}$$. It is given by the parameter vector $$\theta$$.

The data generating process is the causal (deterministic or stochastic) mechanism from where the data originate. The population is the total number of data items at all available. $$\begin{split} \\ \\ \\ \end{split}$$

The probabilistic model

Define a data-generating process $${\cal P}$$ as follows

$$\begin{split} &{\cal P} \mapsto {\cal E} \\ &{\cal P} = f(\,{\cal S}\,; \; {\cal E}\,; \; \{{\cal C} \Rightarrow^* {\cal A}\}) \end{split}$$

with the set $${\cal S}$$ the complete state description, the set $${\cal E}$$ the possible events to occur and the set $$\{{\cal C} \Rightarrow^* {\cal A}\}$$, the (cause $$\rightarrow$$ action) relationships that may be evoked given $${\cal S}$$. The asterisk in $$\Rightarrow^*$$ indicates that an intrinsic stochastic-causal mechanism may be at play, just like in quantum mechanics. The data generating process maps to the (future) event space $${\cal E}$$.

Define a random variable $$X$$ as a function from the event space $${\cal E}$$ to the set of real numbers $$\Re$$ [Evans], $$\;X\,:\; {\cal E} \, \mapsto \, \Re$$ .

The distribution of $$X$$ is the collection of probabilities $$P(X \in {\cal B})$$ for all subsets $${\cal B}$$ of the real numbers. $${\cal B}$$ is a Borel subset [Evans].

Based on the distribution of $$X$$, a parametrized probability distribution is defined as $$P({\bf x}; {\bf \theta})$$. Now we talk about a statistical model. This model $$P$$ has the the parameter vector $${\bf \theta}$$.

In general $$P({\bf x}; {\bf \theta})$$ will specify probability outcomes of possible events $${\cal E}$$, and the inner working of $$P({\bf x}; {\bf \theta})$$ will always be an abstraction of the underlying data-generating process $${\cal P}$$.

Example

These three concepts are illustrated by examples below.

Probability distribution

For a binomially distributed value $$i$$, the probability distribution is

$$P(i ; p) = \binom{n}{i} \; p^i \, (1-p)^{(n-i)}$$

where $$i$$ is the number of '1's in a sample of $$n$$ draws, $$i \leq n$$ and $$\theta=p$$ is the probability of a '1' in each individual draw.

Data generating process

The mechanism that is responsible for generating the data, that can be deterministic or stochastic. Even at the smallest level in our world, stochastic mechanisms apply namely the in quantum mechanics. In a number of cases, the underlying mechanism is deterministic but way too complex to model. And so a stochastic model based on assumptions and abstraction is built. Think for example of a macro econometric model that can simulate the economic interactions between Miljons of citizens.

Population

The population can be all voters in an election in a complete country. The frequently performed polls take samples from this population to see what will be voted for at the coming elections.

Michael J. Evans, Jeffrey S. Rosenthal. Probabilities and Statistics - the Science of Uncertainty, W.H. Freeman and Company, New York, 2004.

• Do you mean “causal (deterministic or stochastic) mechanism” instead of “deterministic (causal or stochastic) mechanism”? Based on the example, I think you mean the former. – T.E.G. Aug 1 '20 at 11:56
• Your are correct, I altered the text now - thanks ! – Match Maker EE Aug 1 '20 at 12:01
• Also, this answer could be improved by showing how these concepts are related to each other. For now, it only provides definitions and examples. This would also clarify the point in the quote from Hansen’s book. – T.E.G. Aug 1 '20 at 12:02
• Moreover, these definitions appear unnecessarily limited: many authorities use these terms in broader or different ways than prescribed here. Thus, some citations to support your definitions would be helpful. – whuber Aug 1 '20 at 12:30
• My answer has been elaborated now. Thanks for your comments. – Match Maker EE Aug 1 '20 at 16:40