The DGP is the processes that cause data to occur as they do. They are a Platonic ideal that we do not and cannot know. Only in simulations can we define a mathematical DGP, in the real world they are hidden from us.
The aim of mathematical modelling is abstraction of the data. This means taking what we know and observe and trying to find a more generalised description of underlying reality that would allow us to make useful predictions in new situations. There is saying widely used on this site that all models are wrong but some are useful, this is the cause of that saying.
'the DGP is given as $y=a+bx+e$ where the error term fulfills all the OLS assumptions.' Is a cop out because the $e$ term encapsulates a wide array of lower order contributors to the data generation. Whatever has produced the data has a precise form, not a hand wavy error term. What we call error is just variation that we can't explain
a) Given knowledge of the value x takes one would describe their belief about the value y takes with the probability distribution on the right hand side.
That is the aim that we try to evaluate or beliefs about y based on x. However the example in isolation is misleading wrt the data generating process, what is quoted is a regression model not a DGP. I'd prefer to write something like $y=a+bx+cU_1+...zU_i$ where the $e$ term is split into a series of unknown underlying factors $U$ from 1 up to an indeterminate $i$. We then try to learn about $y$ by hypothesiseing $y=a+bx+e$ and projecting that model onto the data. We find that x is not quite enough to fit the data and after more poking around we realise that a previously unidentified factor is related, so we can replace $U_1$ with $z$ and collect new data to test the new hypothesis. If it fits better then we update our beliefs about the DGP. We keep going until we run out of ideas, it is no longer economically possible to collect data accurate enough to eliminate more $U_i$ terms, the model performs well enough for our needs or for a whole host of pragmatic reasons. We never stop because we have tried every possible $U_i$ term.
b)something that allows a causal interpretation?
This is getting deeper into extremely philosophical territory. Science is based on the premise that DGPs underpin reality and through careful thought and experimentation we can uncover that underlying reality. We use statistics to compare the outcome of the DGP with our hypothesis of what the DPG is and we look for a small $e$ to give us faith that we have captured a significant portion of the DGP. However because we never truly know the DGP we try to quantify the risk we are taking.
Let us suppose the model we estimate is $y=a+bx+e$ but that the DGP is $y=a+bx+cz+e$ this will yield biased estimates if $x$ and z are correlated". I don't get what this is supposed to mean if the regression equation describes the mean of y conditional on x. The "underspecified" model will yield a higher(or lower) coefficient to take into account the correlation, it will however still correctly describe the expectation of Y conditional on x. Here it seems to me that they are interpreting the regression coefficients as meaning the expected change in y if the regressor is changed by one unit(in a specific instance)
If the model has not been exposed to variation in $U_i$, in this case crystallised as $z$, it cannot account for the correlation. Part of the relationship between $x$ and $y$ is dependent on an unknown third factor which influences the nature of the relationship between $x$ and $y$. If the unseen $z$ changes it has an unpredictable effect on the x-y relationship because it has not been captured. If you are familiar with PCA or PLS or similar methods you will understand how subtle and complex correlations are. A correlation matrix is a high level summary that hides a lot of detail. PCA can unpack a single correlation matrix into several distinct underlying causes of correlated behaviour. Each PC describes a unique set of correlated behaviour. Furthermore each PC is uncorrelated with the others so knowing about one set of correlated behaviour gives you zero information on the others. You have to explicitly look at each possible correlation to account for it.
however still correctly describe the expectation of Y conditional on x
This will hold true while the underlying correlation structure applies, but if you haven't investigated the interaction of $x$ and $z$ then you don't know when it breaks down or changes. This issue is what underlies the need for verification of models in any new population or situation.
A real world example of z may be unmeasured dietary factors affecting an analytic target (x) correlated to disease mortality (y). Over years dietary habits of populations change, which can change the metabolism of the analyte or the underlying physiology the analyte acts on and from there affects mortality in a different way.
@Carl provides some examples of commonly used scenarios for explaining DGP where we use very simple statistical models of probability to allow us to predict long run behaviour. However all these probability models have physics mechanisms underpinning them. Consider rolling dice, what factors may include that? I'll list a few I can think of :
Symmetry of the dice
Starting orientation
Direction of throw
Force of throw
Local topography (shape of surface its thrown towards)
Spin
Coefficient of friction between dice and surface
Roundness of edges and corners
Air movement
Temperature
The theory behind DGP is that if you could identify and accurately measure enough factors then you could predict the outcome of a single throw to within your desired precision. So let's say we build a model for dice rolling in a Las Vegas casino and we win so much we get blacklisted in every major casino (we forgot to lose enough). Now take that model and apply it to a poorly maintained drafty gambling den, will it still apply accurately enough to win more than we lose? We won't know until we test it.
