3
$\begingroup$

I recently read in this page (https://www.timlrx.com/2018/02/26/notes-on-regression-approximation-of-the-conditional-expectation-function/#fn1) that:

"Regression offers a way of approximating the CEF linearly i.e,...,Thus, even if the CEF is non-linear as in the recipe and star rating example, the regression line would provide the best linear approximation to it (drawn in green below)".

I understand that we use conditional expectation in OLS assuming that the conditional expectation of Y for a ceratin value of X is true when Y can be modelled as a linear function of X. Please correct if I am misundestanding something.

So, and this can sound a silly (sorry!), what we are REALLY DOING when performing linear regression, is just aproximating the conditional expectation function? I mean, what we are really trying to estimate in a regression is the line that best fit the conditional expectation function? Or there is something I am no getting right. Please do not hesitate in correcting me :)

Oh, and one last thing If so, what is the necessity to do linear regression? Why we can't just work with the conditional expectation function instead of linear regression since is the conditional expectation function what we are trying to approxitamate to?

$\endgroup$
4
$\begingroup$

what we are REALLY DOING when performing linear regression, is just aproximating the conditional expectation function? I mean, what we are really trying to estimate in a regression is the line that best fit the conditional expectation function?

Exactly. This discussion can help you: Regression's population parameters

what is the necessity to do linear regression? Why we can't just work with the conditional expectation function instead of linear regression since is the conditional expectation function what we are trying to approxitamate to?

Frequently is not clear in presentations but we have to note the regression and conditional expectation function are synonym. The linear regression have the great advantage that, in order to performing it, know the entire joint conditional distribution of involved r.vs. is not needed. Moreover it is easy to compute and generally well justifiable. For definiteness of OLS regression the finiteness of first order moments and absence of multicollinearity among regressors are the main assumptions.

Note that the usefulness and scope of regression is another, very long, story. Read here can help: Regression: Causation vs Prediction vs Description

$\endgroup$
4
  • $\begingroup$ Thanks for answering. Is the CEF function the same that the Population Regression Equation? $\endgroup$ Aug 27 '20 at 18:29
  • 1
    $\begingroup$ “Regression” is a word much more used in estimation that, let me say, in exact stat math theory. However CEF and regression can be intended as synonym. So, both can be intended as sample or population quantity. Terminologically the “regression equation” usually include the error also: $y = E[y|X] + \epsilon$. Frequently is used in population sense $\endgroup$
    – markowitz
    Aug 28 '20 at 9:49
  • $\begingroup$ What If our X is fixed, not random like econometrics books propose, is still CEF a synonym of regression? Because I understand that we can make regression both conditionaly and unconditionaly. I'm so sorry for making this silly questions man, but it is confussing that concept of conditional or unconditional when it comes to regression. $\endgroup$ Aug 28 '20 at 14:06
  • $\begingroup$ Concepts like joint probability distribution and CEF are properly defined only if also regressors (X) are random. However most of things you encounter in regression analysis take X are given. Then, consider X as constant or random, but given, is not so different. $\endgroup$
    – markowitz
    Aug 28 '20 at 14:58
2
$\begingroup$

Oh, and one last thing If so, what is the necessity to do linear regression? Why we can't just work with the conditional expectation function instead of linear regression since is the conditional expectation function what we are trying to approxitamate to?

Here are three (good) reasons why linear regression is widely used to approximate the CEF.

  1. Suppose $X$ has ten variables, and suppose we actually knew the true CEF, $E[Y|X=x]=f(x)$. How can we grasp the implications of this function? That's really hard in general. If the function were linear and additive in those 10 variables we will find this much easier. So we might want to summarise the function with its best linear and additive approximation for interpretation purposes even if we knew its true shape. Interpretation is really important in many applications because at least part of the credibility of an analysis rests on whether the modelled relationship is consistent with domain knowledge, in addition to statistical measures. So you need to understand the modelled relationship, and also have an idea what features in the data drive it. Incidentally, linear regression is a good basis from which to model departures from additivity and linearity: Interactions relax the former, variable transformations the latter. The modeller can easily incorporate these into the analysis. And more systematic approaches (eg regression splines) exist too.

  2. We never know the true CEF in empirical work. We have to estimate it from data. We know from the theory of non-parametric regression that it takes an extraordinary amount of data to estimate such a function with any precision - if we really do not put any restrictions on it. An intuitive way of seeing this is to suppose you have 10 binary variables. Suppose you need about 50 data points for a particular value of $x$ (10 dimensional) to get a reasonable estimate of the average value of $y$. There are 1024 unique values of $x$ for which you need those 50 data points if you want to estimate the CEF at that level of precision for all values of $x$

  3. Estimating a linear regression with OLS is computationally faster and more robust compared to just about any alternative. When you estimate a more complex model you often need to spend much more time on fitting it and on checking how numerically stable and robust the result is.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.