1
$\begingroup$

I have just started learning regression analysis and have doubts on its basic assumptions. Specifically, I do not have a clear idea over the use of the random error component $\epsilon$ in the regression model.

Assumption $1$ : The simple linear regression model is given as $$Y_i=\beta_0+\beta_1.X_i+\epsilon_i$$, where $Y$ is response variable, $X$ is regressor (deterministic), $\beta_0$ is the intercept, $\beta_1$ is slope and $\epsilon_i$ is the random error component.

Assumption $2$ : $\epsilon_i \sim \mathcal {N(0,\sigma^2)}$ and $\epsilon_i$ and $\epsilon_j$ are independent of each other.

Now let me elaborate my problem.

Suppose I am a restaurant owner and I have a huge record of numbers of meals ordered with bill amount and the tip amount for each meal ordered. I want to predict what the next tip amount should be, for any given bill amount. But there I'm stuck.

\begin{array}{| c | c | c |} \hline \\\text{Number of Meals Ordered} & \text{Bill Amount per Meal (in Dollar)} & \text{Tip Amount (in Dollar)} \\ &(X) & (Y)\\\hline 1& x_1& y_1 \\\\\hline 2& x_2& y_2\\\\\hline 3& x_3& y_3\\\\\hline 4& x_4& y_4\\\\\hline .& .& .\\. & .& .\\. & .& .\\ .& .& .\\\hline 100000 & x_{100000} & y_{100000}\\\\\hline \end{array}

If we treat this data-set as population and if we take any sample of size $100$ or of size $1000$, then we can find out $$Y_i=\widehat{\beta_0}+\widehat{\beta_1}.X_i+e_i\\\Longrightarrow Y_i=\widehat{\beta_0}+\widehat{\beta_1}.X_i+Y_i - \widehat{Y_i}\\\Longrightarrow \widehat{Y_i}=\widehat{\beta_0}+\widehat{\beta_1}.X_i$$ and later we go for model adequacy checking etc.

Regression1+ - Here in the diagram I just tried to understand, how does the concept of $\epsilon_i$ arise into the general model.

So, from Assumption $1$ - I have the questions

Nowhere in the analysis we are using or estimating $\epsilon_i$, then how does the concept of random error component arise in the model? If $\epsilon_i$ is unknown, then how can $e_i$ estimate $\epsilon_i$?

From Assumption $2$, we have $\epsilon_i$ and $\epsilon_j$ are independent of each other and the amount of error in $\epsilon_i$ and $\epsilon_j$ are different, therefore their variance should be different. Then $\epsilon_i$ should have heteroscadastic variance rather than homoscadastic variance.

Any help is highly appreciated.

Improve this question
$\endgroup$
11
  • $\begingroup$ Cannot resume the diagram here. Any help please? $\endgroup$
    – vbm
    Jun 26, 2018 at 15:39
  • 1
    $\begingroup$ Regarding your last point, variables can be independent but still be drawn from the same distribution with the same governing parameters (e.g., variance). So, $\epsilon_i$ and $\epsilon_j$ can be independent but come from the same distribution (i.e., have the same variance). $\endgroup$
    – Noah
    Jun 26, 2018 at 16:18
  • 1
    $\begingroup$ Assuming that the variance is constant for your model is not reasonable. Large bills will have large tips. Why would large tips have the same variance as small tips? It doesn't make a sense to me $\endgroup$
    – Aksakal
    Jun 26, 2018 at 16:21
  • 1
    $\begingroup$ By amount of error do you mean the value of $\epsilon_i$ for each unit? If you draw a sample from any single distribution, the value of the variables will differ. They can still be drawn from the same distribution with the same variance. $\endgroup$
    – Noah
    Jun 26, 2018 at 16:22
  • 1
    $\begingroup$ You need to pick the model that is reasonable for your problem. For instance, you could model the percentage ratio of the tip to the bill. For the ratio you could reasonably argue that the variance can be constant. You have to test the assumption, but at least it will have a chance to hold. The absolute amount of the tip has no chance to have the constant variance of error. $\endgroup$
    – Aksakal
    Jun 26, 2018 at 16:27

1 Answer 1

Reset to default
1
$\begingroup$

You do not observe the random errors $\varepsilon$, neither do you observe the coefficients $\beta$. Yet, you have no problem estimating coefficients, but have an issue estimating errors. The purpose of modeling is often estimation of unobserved parameters or quantities.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.