# Finding optimal beta when there are multiple different errors

I am working on an econometrics model that I'm not sure how to approach. I've made a utility function where the weights have noise as well. In short it's:

$$y_i = (\beta + \epsilon_i)x_i + u_i$$

How would one estimate beta hat?

So far, I've approached it by estimating two beta hats from: $$\min_\beta \epsilon_i^2 = \min_\beta \sum (y_i - \beta x_i - \epsilon_i x_i)^2 \implies \hat{\beta}_\epsilon = \frac{\sum x_i y_i -\epsilon_i x_i^2}{\sum x_i^2}$$ $$\min_\beta u_i^2 = \min_\beta \sum (\frac {y_i - \beta x_i -u_i}{x_i})^2 \implies \hat{\beta}_u = \frac{1}{n}\sum \frac {y_i - u_i}{x_i}$$ At the end of the day, $\hat{\beta}$ can be whatever you want it to albeit there are better $\hat{\beta}$'s. Is there a unique optimal $\hat{\beta}$? Is it useful to have $\hat{\beta}$'s which have a probability distribution?

EDIT: I'm working on schelling's segregation model but I've added a wealth and distance from work as additional elements to the utility function. Currently, I am using the following utility, $$y_i = \beta_1 x_{1,i} + \beta_2 x_{2,i} - \beta_3 x_{3,i} + u_i$$ Where each $x_i$ characterizes matching race, wealth or distance from an employment and each $\beta_i$ is the subsequent weighting for each parameter. I wanted to add some more heterogeneity to each agent such that each agent had an individual preference set. The intuition is that some agents prefer living with the same race, others prefer having a shorter commute, etc. That's where I thought changing the model to the following would have an interesting effect: $$y_i = (\beta_1 + \epsilon_{1,i})x_{1,i} + (\beta_2 + \epsilon_{2,i})x_{2,i} - (\beta_3 + \epsilon_{3,i})x_{3,i} + u_i$$ For now, I assume that all the errors are i.i.d. and $\mathcal{N}(0,1)$. I am simulating this in R.

• You need to edit your answer, telling us the properties of $\epsilon_i$. Are they observed (i.e. part of the data)? If unobserved, are they uncorrelated with $x_i$? Are they uncorrelated with $u_i$? Are they mean zero? For instance, if $\epsilon_i$ is an unobserved error term, then the beta hats that you computed can't be right, since they involve $\epsilon_i$ and thus can't be computed from data. Dec 17, 2015 at 3:25
• Actually, it isn't really a typical error in variables (aka measurement error) model, which would be of the form $y_i = \beta x^*_i + u_i$, where the observed variable is the true $x_i$ with measurement error added: $x^*_i = x_i + \epsilon_i$. Here $\epsilon_i$ is independent of $x_i$ and also independent of $u_i$. The VARIABLE $x_i+\epsilon_i$ is observed with error, thus, errors in variables. Dec 17, 2015 at 3:29
• @AlaskaRon is right. On second look this isn't an error in variables. (I glanced at this a bit fast and my brain saw $y_i = \beta(x_i+\epsilon_i) + u_i$ instead of what you have $y_i = \beta x_i + \epsilon_i x_i + u_i$.) If you did regular OLS $\epsilon_i x_i + u_i$ would be the error term and you'd have an inconsistent estimate due to endogeneity. It would be helpful to have more info on the problem to think about reasonable approaches... Dec 17, 2015 at 7:32
• I think this should be a multilevel model, what do guys think? Dec 19, 2015 at 12:24
• It sure sounds like a random effects model of some kind, and I think a multilevel model would probably be suitable Jan 8, 2016 at 3:54

I will assume that your error terms $u_i$ and $\epsilon_i$ are independent homoscedastic normal random variables (possibly with different variances from each other).$^\dagger$ With this assumption, your model equation can be simplified to a heteroscedastic linear regression model as follows:

$$Y_i = \beta x_i + \varepsilon_i \quad \quad \varepsilon_i = u_i + x_i \epsilon_i \sim \text{IID N}(0, \sigma_u^2 + x_i^2 \sigma_\epsilon^2).$$

Maximum-likelihood estimation of the coefficient $\beta$ would be done using weighted least-squares (WLS) estimation with inverse-weights $1 / w_i = \mathbb{V}(\varepsilon) = \sigma_u^2 + x_i^2 \sigma_\epsilon^2$. Conditional on the estimates of the variance terms, this gives the estimator:

$$\hat{\beta} = \frac{\sum \hat{w}_i x_i y_i}{\sum \hat{w}_i x_i^2} \quad \quad \quad \hat{w}_i = \frac{1}{\hat{\sigma}_u^2 + x_i^2 \hat{\sigma}_\epsilon^2}.$$

You would also need to derive the parameter estimators $\hat{\sigma}_u$ and $\hat{\sigma}_\epsilon$, but this should not be overly difficult. (Note that the coefficient estimator is only affected by these terms through their ratio, so what matters is the relative sizes of these parameters.) Having done this, you would obtain all the usual benefits that accrue to estimation using WLS.

$^\dagger$ I note that you have stated in your question that you assume unit variance for the two error terms. This corresponds to my model with $\sigma_u = \sigma_\epsilon = 1$. This assumption is unlikely to be reasonable in most cases and it is better to estimate the variances of the errors from the data, so I will proceed under the more general model.

Have you thought about using GMM? If you assume iid epsilons, you'll have a bunch of orthogonality conditions that you can use for the estimation -- I don't know if it would help you with your underidentification problem, but it may be worth a try if you are still working on this model.