# Similarities and differences between regression and estimation

What's the similarities and differences between parametric regression analysis and estimation theory?

I notice that they are both about parameter estimation, and both require some models for estimation.

One difference is that regress requires both independent and dependent variables, while estimation only requires observed variables. Also, regression minimizes the distance between the observed values and the values predicted by the model (least square), as the estimation, like MMSE estimator, minimizes the mean square error (MSE) of the to-be-estimated parameters.

For linear model with Gaussian noise, the maximum likelihood (ML) estimator will identical with the regression in form of (weighted) least square. In other words, the estimate achieves maximum likelihood, and also minimizes the residual.

Is there any other similarity or difference between these two?

-
This may be just me but I don't know what you mean by "estimation theory" - could you give a reference? As far as I'm concerned, estimation is a process of estimating values for parameters in a model. One such model might be a linear model with a Gaussian response; and the particular estimation technique commonly used is ordinarly least squares although other techniques are possible. You need an estimation theory to know which technique to use but I can't see how you can compare similarities and differences between estimation theory as such and regression models. –  Peter Ellis Jul 29 '12 at 5:40
@PeterEllis I don't know the exact definition of the estimation theory, but what I mean is something like the maximum likelihood (ML) estimation, maximum a posteriori (MAP) estimation, and minimum mean squared error (MMSE) estimation, etc. See here. I don't know if this kind of estimation overlap with regression, or one belongs to the other? –  chaohuang Jul 29 '12 at 15:43