1
$\begingroup$

In general, to use the method of least squares, a linear stochastic system is modeled as: \begin{equation} y = ax + \eta \end{equation}

where, $y$, is an observed variable, $x$ is an input while $\eta$ is error in observation. Then the parameter $a$ is estimated in the least squares sense by minimizing the error $\left(y - ax\right)^2$.

Assume that instead of modeling the system in this way, I do some change of variables and write the equation of the system as:

\begin{equation} \alpha\beta = y + \eta \end{equation}

where, $\alpha$ and $\beta$ need to be estimated. Let us also assume that I know the range of values that $\beta$ can take. Then I rewrite the estimation problem as:

\begin{equation} \alpha = \frac{y}{\beta} + \nu \end{equation}

Now suppose that I have $n$ observations i.e. $y_1, y_2, \cdots, y_n$ and I know that $\beta$ can take $m$ possible values i.e. $\beta_1, \beta_2, \cdot, \beta_m$. For each value of $\beta$, I compute the $n$ values of $\alpha$ and find variance of these $n$ values. Since $\alpha$ is constant, ideally the variance should be zero. But due to error $\nu$, the variance will be non-zero. So, the value of $\beta$ that minimizes the variance in $n$ values of $\alpha$ is the optimal estimate for $\beta$. $\alpha$ can then be found out.

Is this a correct approach to remodel the least squares problem. Potential questions can be:

  1. Why not use least squares? ( Ans: Because in some multivariate cases, $x$ can be badly conditioned.)
  2. If $y$ and $\beta$ are normally distributed then $\frac{y}{\beta}$ has Cauchy distribution and Cauchy distribution has no finite variance. So, minimizing the variance of $\frac{y}{\beta}$ does not make sense.

So, I want to know whether the modeling mentioned above makes sense.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.