0
$\begingroup$

For a given ridge parameter, ridge estimates minimize the sum of squared predictions subject to an inequality constraint.

Are the ridge estimates biased regardless of whether the aforementioned inequality constraint holds (for the "true" parameters) or not?

Would you please direct me to some references on this?

Thank you very much in advance.

$\endgroup$
4
  • $\begingroup$ "Are the ridge estimates biased regardless of whether the aforementioned inequality constraint holds or not?" I don't understand what you're asking here since a solution, by definition, must satisfy the constraint. $\endgroup$ – user20160 Feb 12 at 11:45
  • $\begingroup$ @user20160 I know that the estimates must satisfy the constraints. I meant whether the constraint holds for the "real" parameters or not. $\endgroup$ – Ahmed Ali Feb 14 at 11:43
  • 1
    $\begingroup$ I see. Could you edit the question to include this information? $\endgroup$ – user20160 Feb 14 at 16:34
  • 1
    $\begingroup$ @user20160 Sure, thank you for your feedback. $\endgroup$ – Ahmed Ali Feb 14 at 18:26
1
$\begingroup$

I think the answer would be "yes" in this case, and the reason I would answer in the affirmative is because it is reminiscent of the Voltage Meter story.

In short (paraphrasing from Computer Aged Statistical Inference), an engineer measures the voltages on a batch of 12 tubes using a voltmeter that is normally calibrated

$$ x \sim \mathcal{N}(\mu,1) $$

Here, $x$ is a given measurement and $\mu$ the truth batch voltage. The measurements range from 82 to 99, with an average of 92. The engineer reports 92 as the estimate of $\mu$.

The next day, he discovers a glitch in the voltmeter such that any voltage over 100 registers as $x=100$. Though none of his measurements were effected, his estimate (the sample mean) would not be unbiased in future realizations of $\bar{X}$ from the actual probability model.

How does this apply to ridge? Assuming the minima of the loss function is within the constraint, the estimates $\beta = (X^TX)^{-1}X^Ty$ are still technically biased in the same way that the sample mean would be biased in the voltmeter example above.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.