0
$\begingroup$

The oracle property is an asymptotic property of an estimator, and is about variable selection:

An estimator $\hat \beta_n$ satisfies the oracle property if in the limit of $n\to \infty$, the probability that $\hat beta_i=0$ when $\beta_i=0$ goes to $1$, and the limiting distribution of the other $\hat \beta_j$ is the same as if the variables for which the parameters $\beta_i$ are zero-valued had been not included in the model.

Intuitively, the oracle property says that the estimator behaves in the limit as if an oracle had told us which of the variables have zero or non-zero parameters.


Oracle inequalities, insofar as I understand, are presented to be finite-sample versions of the oracle properties. However, they don't seem to me to be about variable selection at all:

An estimator $\hat \beta$ may satisfy an "oracle inequality" of the form: $||\hat \beta_n-\beta||\leq c_n$.

It seems to me that this no longer has anything to do with variable selection (or "an oracle telling us which variables have non-zero coefficients"). It's simply a finite-sample bound on the error, which may be because the estimator is good at identifying the non-zero coefficents, or for some other reason.

Am I wrong? Do oracle inequalities have something to do with variable selection? should they really be seen as finite-sample versions of oracle properties?

$\endgroup$
1
$\begingroup$

According to the paper "Modern statistical estimation via oracle inequalities":

‘Oracle inequalities’ are a powerful decision-theoretic tool which has served to understand the optimality of thresholding rules, but which has many other potential applications, some of which we will discuss.

and in page 22

An oracle inequality relates the performance of a real estimator with that of an ideal estimator which relies on perfect information supplied by an oracle, and which is not available in practice.

The connection to model selection can be found at "Model Selection, Penalization and Oracle Inequalities"

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.