Hot answers tagged

21 votes

Lasso vs. adaptive Lasso

Brief answers to your questions: Lasso and adaptive lasso are different. (Check Zou (2006) to see how adaptive lasso differs from standard lasso.) Lasso is a special case of elastic net. (See Zou &...
Richard Hardy's user avatar
18 votes

Lasso vs. adaptive Lasso

LASSO solutions are solutions that minimize $$Q(\beta|X,y) = \dfrac{1}{2n}||y-X\beta||^2 + \lambda\sum_{j}|\beta_j|$$ the adaptive lasso simply adds weights to this to try to counteract the known ...
bdeonovic's user avatar
  • 9,877
14 votes

What is the oracle property of an estimator?

An oracle knows the truth: it knows the true subset and is willing to act on it. The oracle property is that the asymptotic distribution of the estimator is the same as the asymptotic distribution of ...
user795305's user avatar
  • 2,872
12 votes

Oracle Inequality : In basic terms

I'll try to explain it in linear case. Consider the linear model $$Y_i=\sum_{j=1}^{p} \beta_jX_{i}^{(j)}+\epsilon_i, i=1,...,n. $$ When $p \leq n$ (number of independent variables less or equal then ...
D.G's user avatar
  • 116
11 votes

Lasso vs. adaptive Lasso

Adaptive LASSO is used for consistent variable selection. The problems we encounter when using the LASSO for variable selection are: The shrinkage parameter must be larger for selection than ...
StatGrrl's user avatar
  • 660
9 votes

What is the oracle property of an estimator?

The definition of Oracle property is related highly to the context. The very short but precise answer in linear regression (precisely high dimensional one) is this: an oracle estimator must be ...
TPArrow's user avatar
  • 2,295
2 votes

Parameter estimation and model selection consistencies

With respect to your first question - first an example, then the logic. Consider a model of the form $y_i = a + bx_i + e_i$, but with parameter estimation performed by a linear regression that ...
jbowman's user avatar
  • 38.3k
1 vote

Word for the compliment of an ablation study: replacing estimated feature with perfect information

Ceiling Analysis You are precisely describing Ceiling Analysis.
Frames Catherine White's user avatar
1 vote

Why are oracle inequalities called that way?

According to the paper "Modern statistical estimation via oracle inequalities": ‘Oracle inequalities’ are a powerful decision-theoretic tool which has served to understand the optimality of ...
Morpheus's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible