The SCAD penalty $p(x | \lambda, \gamma)$ from https://myweb.uiowa.edu/pbreheny/7600/s16/notes/2-29.pdf or the paper "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties" is in the form of $$p(x | \lambda, \gamma) = \begin{cases} \lambda |x|\ \ \ \ \mathrm{if}\ |x|\leq \lambda,\\ \frac{2 \gamma \lambda |x| - x^2 - \lambda^2}{2(\gamma-1)}\ \ \ \ \mathrm{if}\ \lambda<|x|<\gamma\lambda,\\ \frac{\lambda^2(\gamma + 1)}{2}\ \ \ \ \mathrm{if}\ |x|\geq \gamma\lambda, \end{cases}$$ for $\lambda>0$ and $\gamma>2$. I am curious why we need $\gamma>2$ in this definition.

  • 1
    $\begingroup$ have you tried plotting the penalty with $\gamma>2$ and $\gamma \leq 2$? $\endgroup$
    – Ben
    Commented Jul 7, 2023 at 17:47
  • 1
    $\begingroup$ Since the function $p(x|\lambda,\gamma)$ is even, we can only focus on $x \geq 0$. I think $p(x|\lambda,\gamma) = \lambda x$ if $x \in [0, \lambda]$. Then it increases as a concave function in $x \in (\lambda, \gamma \lambda)$. And after $x \geq \lambda \gamma$, it is a flat line. The whole function of $x \geq 0$ is continuous with continuous derivative. If we just check the existence of this function, I think $\gamma > 1$ is enough. $\endgroup$ Commented Jul 7, 2023 at 19:25


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.