Skip to main content
edited body
Source Link
Ben
  • 132.9k
  • 7
  • 255
  • 588

A sufficient condition for sparsity is that $\min_{\theta \neq 0} [ |\theta| + p'_\lambda (|\theta|) ]$ is positive.

The reason for this condition is explained on p. 1350 of the paper, but it is only a brief outline, as the authors have chosen to omit most of the demonstration of the asserted results. I will try to fill in the blanks of what they have omitted.


The mathematical problem being solved here is to find a value $\hat{\theta} > 0$ that minimises a particular objective function for a fixed value of $z > 0$. The objective function specified in equation $(2.2)$$(2.3)$ is:

$$\begin{equation} \begin{aligned} F(\theta) &\equiv \tfrac{1}{2} (z-\theta)^2 + p_\lambda(|\theta|) \\[6pt] &= \tfrac{1}{2} z^2 - z \theta + \tfrac{1}{2} \theta^2 + p_\lambda(|\theta|) \\[6pt] &= \tfrac{1}{2} \theta^2 - z \theta + p_\lambda(|\theta|) + \text{const.} \\[6pt] \end{aligned} \end{equation}$$$$\begin{equation} \begin{aligned} F(z,\theta) &\equiv \tfrac{1}{2} (z-\theta)^2 + p_\lambda(|\theta|) \\[6pt] &= \tfrac{1}{2} z^2 - z \theta + \tfrac{1}{2} \theta^2 + p_\lambda(|\theta|) \\[6pt] &= \tfrac{1}{2} \theta^2 - z \theta + p_\lambda(|\theta|) + \text{const.} \\[6pt] \end{aligned} \end{equation}$$

Finding the minimising argument is done by ordinary calculus techniques. To facilitate this analysis, we define the function $H_\lambda(\theta) \equiv |\theta| + p'_\lambda (|\theta|)$. Using the chain rule, for all $\theta \neq 0$ the first derivative of this objective function is:

$$\begin{equation} \begin{aligned} \quad \text{ } \text{ } F'(\theta) &= \theta + \text{sgn}(\theta) \cdot p'_\lambda (|\theta|) -z \\[6pt] &= \text{sgn}(\theta) \cdot |\theta| + \text{sgn}(\theta) \cdot p'_\lambda (|\theta|) -z \\[6pt] &= \text{sgn}(\theta) \Big[ |\theta| + p'_\lambda (|\theta|) \Big] -z \\[6pt] &= \text{sgn}(\theta) \cdot H_\lambda(\theta) -z. \\[6pt] \end{aligned} \end{equation}$$$$\begin{equation} \begin{aligned} \quad \text{ } \text{ } F'(z,\theta) &= \theta + \text{sgn}(\theta) \cdot p'_\lambda (|\theta|) -z \\[6pt] &= \text{sgn}(\theta) \cdot |\theta| + \text{sgn}(\theta) \cdot p'_\lambda (|\theta|) -z \\[6pt] &= \text{sgn}(\theta) \Big[ |\theta| + p'_\lambda (|\theta|) \Big] -z \\[6pt] &= \text{sgn}(\theta) \cdot H_\lambda(\theta) -z. \\[6pt] \end{aligned} \end{equation}$$

This gets us to the point where the derivative of the objective function depends on the function $H_\lambda(\theta)$, which is the function subject to the condition in question. To complete the examination of the minimisation problem, the authors look at what happens in different cases when the minimum of this function is positive, negative, or zero. This discussion occurs on p. 1350 of the paper, and with the above form for the derivative of the objective function, it should now be easier to understand.

(As some other commentators have also noted, Figure 3 appears to me to be intended as a generic figure, rather than one corresponding to a specific form of probability function.)

A sufficient condition for sparsity is that $\min_{\theta \neq 0} [ |\theta| + p'_\lambda (|\theta|) ]$ is positive.

The reason for this condition is explained on p. 1350 of the paper, but it is only a brief outline, as the authors have chosen to omit most of the demonstration of the asserted results. I will try to fill in the blanks of what they have omitted.


The mathematical problem being solved here is to find a value $\hat{\theta} > 0$ that minimises a particular objective function for a fixed value of $z > 0$. The objective function specified in equation $(2.2)$ is:

$$\begin{equation} \begin{aligned} F(\theta) &\equiv \tfrac{1}{2} (z-\theta)^2 + p_\lambda(|\theta|) \\[6pt] &= \tfrac{1}{2} z^2 - z \theta + \tfrac{1}{2} \theta^2 + p_\lambda(|\theta|) \\[6pt] &= \tfrac{1}{2} \theta^2 - z \theta + p_\lambda(|\theta|) + \text{const.} \\[6pt] \end{aligned} \end{equation}$$

Finding the minimising argument is done by ordinary calculus techniques. To facilitate this analysis, we define the function $H_\lambda(\theta) \equiv |\theta| + p'_\lambda (|\theta|)$. Using the chain rule, for all $\theta \neq 0$ the first derivative of this objective function is:

$$\begin{equation} \begin{aligned} \quad \text{ } \text{ } F'(\theta) &= \theta + \text{sgn}(\theta) \cdot p'_\lambda (|\theta|) -z \\[6pt] &= \text{sgn}(\theta) \cdot |\theta| + \text{sgn}(\theta) \cdot p'_\lambda (|\theta|) -z \\[6pt] &= \text{sgn}(\theta) \Big[ |\theta| + p'_\lambda (|\theta|) \Big] -z \\[6pt] &= \text{sgn}(\theta) \cdot H_\lambda(\theta) -z. \\[6pt] \end{aligned} \end{equation}$$

This gets us to the point where the derivative of the objective function depends on the function $H_\lambda(\theta)$, which is the function subject to the condition in question. To complete the examination of the minimisation problem, the authors look at what happens in different cases when the minimum of this function is positive, negative, or zero. This discussion occurs on p. 1350 of the paper, and with the above form for the derivative of the objective function, it should now be easier to understand.

(As some other commentators have also noted, Figure 3 appears to me to be intended as a generic figure, rather than one corresponding to a specific form of probability function.)

A sufficient condition for sparsity is that $\min_{\theta \neq 0} [ |\theta| + p'_\lambda (|\theta|) ]$ is positive.

The reason for this condition is explained on p. 1350 of the paper, but it is only a brief outline, as the authors have chosen to omit most of the demonstration of the asserted results. I will try to fill in the blanks of what they have omitted.


The mathematical problem being solved here is to find a value $\hat{\theta} > 0$ that minimises a particular objective function for a fixed value of $z > 0$. The objective function specified in equation $(2.3)$ is:

$$\begin{equation} \begin{aligned} F(z,\theta) &\equiv \tfrac{1}{2} (z-\theta)^2 + p_\lambda(|\theta|) \\[6pt] &= \tfrac{1}{2} z^2 - z \theta + \tfrac{1}{2} \theta^2 + p_\lambda(|\theta|) \\[6pt] &= \tfrac{1}{2} \theta^2 - z \theta + p_\lambda(|\theta|) + \text{const.} \\[6pt] \end{aligned} \end{equation}$$

Finding the minimising argument is done by ordinary calculus techniques. To facilitate this analysis, we define the function $H_\lambda(\theta) \equiv |\theta| + p'_\lambda (|\theta|)$. Using the chain rule, for all $\theta \neq 0$ the first derivative of this objective function is:

$$\begin{equation} \begin{aligned} \quad \text{ } \text{ } F'(z,\theta) &= \theta + \text{sgn}(\theta) \cdot p'_\lambda (|\theta|) -z \\[6pt] &= \text{sgn}(\theta) \cdot |\theta| + \text{sgn}(\theta) \cdot p'_\lambda (|\theta|) -z \\[6pt] &= \text{sgn}(\theta) \Big[ |\theta| + p'_\lambda (|\theta|) \Big] -z \\[6pt] &= \text{sgn}(\theta) \cdot H_\lambda(\theta) -z. \\[6pt] \end{aligned} \end{equation}$$

This gets us to the point where the derivative of the objective function depends on the function $H_\lambda(\theta)$, which is the function subject to the condition in question. To complete the examination of the minimisation problem, the authors look at what happens in different cases when the minimum of this function is positive, negative, or zero. This discussion occurs on p. 1350 of the paper, and with the above form for the derivative of the objective function, it should now be easier to understand.

(As some other commentators have also noted, Figure 3 appears to me to be intended as a generic figure, rather than one corresponding to a specific form of probability function.)

Source Link
Ben
  • 132.9k
  • 7
  • 255
  • 588

A sufficient condition for sparsity is that $\min_{\theta \neq 0} [ |\theta| + p'_\lambda (|\theta|) ]$ is positive.

The reason for this condition is explained on p. 1350 of the paper, but it is only a brief outline, as the authors have chosen to omit most of the demonstration of the asserted results. I will try to fill in the blanks of what they have omitted.


The mathematical problem being solved here is to find a value $\hat{\theta} > 0$ that minimises a particular objective function for a fixed value of $z > 0$. The objective function specified in equation $(2.2)$ is:

$$\begin{equation} \begin{aligned} F(\theta) &\equiv \tfrac{1}{2} (z-\theta)^2 + p_\lambda(|\theta|) \\[6pt] &= \tfrac{1}{2} z^2 - z \theta + \tfrac{1}{2} \theta^2 + p_\lambda(|\theta|) \\[6pt] &= \tfrac{1}{2} \theta^2 - z \theta + p_\lambda(|\theta|) + \text{const.} \\[6pt] \end{aligned} \end{equation}$$

Finding the minimising argument is done by ordinary calculus techniques. To facilitate this analysis, we define the function $H_\lambda(\theta) \equiv |\theta| + p'_\lambda (|\theta|)$. Using the chain rule, for all $\theta \neq 0$ the first derivative of this objective function is:

$$\begin{equation} \begin{aligned} \quad \text{ } \text{ } F'(\theta) &= \theta + \text{sgn}(\theta) \cdot p'_\lambda (|\theta|) -z \\[6pt] &= \text{sgn}(\theta) \cdot |\theta| + \text{sgn}(\theta) \cdot p'_\lambda (|\theta|) -z \\[6pt] &= \text{sgn}(\theta) \Big[ |\theta| + p'_\lambda (|\theta|) \Big] -z \\[6pt] &= \text{sgn}(\theta) \cdot H_\lambda(\theta) -z. \\[6pt] \end{aligned} \end{equation}$$

This gets us to the point where the derivative of the objective function depends on the function $H_\lambda(\theta)$, which is the function subject to the condition in question. To complete the examination of the minimisation problem, the authors look at what happens in different cases when the minimum of this function is positive, negative, or zero. This discussion occurs on p. 1350 of the paper, and with the above form for the derivative of the objective function, it should now be easier to understand.

(As some other commentators have also noted, Figure 3 appears to me to be intended as a generic figure, rather than one corresponding to a specific form of probability function.)