Skip to main content
deleted 51 characters in body
Source Link
Tim
  • 141.2k
  • 26
  • 270
  • 512

First notice that median minimizes the L1 norm (see here or here for learning more on L1 and L2)

$$ \DeclareMathOperator*{\argmin}{arg\,min} \text{median}(x) = \argmin_s \sum_i |x_i - s|^1 $$

while mean minimizes L2

$$ \text{mean}(x) = \argmin_s \sum_i |x_i - s|^2 $$

now, recall that Normal distributions' $\mu$ parameter can be estimated using sample mean, while the MLE estimator for Laplace distribution $\mu$ parameter is median. So using Normal distribution is equivalent to L2 norm optimization and using Laplace distribution, to using L1 optimization. In practice you can think of it as that median is less sensitive to outliers than mean, and the same, using fatter-tailed Laplace distribution as a prior makes your model less prone to outliers, than using Normal distribution.


Hurley, W. J. (2009) An Inductive Approach to Calculate the MLE for the Double Exponential Distribution. Journal of Modern Applied Statistical Methods: 8(2), Article 25.

Kotz, S., Kozubowski, T., Podgorski, K. (2001). The Laplace Distribution and Generalizations: A Revisit with Applications to Communications, Economics, Engineering, and Finance. Boston: Birkhäuser.

First notice that median minimizes the L1 norm (see here or here for learning more on L1 and L2)

$$ \DeclareMathOperator*{\argmin}{arg\,min} \text{median}(x) = \argmin_s \sum_i |x_i - s|^1 $$

while mean minimizes L2

$$ \text{mean}(x) = \argmin_s \sum_i |x_i - s|^2 $$

now, recall that Normal distributions' $\mu$ parameter can be estimated using sample mean, while the MLE estimator for Laplace distribution $\mu$ parameter is median. So using Normal distribution is equivalent to L2 norm optimization and using Laplace distribution, to using L1 optimization.


Hurley, W. J. (2009) An Inductive Approach to Calculate the MLE for the Double Exponential Distribution. Journal of Modern Applied Statistical Methods: 8(2), Article 25.

Kotz, S., Kozubowski, T., Podgorski, K. (2001). The Laplace Distribution and Generalizations: A Revisit with Applications to Communications, Economics, Engineering, and Finance. Boston: Birkhäuser.

First notice that median minimizes the L1 norm (see here or here for learning more on L1 and L2)

$$ \DeclareMathOperator*{\argmin}{arg\,min} \text{median}(x) = \argmin_s \sum_i |x_i - s|^1 $$

while mean minimizes L2

$$ \text{mean}(x) = \argmin_s \sum_i |x_i - s|^2 $$

now, recall that Normal distributions' $\mu$ parameter can be estimated using sample mean, while the MLE estimator for Laplace distribution $\mu$ parameter is median. So using Normal distribution is equivalent to L2 norm optimization and using Laplace distribution, to using L1 optimization. In practice you can think of it as that median is less sensitive to outliers than mean, and the same, using fatter-tailed Laplace distribution as a prior makes your model less prone to outliers, than using Normal distribution.


Hurley, W. J. (2009) An Inductive Approach to Calculate the MLE for the Double Exponential Distribution. Journal of Modern Applied Statistical Methods: 8(2), Article 25.

deleted 100 characters in body
Source Link
Tim
  • 141.2k
  • 26
  • 270
  • 512

First notice that median minimizes the L1 norm (see here or here for learning more on L1 and L2)

$$ \DeclareMathOperator*{\argmin}{arg\,min} \text{median}(x) = \argmin_s \sum_i |x_i - s|^1 $$

while mean minimizes L2

$$ \text{mean}(x) = \argmin_s \sum_i |x_i - s|^2 $$

now, recall that Normal distributions' $\mu$ parameter can be estimated using sample mean, while the MLE estimator for Laplace distribution $\mu$ parameter is median. So using Normal distribution is equivalent to L2 norm optimization and using Laplace distribution, to using L1 optimization.


Hurley, W. J. (2009) An Inductive Approach to Calculate the MLE for the Double Exponential Distribution. Journal of Modern Applied Statistical Methods: 8(2), Article 25.

Kotz, S., Kozubowski, T., Podgorski, K. (2001). The Laplace Distribution and Generalizations: A Revisit with Applications to Communications, Economics, Engineering, and Finance.The Laplace Distribution and Generalizations: A Revisit with Applications to Communications, Economics, Engineering, and Finance. Boston: Birkhäuser.

First notice that median minimizes the L1 norm (see here or here for learning more on L1 and L2)

$$ \DeclareMathOperator*{\argmin}{arg\,min} \text{median}(x) = \argmin_s \sum_i |x_i - s|^1 $$

while mean minimizes L2

$$ \text{mean}(x) = \argmin_s \sum_i |x_i - s|^2 $$

now, recall that Normal distributions' $\mu$ parameter can be estimated using sample mean, while the MLE estimator for Laplace distribution $\mu$ parameter is median. So using Normal distribution is equivalent to L2 norm optimization and using Laplace distribution, to using L1 optimization.


Hurley, W. J. (2009) An Inductive Approach to Calculate the MLE for the Double Exponential Distribution. Journal of Modern Applied Statistical Methods: 8(2), Article 25.

Kotz, S., Kozubowski, T., Podgorski, K. (2001). The Laplace Distribution and Generalizations: A Revisit with Applications to Communications, Economics, Engineering, and Finance. Boston: Birkhäuser.

First notice that median minimizes the L1 norm (see here or here for learning more on L1 and L2)

$$ \DeclareMathOperator*{\argmin}{arg\,min} \text{median}(x) = \argmin_s \sum_i |x_i - s|^1 $$

while mean minimizes L2

$$ \text{mean}(x) = \argmin_s \sum_i |x_i - s|^2 $$

now, recall that Normal distributions' $\mu$ parameter can be estimated using sample mean, while the MLE estimator for Laplace distribution $\mu$ parameter is median. So using Normal distribution is equivalent to L2 norm optimization and using Laplace distribution, to using L1 optimization.


Hurley, W. J. (2009) An Inductive Approach to Calculate the MLE for the Double Exponential Distribution. Journal of Modern Applied Statistical Methods: 8(2), Article 25.

Kotz, S., Kozubowski, T., Podgorski, K. (2001). The Laplace Distribution and Generalizations: A Revisit with Applications to Communications, Economics, Engineering, and Finance. Boston: Birkhäuser.

added 663 characters in body
Source Link
Tim
  • 141.2k
  • 26
  • 270
  • 512

First notice that median minimizes the L1 norm (see median minimizes the L1 normhere or here for learning more on L1 and L2)

$$ \DeclareMathOperator*{\argmin}{arg\,min} \text{median}(x) = \argmin_s \sum_i |x_i - s|^1 $$

while mean minimizes L2

$$ \text{mean}(x) = \argmin_s \sum_i |x_i - s|^2 $$

now, recall that Normal distribution hasdistributions' $\mu$ andparameter can be estimated using $\sigma$ parameters for mean and standard deviationsample mean, while the MLE estimator for Laplace distribution $\mu$ parameter is median. Now it should be clear thatSo using Normal distribution is equivalent to L2 norm optimization and using Laplace distribution, to using L1 optimization.


Hurley, W. J. (2009) An Inductive Approach to Calculate the MLE for the Double Exponential Distribution. Journal of Modern Applied Statistical Methods: 8(2), Article 25.

Kotz, S., Kozubowski, T., Podgorski, K. (2001). The Laplace Distribution and Generalizations: A Revisit with Applications to Communications, Economics, Engineering, and Finance. Boston: Birkhäuser.

First notice that median minimizes the L1 norm

$$ \DeclareMathOperator*{\argmin}{arg\,min} \text{median}(x) = \argmin_s \sum_i |x_i - s|^1 $$

mean minimizes L2

$$ \text{mean}(x) = \argmin_s \sum_i |x_i - s|^2 $$

now, recall that Normal distribution has $\mu$ and $\sigma$ parameters for mean and standard deviation, while the MLE estimator for Laplace distribution $\mu$ parameter is median. Now it should be clear that using Normal distribution is equivalent to L2 norm optimization and using Laplace distribution, to using L1 optimization.

First notice that median minimizes the L1 norm (see here or here for learning more on L1 and L2)

$$ \DeclareMathOperator*{\argmin}{arg\,min} \text{median}(x) = \argmin_s \sum_i |x_i - s|^1 $$

while mean minimizes L2

$$ \text{mean}(x) = \argmin_s \sum_i |x_i - s|^2 $$

now, recall that Normal distributions' $\mu$ parameter can be estimated using sample mean, while the MLE estimator for Laplace distribution $\mu$ parameter is median. So using Normal distribution is equivalent to L2 norm optimization and using Laplace distribution, to using L1 optimization.


Hurley, W. J. (2009) An Inductive Approach to Calculate the MLE for the Double Exponential Distribution. Journal of Modern Applied Statistical Methods: 8(2), Article 25.

Kotz, S., Kozubowski, T., Podgorski, K. (2001). The Laplace Distribution and Generalizations: A Revisit with Applications to Communications, Economics, Engineering, and Finance. Boston: Birkhäuser.

Source Link
Tim
  • 141.2k
  • 26
  • 270
  • 512
Loading