Skip to main content
Bounty Ended with 50 reputation awarded by Alejandro Rodriguez
added 76 characters in body
Source Link
TemplateRex
  • 713
  • 4
  • 11
  1. The logarithm of the probability of multiple joint probabilities simplifies to the sum of the logarithms of the individual probabilities (and the sum rule is easier than the product rule for differentiation)
$\log \left(\prod_i P(x_i)\right) = \sum_i \log \left( P(x_i)\right)$
  1. The logarithm of a member of the family of exponential probability distributions (which includes the ubiquitous normal) is polynomial in the parameters (i.e. max-likelihood reduces to least-squares for normal distributions)
$\log\left(\exp\left(-\frac{1}{2}x^2\right)\right) = -\frac{1}{2}x^2$
  1. The latter form is both more numerically stable and symbolically easier to differentiate than the former.

  2. Last but not least, the logarithm is a monotonic transformation that preserves the locations of the extrema (in particular, the estimated parameters in max-likelihood are identical for the original and the log-transformed formulation)

  1. The logarithm of the probability of multiple joint probabilities simplifies to the sum of the logarithms of the individual probabilities (and the sum rule is easier than the product rule for differentiation)
$\log \left(\prod_i P(x_i)\right) = \sum_i \log \left( P(x_i)\right)$
  1. The logarithm of a member of the family of exponential probability distributions (which includes the ubiquitous normal) is polynomial in the parameters
$\log\left(\exp\left(-\frac{1}{2}x^2\right)\right) = -\frac{1}{2}x^2$
  1. The latter form is both more numerically stable and symbolically easier to differentiate than the former.

  2. Last but not least, the logarithm is a monotonic transformation that preserves the locations of the extrema (in particular, the estimated parameters in max-likelihood are identical for the original and the log-transformed formulation)

  1. The logarithm of the probability of multiple joint probabilities simplifies to the sum of the logarithms of the individual probabilities (and the sum rule is easier than the product rule for differentiation)
$\log \left(\prod_i P(x_i)\right) = \sum_i \log \left( P(x_i)\right)$
  1. The logarithm of a member of the family of exponential probability distributions (which includes the ubiquitous normal) is polynomial in the parameters (i.e. max-likelihood reduces to least-squares for normal distributions)
$\log\left(\exp\left(-\frac{1}{2}x^2\right)\right) = -\frac{1}{2}x^2$
  1. The latter form is both more numerically stable and symbolically easier to differentiate than the former.

  2. Last but not least, the logarithm is a monotonic transformation that preserves the locations of the extrema (in particular, the estimated parameters in max-likelihood are identical for the original and the log-transformed formulation)

added 120 characters in body
Source Link
TemplateRex
  • 713
  • 4
  • 11
  1. The logarithm of the probability of multiple joint probabilities simplifies to the sum of the logarithms of the individual probabilities (and the sum rule is easier than the product rule for differentiation)
$\log \left(\prod_i P(x_i)\right) = \sum_i \log \left( P(x_i)\right)$
  1. The logarithm of a member of the family of exponential probability distributions (which includes the ubiquitous normal) is polynomial in the parameters
$\log\left(\exp\left(-\frac{1}{2}x^2\right)\right) = -\frac{1}{2}x^2$
  1. The latter form is both more numerically stable and symbolically easier to differentiate than the former.

    The latter form is both more numerically stable and symbolically easier to differentiate than the former.

  2. Last but not least, the logarithm is a monotonic transformation that preserves the locations of the extrema (in particular, the estimated parameters in max-likelihood are identical for the original and the log-transformed formulation)

  1. The logarithm of the probability of multiple joint probabilities simplifies to the sum of the logarithms of the individual probabilities (and the sum rule is easier than the product rule for differentiation)
$\log \left(\prod_i P(x_i)\right) = \sum_i \log \left( P(x_i)\right)$
  1. The logarithm of a member of the family of exponential probability distributions (which includes the ubiquitous normal) is polynomial in the parameters
$\log\left(\exp\left(-\frac{1}{2}x^2\right)\right) = -\frac{1}{2}x^2$
  1. The latter form is both more numerically stable and symbolically easier to differentiate than the former.
  1. The logarithm of the probability of multiple joint probabilities simplifies to the sum of the logarithms of the individual probabilities (and the sum rule is easier than the product rule for differentiation)
$\log \left(\prod_i P(x_i)\right) = \sum_i \log \left( P(x_i)\right)$
  1. The logarithm of a member of the family of exponential probability distributions (which includes the ubiquitous normal) is polynomial in the parameters
$\log\left(\exp\left(-\frac{1}{2}x^2\right)\right) = -\frac{1}{2}x^2$
  1. The latter form is both more numerically stable and symbolically easier to differentiate than the former.

  2. Last but not least, the logarithm is a monotonic transformation that preserves the locations of the extrema (in particular, the estimated parameters in max-likelihood are identical for the original and the log-transformed formulation)

Source Link
TemplateRex
  • 713
  • 4
  • 11

  1. The logarithm of the probability of multiple joint probabilities simplifies to the sum of the logarithms of the individual probabilities (and the sum rule is easier than the product rule for differentiation)
$\log \left(\prod_i P(x_i)\right) = \sum_i \log \left( P(x_i)\right)$
  1. The logarithm of a member of the family of exponential probability distributions (which includes the ubiquitous normal) is polynomial in the parameters
$\log\left(\exp\left(-\frac{1}{2}x^2\right)\right) = -\frac{1}{2}x^2$
  1. The latter form is both more numerically stable and symbolically easier to differentiate than the former.