Skip to main content
4 of 10
added 502 characters in body
Tim
  • 141.2k
  • 26
  • 270
  • 512

You already got two nice answers, but since it still seems unclear for you let me provide one. Likelihood is defined as

$$ \mathcal{L}(\theta|X) = P(X|\theta) = \prod_i f_\theta(x_i) $$

so we have likelihood of some parameter value $\theta$ given the data $X$. It is equal to product of probability mass (discrete case), or density (continuous case) functions $f$ of $X$ parametrized by $\theta$. Likelihood is a function of parameter given the data. Notice that $\theta$ is a parameter that we are optimizing, not a random variable, so it does not have any probabilities assigned to it. This is why Wikipedia states that using conditional probability notation may be ambiguous, since we are not conditioning on any random variable. On another hand, in Bayesian setting $\theta$ is a random variable and does have distribution, so we can work with it as with any other random variable and we can use Bayes theorem to calculate the posterior probabilities. Bayesian likelihood is still likelihood since it tells us about likelihood of data given the parameter, the only difference is that the parameter is considered as random variable.

If you know programming, you can think of likelihood function as of overloaded function in programming. Some programming languages allow you to have function that works differently when called using different parameter types. If you think of likelihood like this, then by default if takes as argument some parameter value and returns likelihood of data given this parameter. On another hand, you can use such function in Bayesian setting, where parameter is random variable, this leads to basically the same output, but that can be understood as conditional probability since we are conditioning on random variable. In both cases the function works the same, just you use it and understand it a little bit differently.

// likelihood "as" overloaded function
Default Likelihood(Numeric theta, Data X) {
    return f(X, theta); // returns likelihood, not probability
}

Bayesian Likelihood(RandomVariable theta, Data X) {
    return f(X, theta); // since theta is r.v., the output can be
                        // understood as conditional probability
}

Moreover, you won't be able to find Bayesian who writes Bayes theorem as

$$ P(\theta|X) \propto \mathcal{L}(\theta|X) P(\theta) $$

...this would be very confusing. First, you would have $\theta|X$ on both sides of equation and it wouldn't have much sens. Second, we have posterior probability to know about probability of $\theta$ given data. Third, since $\theta$ is a random variable, we have and write it as conditional probability. The L-notation is rather reserved for likelihoodist setting.

Tim
  • 141.2k
  • 26
  • 270
  • 512