0
$\begingroup$

With regards to Bayesian statistics, I understand the kernel of a probability density function (pdf) or probability mass function (pmf) to be the form of the pdf or pmf in which any factors that are not functions of any of the variables in the domain are omitted. So to test my understanding, for a Normal distribution we would have:

$N(\mu,\sigma^2)=\frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{(x-\mu)^2}{2\sigma^2}}$ $\rightarrow$ and its kernel would be $\rightarrow$ $e^{-\frac{(x-\mu)^2}{2\sigma^2}}$

and for a Binomial distribution, we would have:

$Bin(n,p) = \left(n \atop x\right)p^x(1-p)^{n-x}$ $\rightarrow$ and its kernel would be $\rightarrow$ $\frac{p^x(1-p)^{n-x}}{x!(n-x)!}$

First, is my definition correct and second, is my understanding correct?

$\endgroup$
8
  • 3
    $\begingroup$ See: stats.stackexchange.com/questions/146995/… $\endgroup$ – kjetil b halvorsen Mar 30 at 12:45
  • 1
    $\begingroup$ (a) Missing minus signs in exponent of $e$ in the normal. Need to clarify whether $\sigma$ is taken to be constant. (b) For the kernel of a binomial likelihood is just $p^x(1-p)^{n-x}$ because your denominator does not contain $p.$ $\endgroup$ – BruceET Mar 30 at 13:11
  • $\begingroup$ Thanks @BruceET, I made the correction for the Normal distribution. This is where I get confused. For the normal, it was clear that since $x$ was only present in the exponential term, you would only keep that term. Why is it that you do not keep the $\left(n \atop x\right)$ term even though it includes $x$? Isn't this part a function of a variable in the domain, namely $x$? $\endgroup$ – TSP Mar 30 at 19:19
  • $\begingroup$ The key values to watch are parameters $(\mu, \sigma$ for normal; $p$ for binomial, when $n$ is given). We don't keep ${n\choose x}$ because it contains no $p.$ $\endgroup$ – BruceET Mar 30 at 19:34
  • $\begingroup$ @Bruce Whether or not that is correct, it contradicts the definition provided in the link given by Kjetil above. $\endgroup$ – whuber Mar 30 at 19:36
0
$\begingroup$

Let's stay on topic with Bayesian statistics. Call the binomial success parameter $\theta.$

If you put a prior probability distribution on $\theta,$ it might be $\mathsf{Beta}(2,3)$ and the relevant beta density might be written as $$p(\theta) = \frac{\Gamma(2+3)}{\Gamma(2)\Gamma(3)}\theta^{2-1}(1-\theta)^{3-1} = 12\theta^{2-1}(1-\theta)^{3-1}.$$ for $0<\theta<1.$

This is a function in $\theta$ with kernel $p(\theta) \propto \theta(1-\theta)^2.$

Then if you observe $x=41$ successes in $n=60$ trials, the likelihood function $f(x|\theta)$ is the binomial PDF considered as a function of $\theta$ proportional to $$\theta^{x}(1-\theta)^{n-x} = \theta^{41}(1-\theta)^{19},$$ where $0 <\theta<1.$ (Even in frequentist statistics it is customary to omit irrelevant constants of integration (or summation) from the likelihood function.)

So neither the prior nor the likelihood has gamma functions or factorials. Then, because the beta prior and the binomial likelihood are 'conjugate' (mathematically compatible) you can use Bayes' Theorem to multiply the prior by the likelihood to get an expression which you can recognize as the kernel $$\theta^{2+41 -1}(1-\theta)^{3+19=1} = \theta^{43-1}(1-\theta)^{22-1}$$ of the posterior distribution $\mathsf{Beta}(2 + 41, 3 + 19) \equiv \mathsf{Beta}(43, 22)$

Note: Notice that the parameters of a beta distribution are 'out of phase by 1' with the exponents of $\theta$ and $(1-\theta)$ in the PDF and its kernel.

$\endgroup$
4
  • $\begingroup$ Thanks for the answer. Would it be fair to say that for discrete distributions, you keep factors containing the parameter of interest (e.g., for Binomial distributions where the parameter of interest is p, you keep factors containing p)? Additionally, for continuous distributions, do you just keep factors containing the parameter x? $\endgroup$ – TSP Mar 31 at 3:04
  • $\begingroup$ Throughout (discrete or continuous), because $\theta$ is the parameter being investigated, you keep factors containing $\theta.$ Sometimes $x$ is in the exponent of such a factor. Similarly, In the likelihood data $x = 41$ is kept for the same reason. $\endgroup$ – BruceET Mar 31 at 4:56
  • $\begingroup$ Just to make sure my understanding is concrete, lets say for the Binomial distribution, one was investigating the parameter $n$ similar to the paper "Inference for the binomial $N$ parameter". In this case, would the kernel be $\left(n \atop x\right)(1-p)^{n-x}$ keeping only those factors containing n? $\endgroup$ – TSP Apr 1 at 0:59
  • 1
    $\begingroup$ That is an unusual and more difficult problem than doing inference on $p.$ But, Yes. See end of first section in linked paper. $\endgroup$ – BruceET Apr 1 at 1:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.