Title says it all.

I have seen both "the hyperparameter of the Dirichlet distribution" and "the parameter of the Dirichlet distribution"

What are the differences?

  • 2
    $\begingroup$ en.wikipedia.org/wiki/Hyperparameter $\endgroup$ – Glen_b Jan 12 '15 at 4:30
  • 2
    $\begingroup$ See also the answer here $\endgroup$ – Glen_b Oct 2 '15 at 9:58
  • $\begingroup$ @Glen_b, it seems that we have three identical threads asking about a definiton of hyperparameter. I suggest to merge (moving the answers) all of them into stats.stackexchange.com/questions/208225. But your answer here is accepted and you will lose this green tick if this answer is moved. What do you think? $\endgroup$ – amoeba May 14 '16 at 18:27
  • $\begingroup$ @amoeba I don't really mind about the tick. I'll have a closer look at the threads; a certain degree of caution is called for with merging. $\endgroup$ – Glen_b May 15 '16 at 1:05
  • $\begingroup$ @Glen_b: Shall I raise this as a suggestion on meta, or is it okay to leave it as a pending flag for the moderators to figure out? $\endgroup$ – amoeba May 17 '16 at 16:05

A hyperparameter is a parameter for the (prior) distribution of some parameter.

So for a simple example, let's say we state that the variance parameter $\tau^2$ in some problem has a uniform prior on $(0,\theta)$.

(I personally would be unlikely to do such a thing, but it happens; I might in some very particular circumstance)

Then $\tau^2$ is a parameter (in the distribution of the data) and $\theta$ is a hyperparameter.

If we then in turn specify a (prior) distribution for $\theta$ (e.g. that it's Gamma with mean 100 and shape parameter 2), that's a hyperprior - a prior distribution on a parameter of a prior distribution.

| cite | improve this answer | |
  • 1
    $\begingroup$ how is that related to the hyperparameters of ML algorithms? $\endgroup$ – Antoine Oct 2 '15 at 9:04
  • $\begingroup$ never mind, I just went through the duplicate question $\endgroup$ – Antoine Oct 2 '15 at 9:05
  • 3
    $\begingroup$ stats.stackexchange.com/questions/149098/… $\endgroup$ – Antoine Oct 2 '15 at 9:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.