# What exactly is a hyperparameter?

Title says it all.

I have seen both "the hyperparameter of the Dirichlet distribution" and "the parameter of the Dirichlet distribution"

What are the differences?

• en.wikipedia.org/wiki/Hyperparameter – Glen_b Jan 12 '15 at 4:30
• @Glen_b, it seems that we have three identical threads asking about a definiton of hyperparameter. I suggest to merge (moving the answers) all of them into stats.stackexchange.com/questions/208225. But your answer here is accepted and you will lose this green tick if this answer is moved. What do you think? – amoeba May 14 '16 at 18:27
• @amoeba I don't really mind about the tick. I'll have a closer look at the threads; a certain degree of caution is called for with merging. – Glen_b May 15 '16 at 1:05
• @Glen_b: Shall I raise this as a suggestion on meta, or is it okay to leave it as a pending flag for the moderators to figure out? – amoeba May 17 '16 at 16:05

A hyperparameter is a parameter for the (prior) distribution of some parameter.

So for a simple example, let's say we state that the variance parameter $\tau^2$ in some problem has a uniform prior on $(0,\theta)$.

(I personally would be unlikely to do such a thing, but it happens; I might in some very particular circumstance)

Then $\tau^2$ is a parameter (in the distribution of the data) and $\theta$ is a hyperparameter.

If we then in turn specify a (prior) distribution for $\theta$ (e.g. that it's Gamma with mean 100 and shape parameter 2), that's a hyperprior - a prior distribution on a parameter of a prior distribution.

• how is that related to the hyperparameters of ML algorithms? – Antoine Oct 2 '15 at 9:04
• never mind, I just went through the duplicate question – Antoine Oct 2 '15 at 9:05
• stats.stackexchange.com/questions/149098/… – Antoine Oct 2 '15 at 9:56