Title says it all.
I have seen both "the hyperparameter of the Dirichlet distribution" and "the parameter of the Dirichlet distribution"
What are the differences?
$\begingroup$
$\endgroup$
6
asked Jan 12, 2015 at 4:01
-
2$\begingroup$ en.wikipedia.org/wiki/Hyperparameter $\endgroup$– Glen_bCommented Jan 12, 2015 at 4:30
-
2$\begingroup$ See also the answer here $\endgroup$– Glen_bCommented Oct 2, 2015 at 9:58
-
$\begingroup$ @Glen_b, it seems that we have three identical threads asking about a definiton of hyperparameter. I suggest to merge (moving the answers) all of them into stats.stackexchange.com/questions/208225. But your answer here is accepted and you will lose this green tick if this answer is moved. What do you think? $\endgroup$– amoebaCommented May 14, 2016 at 18:27
-
$\begingroup$ @amoeba I don't really mind about the tick. I'll have a closer look at the threads; a certain degree of caution is called for with merging. $\endgroup$– Glen_bCommented May 15, 2016 at 1:05
-
$\begingroup$ @Glen_b: Shall I raise this as a suggestion on meta, or is it okay to leave it as a pending flag for the moderators to figure out? $\endgroup$– amoebaCommented May 17, 2016 at 16:05
|
Show 1 more comment
1 Answer
$\begingroup$
$\endgroup$
3
A hyperparameter is a parameter for the (prior) distribution of some parameter.
So for a simple example, let's say we state that the variance parameter $\tau^2$ in some problem has a uniform prior on $(0,\theta)$.
(I personally would be unlikely to do such a thing, but it happens; I might in some very particular circumstance)
Then $\tau^2$ is a parameter (in the distribution of the data) and $\theta$ is a hyperparameter.
If we then in turn specify a (prior) distribution for $\theta$ (e.g. that it's Gamma with mean 100 and shape parameter 2), that's a hyperprior - a prior distribution on a parameter of a prior distribution.
-
1$\begingroup$ how is that related to the hyperparameters of ML algorithms? $\endgroup$– AntoineCommented Oct 2, 2015 at 9:04
-
$\begingroup$ never mind, I just went through the duplicate question $\endgroup$– AntoineCommented Oct 2, 2015 at 9:05
-
3$\begingroup$ stats.stackexchange.com/questions/149098/… $\endgroup$– AntoineCommented Oct 2, 2015 at 9:56