Skip to main content
added 6579 characters in body
Source Link
Sextus Empiricus
  • 86.5k
  • 6
  • 115
  • 301

Why does the von Mises-Fisher distribution need two parameters?

We do not necessarily need two sets of parameters: location and scale. You can combine them into one set or in another alternative parameterization. But, nobody has found it useful to describe the distribution that way (or at least not enough people such that it has become mainstream).

Why does the von Mises-Fisher distribution need two parameters?

We do not necessarily need two sets of parameters: location and scale. You can combine them into one set or in another alternative parameterization. But, nobody has found it useful to describe the distribution that way (or at least not enough people such that it has become mainstream).

added 6579 characters in body
Source Link
Sextus Empiricus
  • 86.5k
  • 6
  • 115
  • 301

The von Mises(-Fisher) distribution

The direction/coordinate $\mu$ describingvon Mises-Fisher distribution can be parameterized in terms of the positionlocation on a circle/sphere couldsphere. But this can also be described byparameterized in terms of spherical coordinates with unit length.

For instance in, we have for the 2D circlevon Mises distribution (the 2d special case) the direction/angletwo descriptions

  • in terms of the angle $\theta$

    $$f(\theta; \mu_{\theta},\kappa) = \frac{1}{2\pi I_0 (\kappa)} e^{\kappa \cos(\theta-\mu_\theta)}$$

  • in terms of the point $\vec{x} = \lbrace x_1, x_2 \rbrace$ on a circle of unit length

$$f(\vec{x}; \vec{\mu},\kappa) = \frac{1}{2\pi I_0 (\kappa)} e^{\kappa \vec{x} \cdot\vec{\mu} }$$

Why two parameters?

Why does the von Mises-Fisher distribution need two parameters?

The term 'parameter' here is a bit confusing. The distribution actually needs $\theta$ can be transformed$n$ parameters. What you are referring to is the split-up of those $n$ parameters into a coordinateset of $x,y$$n-1$ parameters relating to the location and a parameter relating to the scale. Those two sets, instead of combining them into a parameterization with a single set, are convenient. Location and scale are ambiguous in statistics and have intuitive meanings.

$$x,y= \cos(\theta),\sin(\theta)$$ The distribution can be written in more than $n$ parameters. There is no need to express the distribution which requires $n$ parameters in exactly $n$ parameters.

The set of location parameters, requiring $n-1$ parameters, can be expressed as an Euclidian coordinate on the unit sphere, in which case there are $n$ instead of $n-1$ parameters, of which one is superfluous parameter due to the constraint. You could leave out one of the coordinates or express it in terms of spherical coordinates.

Why not alternative parameterisation

Why not instead define the distribution for one unconstrained parameter i.e. $y := \kappa \mu \in \mathbb{R}^p$?

There's nothing that stops you from multiplying this withusing $\kappa$ and have a different parameterisation$\vec{y} = \kappa \vec{x}$ as an alternative parameterization. You can do it if you like, but it is not done because it is not useful.

$$x,y= \kappa \cos(\theta), \kappa \sin(\theta)$$ Expressing the position in terms of an Euclidian coordinate makes you add an extra parameter to the parameterization. This is a redundancy and you could use the redundancy to mix the $\kappa$ into the location parameter.

But, that is not why the redundancy has been created. The reason to use $n$ location parameters instead of $n-1$ is because the expression $\vec{x} \cdot \vec{\mu}$ is easier than the trigonometric functions when we would you?use spherical coordinates.

The constraint on $\mu$ does not relate to $\kappa$. It is not necessary to get the $\kappa$ mixed into the redundant parameter.

The $\mu$ and $\kappa$ have a physical and intuitive meaning. But, mixing them together into one single parameter does not seem to make sense.


 

Reasons for alternative parameterizations

There are various types of parameterizations for distributions and various reasons to use them. In your comments, you have mentioned four.

(1) notational succinctness

The constraint onnotation of the distribution function $\mu$ does not relate to$\frac{1}{2\pi I_0 (\kappa)} e^{\kappa \vec{x} \cdot\vec{\mu} }$ is very succinct when we express the location as $\kappa$$\vec{x}$.

The parameter $\mu$ gives Succinctness is exactly the very reason why we use the notation of the location, which only needs direction$n-1$ parameters, in terms of $n$ parameters with a constraint. Using spherical coordinates, or expressing just the first position$n-1$ coordinates of the center of$n$-dimensional vector $\vec{x}$ would reduce the parameters, but make the formulas more complicated.

Also, notational succinctness is not a goal in itself, but it should be a tool to reach another goal. In this answer there are some ways to reparameterize a 3 parameter distribution andfamily in terms of 2 parameters. Yes, we can reduce the number of parameters, we can make parameterization more succinct, but it has no magnitudeuse in itself. Scaling it is irrelevant

(2) avoiding confusion (e.g. my own)

There are indeed many distributions that have confusing parameterizations.

  • For instance, the geometric distribution expresses the probability of the number of Bernoulli trials until a successful trial is observed. The number of trials can be counted including the successful trial, or counting only the unsuccessful trials. This leads to the use of two different supports.

  • For instance, the non-central parameter in non-central distributions can be expressed in different ways. It can relate to a shift of the mean of the normal distributions from which the distribution is derived, or it can relate to a shift of the mean of the distribution itself.

  • For instance, the scale parameter in the normal distribution. Often we use the notation $\mathcal{N}(\mu, \sigma^2)$, which parameterizes the normal distribution by the mean and the variance. But some computer codes (for instance R) uses instead of $\sigma^2$ as the parameter, the standard deviation $\sigma$ as a parameter.

The parameter $\kappa$ relates to In the spreadcase of the von Mises distribution I find the parameterization in terms of a separate location parameter $\vec{x}$ (with the constraint that it is on the unit-sphere) and a scale parameter $\kappa$ not at all confusing. Combining them together would be more confusing.

(3) reducing computation

SeeMany distributions have parameters that are easier for instance this image from Wikipedia ofcomputations.

  • For instance, the beta distribution has parameters $\alpha$ and $\beta$ that relate to the powers in the distribution density $$f(x) \propto x^{\alpha-1}(1-x)^{\beta-1}$$ The more intuitive parameters are the mean $\mu$ and the variance $\sigma^2$ but then the distribution function would be $$f(x) \propto x^{\frac{\mu^2(1-\mu)}{\sigma^2}-\mu-1}(1-x)^{\frac{\mu(1-\mu)^2}{\sigma^2}+\mu}$$

I do not see how a computational simplification would be possible for the Vonvon Mises-Fisher distribution for the circleby using $\vec{y} = \kappa \vec{\mu}$. You state that you encounter an updating relation

$$\kappa_{nk} \mu_{nk} = \kappa_{n-1,k} \mu_{n-1,k} + \pi_{nk} o_n$$

Indeed with the alternative parameterization, you would get

$$\vec{y}_{nk} = \vec{y}_{n-1,k} + \pi_{nk} o_n$$

How did you get this parametrization? For your computations you could, of course, always switch to an alternative parametrization. The conversion between the single parameter $\kappa$ relates to$\vec{y}$ and the shape ortwo parameters spread of the distribution$(\vec{\mu},\kappa)$ is not so difficult.

I might also suggest that $\kappa \mu$ has a physically intuitive meaning

Image Physical meaning is another reason for the choice of parameterizations.

  • Like in the previous example with the beta distribution, the $\alpha$ and $\beta$ parameters are are useful for computations, but the mean and variance can be interpreted more easily.

  • In distributions like the exponential distribution and the Poisson distribution the 'rate' parameter is used in place of a 'scale' parameter.

    • The 'rate' has a direct meaning relating to the physical process. The rate parameter is a good choice to describe the properties of the physical process that creates the distribution.

    • The rate is the inverse of the 'scale' parameter. The scale parameter is often a preferable choice to describe distributions. It relates to the variation and to transformations of data.

NoteYou mention that $x$ isin the case of the von Mises(-Fisher) distribution the combination of the angular coordinate in this image$\mu\kappa$ parameter as some physical meaning like the combination of location and intensity, with the example of pulling a table cloth. But in 2 dimensional Euclidian space

What this can be expressed as two coordinates inexample lacks is a circledirect use or application of this physical meaning. For example, the peak height of the normal distribution is (with a fixed unit length)$f_{max} = \frac{1}{\sigma\sqrt{2\pi}}$. This has a physical meaning, namely the peak height, but should it be used just because it has a physical meaning?

The direction/coordinate $\mu$ describing the position on a circle/sphere could be described by coordinates with unit length.

For instance in the 2D circle case the direction/angle $\theta$ can be transformed into a coordinate $x,y$

$$x,y= \cos(\theta),\sin(\theta)$$

There's nothing that stops you from multiplying this with $\kappa$ and have a different parameterisation.

$$x,y= \kappa \cos(\theta), \kappa \sin(\theta)$$

But why would you?

The $\mu$ and $\kappa$ have a physical and intuitive meaning, mixing them together into one parameter does not seem to make sense.


 

The constraint on $\mu$ does not relate to $\kappa$.

The parameter $\mu$ gives the direction or the position of the center of the distribution and has no magnitude. Scaling it is irrelevant.

The parameter $\kappa$ relates to the spread of the distribution.

See for instance this image from Wikipedia of the Von Mises distribution for the circle.

The parameter $\kappa$ relates to the shape or spread of the distribution.

Image

Note that $x$ is the angular coordinate in this image. But in 2 dimensional Euclidian space this can be expressed as two coordinates in a circle (with a fixed unit length).

The von Mises(-Fisher) distribution

The von Mises-Fisher distribution can be parameterized in terms of the location on a sphere. But this can also be parameterized in terms of spherical coordinates.

For instance, we have for the von Mises distribution (the 2d special case) the two descriptions

  • in terms of the angle $\theta$

    $$f(\theta; \mu_{\theta},\kappa) = \frac{1}{2\pi I_0 (\kappa)} e^{\kappa \cos(\theta-\mu_\theta)}$$

  • in terms of the point $\vec{x} = \lbrace x_1, x_2 \rbrace$ on a circle of unit length

$$f(\vec{x}; \vec{\mu},\kappa) = \frac{1}{2\pi I_0 (\kappa)} e^{\kappa \vec{x} \cdot\vec{\mu} }$$

Why two parameters?

Why does the von Mises-Fisher distribution need two parameters?

The term 'parameter' here is a bit confusing. The distribution actually needs $n$ parameters. What you are referring to is the split-up of those $n$ parameters into a set of $n-1$ parameters relating to the location and a parameter relating to the scale. Those two sets, instead of combining them into a parameterization with a single set, are convenient. Location and scale are ambiguous in statistics and have intuitive meanings.

The distribution can be written in more than $n$ parameters. There is no need to express the distribution which requires $n$ parameters in exactly $n$ parameters.

The set of location parameters, requiring $n-1$ parameters, can be expressed as an Euclidian coordinate on the unit sphere, in which case there are $n$ instead of $n-1$ parameters, of which one is superfluous parameter due to the constraint. You could leave out one of the coordinates or express it in terms of spherical coordinates.

Why not alternative parameterisation

Why not instead define the distribution for one unconstrained parameter i.e. $y := \kappa \mu \in \mathbb{R}^p$?

There's nothing that stops you from using $\vec{y} = \kappa \vec{x}$ as an alternative parameterization. You can do it if you like, but it is not done because it is not useful.

Expressing the position in terms of an Euclidian coordinate makes you add an extra parameter to the parameterization. This is a redundancy and you could use the redundancy to mix the $\kappa$ into the location parameter.

But, that is not why the redundancy has been created. The reason to use $n$ location parameters instead of $n-1$ is because the expression $\vec{x} \cdot \vec{\mu}$ is easier than the trigonometric functions when we would use spherical coordinates.

The constraint on $\mu$ does not relate to $\kappa$. It is not necessary to get the $\kappa$ mixed into the redundant parameter.

The $\mu$ and $\kappa$ have a physical and intuitive meaning. But, mixing them together into one single parameter does not seem to make sense.

Reasons for alternative parameterizations

There are various types of parameterizations for distributions and various reasons to use them. In your comments, you have mentioned four.

(1) notational succinctness

The notation of the distribution function $\frac{1}{2\pi I_0 (\kappa)} e^{\kappa \vec{x} \cdot\vec{\mu} }$ is very succinct when we express the location as $\vec{x}$.

Succinctness is exactly the very reason why we use the notation of the location, which only needs $n-1$ parameters, in terms of $n$ parameters with a constraint. Using spherical coordinates, or expressing just the first $n-1$ coordinates of the $n$-dimensional vector $\vec{x}$ would reduce the parameters, but make the formulas more complicated.

Also, notational succinctness is not a goal in itself, but it should be a tool to reach another goal. In this answer there are some ways to reparameterize a 3 parameter distribution family in terms of 2 parameters. Yes, we can reduce the number of parameters, we can make parameterization more succinct, but it has no use in itself.

(2) avoiding confusion (e.g. my own)

There are indeed many distributions that have confusing parameterizations.

  • For instance, the geometric distribution expresses the probability of the number of Bernoulli trials until a successful trial is observed. The number of trials can be counted including the successful trial, or counting only the unsuccessful trials. This leads to the use of two different supports.

  • For instance, the non-central parameter in non-central distributions can be expressed in different ways. It can relate to a shift of the mean of the normal distributions from which the distribution is derived, or it can relate to a shift of the mean of the distribution itself.

  • For instance, the scale parameter in the normal distribution. Often we use the notation $\mathcal{N}(\mu, \sigma^2)$, which parameterizes the normal distribution by the mean and the variance. But some computer codes (for instance R) uses instead of $\sigma^2$ as the parameter, the standard deviation $\sigma$ as a parameter.

In the case of the von Mises distribution I find the parameterization in terms of a separate location parameter $\vec{x}$ (with the constraint that it is on the unit-sphere) and a scale parameter $\kappa$ not at all confusing. Combining them together would be more confusing.

(3) reducing computation

Many distributions have parameters that are easier for computations.

  • For instance, the beta distribution has parameters $\alpha$ and $\beta$ that relate to the powers in the distribution density $$f(x) \propto x^{\alpha-1}(1-x)^{\beta-1}$$ The more intuitive parameters are the mean $\mu$ and the variance $\sigma^2$ but then the distribution function would be $$f(x) \propto x^{\frac{\mu^2(1-\mu)}{\sigma^2}-\mu-1}(1-x)^{\frac{\mu(1-\mu)^2}{\sigma^2}+\mu}$$

I do not see how a computational simplification would be possible for the von Mises-Fisher distribution by using $\vec{y} = \kappa \vec{\mu}$. You state that you encounter an updating relation

$$\kappa_{nk} \mu_{nk} = \kappa_{n-1,k} \mu_{n-1,k} + \pi_{nk} o_n$$

Indeed with the alternative parameterization, you would get

$$\vec{y}_{nk} = \vec{y}_{n-1,k} + \pi_{nk} o_n$$

How did you get this parametrization? For your computations you could, of course, always switch to an alternative parametrization. The conversion between the single parameter $\vec{y}$ and the two parameters $(\vec{\mu},\kappa)$ is not so difficult.

I might also suggest that $\kappa \mu$ has a physically intuitive meaning

Physical meaning is another reason for the choice of parameterizations.

  • Like in the previous example with the beta distribution, the $\alpha$ and $\beta$ parameters are are useful for computations, but the mean and variance can be interpreted more easily.

  • In distributions like the exponential distribution and the Poisson distribution the 'rate' parameter is used in place of a 'scale' parameter.

    • The 'rate' has a direct meaning relating to the physical process. The rate parameter is a good choice to describe the properties of the physical process that creates the distribution.

    • The rate is the inverse of the 'scale' parameter. The scale parameter is often a preferable choice to describe distributions. It relates to the variation and to transformations of data.

You mention that in the case of the von Mises(-Fisher) distribution the combination of the $\mu\kappa$ parameter as some physical meaning like the combination of location and intensity, with the example of pulling a table cloth.

What this example lacks is a direct use or application of this physical meaning. For example, the peak height of the normal distribution is $f_{max} = \frac{1}{\sigma\sqrt{2\pi}}$. This has a physical meaning, namely the peak height, but should it be used just because it has a physical meaning?

added 427 characters in body
Source Link
Sextus Empiricus
  • 86.5k
  • 6
  • 115
  • 301

The direction/coordinate $\mu$ describing the position on a circle/sphere could be described by coordinates with unit length.

For instance in the 2D circle case the direction/angle $\theta$ can be transformed into a coordinate $x,y$

$$x,y= \cos(\theta),\sin(\theta)$$

There's nothing that stops you from multiplying this with $\kappa$ and have a different parameterisation.

$$x,y= \kappa \cos(\theta), \kappa \sin(\theta)$$

But why would you?

The $\mu$ and $\kappa$ have a physical and intuitive meaning, mixing them together into one parameter does not seem to make sense.


The constraint on $\mu$ does not relate to $\kappa$.

The parameter $\mu$ gives the direction or the position of the center of the distribution and has no magnitude. Scaling it is irrelevant.

The parameter $\kappa$ relates to the spread of the distribution.

See for instance this image from Wikipedia of the Von Mises distribution for the circle.

The parameter $\kappa$ relates to the shape or spread of the distribution.

Image

Note that $x$ is the angular coordinate in this image. But in 2 dimensional Euclidian space this can be expressed as two coordinates in a circle (with a fixed unit length).

The constraint on $\mu$ does not relate to $\kappa$.

The parameter $\mu$ gives the direction or the position of the center of the distribution and has no magnitude. Scaling it is irrelevant.

The parameter $\kappa$ relates to the spread of the distribution.

See for instance this image from Wikipedia of the Von Mises distribution for the circle.

The parameter $\kappa$ relates to the shape or spread of the distribution.

Image

Note that $x$ is the angular coordinate in this image. But in 2 dimensional Euclidian space this can be expressed as two coordinates in a circle (with a fixed unit length).

The direction/coordinate $\mu$ describing the position on a circle/sphere could be described by coordinates with unit length.

For instance in the 2D circle case the direction/angle $\theta$ can be transformed into a coordinate $x,y$

$$x,y= \cos(\theta),\sin(\theta)$$

There's nothing that stops you from multiplying this with $\kappa$ and have a different parameterisation.

$$x,y= \kappa \cos(\theta), \kappa \sin(\theta)$$

But why would you?

The $\mu$ and $\kappa$ have a physical and intuitive meaning, mixing them together into one parameter does not seem to make sense.


The constraint on $\mu$ does not relate to $\kappa$.

The parameter $\mu$ gives the direction or the position of the center of the distribution and has no magnitude. Scaling it is irrelevant.

The parameter $\kappa$ relates to the spread of the distribution.

See for instance this image from Wikipedia of the Von Mises distribution for the circle.

The parameter $\kappa$ relates to the shape or spread of the distribution.

Image

Note that $x$ is the angular coordinate in this image. But in 2 dimensional Euclidian space this can be expressed as two coordinates in a circle (with a fixed unit length).

added 5 characters in body
Source Link
Sextus Empiricus
  • 86.5k
  • 6
  • 115
  • 301
Loading
Source Link
Sextus Empiricus
  • 86.5k
  • 6
  • 115
  • 301
Loading