6
$\begingroup$

If I have given distribution family, say normal, is there a way how to derive what are the location and scale parameters based on the probability density function (PDF)?

I know in case of normal distribution, it is $\mu$ and $\sigma$ respectively. I know how to show, that these two are in fact location and scale parameters.

But what if I had no idea, what are the scale and location parameters and wanted to use PDF to find them?

Alternatively, if I can't tell from PDF as such, is there some other approach of deriving the location and scale?

It doesn't have to be restricted to Normal distribution. Alternatively, I would like to know, how to derive, that for distribution Uniform[a, b] i have location a and scale b - a.

$\endgroup$
  • $\begingroup$ Bear in mind that a family of distributions need not be characterizable by location & scale parameters alone. $\endgroup$ – Scortchi - Reinstate Monica Mar 7 '17 at 11:52
  • $\begingroup$ I am aware of that, but I am interested in whether it is possible to derive from PDF the location and scale parameters (if the distribution has those). Or whether such attempt doesn't make much sense, because of some property of location and scale parameters that I have missed. $\endgroup$ – ira Mar 7 '17 at 13:46
5
$\begingroup$

Let's talk about what a location parameter is. (The discussion of scale parameters will exactly parallel it and offers little new.)

The setting concerns a set $\mathcal{F}$ of probability distributions $F_\theta$ indexed by a parameter $\theta \in \Theta \subset \mathbb{R}^p$. "Indexed by" not only means each $\theta\in\Theta$ denotes a distribution $F_\theta$: it also means that no distribution is identified by more than one such $\theta$.

Consider any location-covariant property of $F_\theta$, such as a specific quantile (which will always exist) or its first moment (which might exist for some distribution families). By "property" I mean quite generally some "nice" real-valued function $t$ ("nice" will be explained momentarily) defined on this family of distributions and "location-covariant" means that for any $F$ in the family, $$t(F^{(\mu)}) = t(F) + \mu$$

where the translate $F^{(\mu)}$ is the distribution function given by

$$F^{(\mu)}(x) = F(x-\mu).$$

Consider any $\theta_0$ in the interior of $\Theta$. Parameterizations are assumed to have the property that such a $\theta_0$ will have a $p$-dimensional neighborhood $\mathcal{B}$ in which $t$ is differentiable with nonzero derivative throughout (that's what "nice" means). The Implicit Function Theorem then implies there is a coordinate system in this neighborhood in which the first coordinate is $t$ and the remaining $p-1$ coordinates are differentiable functions. Locally, at least, the distributions can be parameterized by $t$ and by the codimension-1 subset of $\mathcal{B}$ whose first coordinate is $t(\theta_0)$. That gives a new set of coordinates $\gamma=(\gamma_1,\ldots, \gamma_p)$ with $\gamma_1=t$.

We say that $t$ is a location parameter for the family. It has the property that if we fix the last $p-1$ coordinates in $\mathcal{B}$, then the distribution function can be written in the form

$$F_{(t, \gamma_2, \ldots,\gamma_p)}(x) = H_{(\gamma_2,\ldots, \gamma_p)}(x-t)$$

where $H$ (and therefore anything equivalent to it, such as its pdf if it has one) depends only on the last $p-1$ parameters. This is how you recognize a location parameter: the argument of the distribution function shows up in the formula only as the combination $x-t$.


Let's work an example. Suppose each $F_\theta$ in the interior of $\Theta = \{(\theta_1,\theta_2)\mid \theta_1 \ge 0\}$ is continuous and is given in terms of its pdf $f_\theta$ as

$$f_\theta(x) \propto \exp(-\theta_1 x^2 + \theta_2 x) \propto \exp\left(-\theta_1\left(x - \frac{\theta_2}{2\theta_1}\right)^2\right).\tag{1}$$

Is this a location family? If so, what is its location parameter?

In the first formula of $(1)$, there is no obvious location parameter: $f_\theta$ is not explicitly a function of $x-\theta_1$ or $x-\theta_2$.

The second formula of $(1)$ shows each pdf is symmetric around the value $$t(\theta) = \frac{\theta_2}{2\theta_1},$$ which therefore must be the median. Since the median is a location-covariant property, we can exploit this observation to construct a location parameter.

Consider $\theta_0 = (1,0)$ for instance. I chose this to make $t(\theta_0)=0$ have a simple value. The level set of $t$ passing through $\theta_0$ is given by

$$0 = t(1,0) = t(\theta_1, \theta_2) = \frac{\theta_2}{2\theta_1},$$

showing that (locally) it's the set where $\theta_2=0$ and we may parameterize it by $\theta_1$. Let us therefore change the parameterization from $\theta$ to

$$\gamma = (\gamma_1, \gamma_2) = (t(\theta_1, \theta_2), \theta_1) = \left( \frac{\theta_2}{2\theta_1}, \theta_1\right).$$

The base point $\theta_0$ corresponds to $\gamma_0 = (0, 1)$.

The inverse of this transformation from $\theta$ to $\gamma$ is

$$\theta = (\theta_1, \theta_2) = \left(\gamma_2, 2\gamma_1\gamma_2\right).$$

The new parameterization is therefore

$$G_\gamma(x) = F_{\left(\gamma_2, 2\gamma_1\gamma_2\right)}(x) \propto \exp\left(-\gamma_2(x - \gamma_1)^2\right).$$

Now it is perfectly obvious that $\gamma_1$ is a location parameter, because its sole effect is to shift $x$ in the formula.

By finding a candidate for a location parameter and demonstrating it acts as one, we have verified that this is a location family and we have found a location parameter for it. Incidentally, we have also identified this parameter with a location-covariant property: the median.

| cite | improve this answer | |
$\endgroup$
2
$\begingroup$

Consider a random variable $Z$ with any density function $f_Z(z)$. You can define a location–scale family with the transform $X=\phi Z +\theta$, & the new density function is given by

$$f_X(x;\theta,\phi) = \frac{1}{\phi}\cdot f_Z\left(\frac{x-\theta}{\phi}\right)$$

So any family of distributions whose density function can be written in this form is a location–scale family.

For example the family of uniform distributions on $(a,b)$ is defined by the density function

$$f_X(x;a,b) = \frac{1}{b-a}\cdot \boldsymbol{1}_{[a,b]}(x)$$

which can be re-written as

$$f_X(x;a,b) = \frac{1}{b-a}\cdot \boldsymbol{1}_{[0,1]}\left(\frac{x-a}{b-a}\right)$$

$$f_X(x;a,b) = \frac{1}{b-a}\cdot f_X\left(\frac{x-a}{b-a};0,1\right)$$

showing that $a$ is a location parameter & $b-a$ is a scale parameter.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Yes, that is the definition of location-scale family. But I do not see how (and if) is it possible to go from this and some given PDF to what are the underlying location and scale parameter (if they exist) $\endgroup$ – ira Mar 7 '17 at 15:08
  • 2
    $\begingroup$ Suppose you have a formula for $f(x;\theta)$ where $\theta$ is in a subset of $\mathbb{R}^p$ and for each $\theta$, the function $x\to f(x,\theta)$ is uniquely determined by $\theta$ and is a valid PDF. Is there some test you can apply to determine whether this formula defines a location-scale family? For instance, suppose for $r\gt 0, -\pi/2\lt\phi\lt\pi/2,$ and $q\gt 0$, $$f(x,(r,\phi,q))\propto\exp\left(-\frac{1}{2}\left(e^{-2r}x^{2qr}-2e^{-2r}x^{qr}\tan\phi+e^{-2r}(\sec^2\phi-1)\right)\right).$$Is this a location-scale family? If so, what are the location and scale parameters? $\endgroup$ – whuber Mar 7 '17 at 16:24
  • 1
    $\begingroup$ It's actually pretty simple. Suppose every distribution in the family has finite second moment, for instance. Then the location parameter associated with the pdf $f(x;\theta)$ must be its expectation (which is a function of $\theta$) and the scale parameter must be directly proportional to the square root of the variance (also a function of $\theta$). You just have to check that shifting back by the expectation and scaling inversely by the square root of the variance always gives you a pdf in the same family. $\endgroup$ – whuber Mar 7 '17 at 17:36
  • 2
    $\begingroup$ @Ira: $\frac{b-a}{2}$ would work just as well, with appropriate changes in the standard density: there's not a unique location-scale parametrization. $\endgroup$ – Scortchi - Reinstate Monica Mar 7 '17 at 23:08
  • 1
    $\begingroup$ Re the expectation. I apologize for leaving the impression that a location parameter might be unique. Indeed, since whenever $t$ is a location-covariant property then so is $t+C$ for any constant $C$. Therefore one could use (say) $b+10^{999}$ as a location parameter, too. $\endgroup$ – whuber Mar 7 '17 at 23:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.