How do location and scale parameters affect likelihoods?

Question

Many statistical procedures rely on analyses of a likelihood function. Frequently, some (or all) of the parameters in that function are "location" and "scale" parameters. (See Finding location and scale parameters from PDF for definitions and examples.)

How does this knowledge help simplify the analysis? In particular, if we are given a likelihood function that does not involve a location or scale parameter, how do we re-introduce these parameters into the likelihood?

Answer 1

It reduces the analysis to a (generally simpler) "standard" likelihood in which the location and scale parameters have been set to convenient values. There is a pitfall for the unwary, though: to obtain the general likelihood from the standard likelihood, you must introduce a factor that depends on the scale parameter.

---

A family $\mathcal F$ of probability distributions is a "location" family if, whenever the cumulative distribution function $F$ is in the family, then so is the function $x \to F(x-\mu).$ It is a "scale" family when $F\in \mathcal F$ implies the function $x\to F(x/\sigma)$ is in the family for any positive number $\sigma.$ (It is a location-scale family when both properties hold.)

We may take $\mu$ and $\sigma$ to be parameters of the family. Additional parameters, which I will just generically call $\theta,$ may be needed to specify members of the family uniquely.

In this setting, for each value of $\theta$ we may choose a "standard" distribution $F_\theta$ once and for all to represent it, making the general distribution in the family of the form

$$F_\theta(x;\mu,\sigma) = F_\theta\left(\frac{x-\mu}{\sigma}\right).$$

(This first shifts the location by $\mu$ and afterwards scales it by $\sigma.$ If we were to reverse the order--which is perfectly reasonable--the argument would instead be $x/\sigma - \mu.$)

Almost always, such families consist of absolutely continuous distributions. Let $f_\theta$ be the density associated with $F_\theta.$ Consequently, when $F$ is the distribution with parameters $(\mu,\sigma,\theta)$ its associated probability element is

$$\mathrm{d}F_\theta(x;\mu,\sigma) = \mathrm{d}F_\theta\left(\frac{x-\mu}{\sigma}\right) = \frac{1}{\sigma}f_\theta\left(\frac{x-\mu}{\sigma}\right)\,\mathrm{d}x.$$

Notice the appearance of the factor $1/\sigma.$ This subtlety is the reason for going through this analysis!

When a sample of (independent) data $\mathbf x = (x_1, \ldots, x_n)$ is observed from this family, its likelihood $\mathcal L$ is, by definition, its probability density. Since independence means probability densities multiply, we obtain

$$\mathcal{L}(\mu, \sigma, \theta; \mathbf x) = \prod_{i=1}^n \frac{1}{\sigma}f_\theta\left(\frac{x_i-\mu}{\sigma}\right)= \sigma^{-n}\prod_{i=1}^n f_\theta\left(\frac{x_i-\mu}{\sigma}\right) = \sigma^{-n} \mathcal{L}\left(0, 1, \theta; \frac{\mathbf x - \mu}{\sigma}\right) .$$

On the right, $\mathbf x - \mu$ is shorthand for $(x_1-\mu, \ldots, x_n-\mu).$

Here, then, is the result: suppose you have an expression for the likelihood for a standardized distribution where $\mu=0$ and $\sigma=1;$ say, some function $\Lambda(\theta;\mathbf x).$ (This plays the role of $\mathcal L$ on the right hand side just above.) Then the likelihood in full generality is

$$\mathcal{L}(\mu, \sigma, \theta; \mathbf x) = \sigma^{-n} \Lambda\left(\theta;\frac{\mathbf x - \mu}{\sigma}\right) .$$

That is, to obtain the general likelihood from the standard likelihood, (1) replace all $x_i$ by $(x_i-\mu)/\sigma$ and (2) multiply it by $\sigma^{-n}.$ Don't forget the second step!

Of course, in location-only families you can ignore $\sigma$ (set it to $1$) and in scale-only families you can ignore $\mu$ (set it to $0$).

Example

The Normal distribution family is a scale-location family (with no additional parameters). For the last century, most people have chosen the function

$$f(x) = \exp\left(-\frac{x^2}{2}\right)$$

(defined for all real numbers) for the standard density. Accordingly, using the rules of exponentiation, the likelihood of a dataset from the standard Normal distribution would be

$$\Lambda(\mathbf x) = \exp\left(-\sum_{i=1}^n \frac{x_i^2}{2}\right).$$

Therefore, the likelihood generally must be

$$\mathcal L(\mu,\sigma;\mathbf x) = \sigma^{-n} \exp\left(-\sum_{i=1}^n \frac{(x_i-\mu)^2}{2\sigma^2}\right).$$