Scale parameters -- How do they work, why are they sometimes dropped?

Question

I'm having difficulty wrapping my head around scale parameters. How exactly do they work? Why are they sometimes ignored? (in other words, when is it important to preserve them in a calculation?)

Thank you for any simple, lay explanations.

---

UPDATE:
What motivated the question was the Gamma distribution; in reading of it with relation to Poisson priors, I saw it described as simply

$\propto x^{\alpha-1} e ^{-x /\beta }$

However, the question is more of a general nature. While I get the concept of proportionality, I do not yet fully grasp when it is important for items to be equal versus simply proportional; or rather why so often in the literature I see items referred to as proportional with the scaling factor dropped.

Answer 1

A scale parameter merely establishes a unit of measurement, such as a foot, inch, angstrom, or parsec.

Without the scale parameter, we still know the shape and location of the distribution but we cannot label the axes, except for showing where the origin is. Here is a distribution (with the origin at its left) shown as a PDF.

We may establish its scale by showing one (or more) values along the x-axis. Here is a picture of the same shape with three separate scales shown:

Because a PDF uses area to show probability, it is unnecessary to label the vertical axis: we know the total area must be $1$. For instance, if the scale is set so the ticks are at $1$ and $2$, then I can see that this shape--which is close to triangular--must have a height of about $1$ in order for its total area to be $1$. (In fact, its peak is at $0.812$--close enough.)

Suppose the numbers on this scale are feet and we re-express the values in inches. That does nothing other than relabel the x-axis: the ticks now are labeled $12$ and $24$ inches, respectively, as shown in the middle row of labels. Obviously relabeling does not change the shape (or the origin). The height must change, though: since numerically the base is now $12$ times greater, the height must shrink by the same factor. We deduce the maximum value equals $0.812/12$--but again, there's no need to show this, because we know the total area is unity. If we have to, we draw a y-axis and we merely relabel it.

In general, when the scale is $\sigma \gt 0$ times greater than the original, we label tick $1$ with $\sigma$, tick $2$ with $2\sigma$, and so on. The axis labels may change, but the shape is constant.

In many statistical problems we do not want our conclusions to depend on the units we use to express the measurements. In most cases the units are arbitrary and we don't want the conclusions to be arbitrary! The only exceptions occur when the units are unique; the best example is where the x-axis is a count (but such nice continuous PDFs do not arise in that situation: they are usually shown as bar graphs instead). Therefore, for many purposes we may ignore the scale throughout all calculations and freely introduce it back at the end. Although I'm not sure, I think this might address the concerns behind the question.

---

Let's end with some remarks about mathematical notation. If the equation of the PDF for the initial labeling (with ticks at $1$ and $2$) is $f(x)$, then the relabeling turns $x$ into $x \sigma$, as is evident from the second figure. The height at $y = x \sigma$ is found by first dividing by $\sigma$ to find the original expression for $x$ and then applying $f$--but don't forget to divide by $\sigma$ to keep the total area to $1$! Therefore, the same distribution using units of $\sigma$--that is, a "scale factor" of $\sigma$--has the PDF

$$\frac{1}{\sigma} f(y/\sigma).$$

An excellent way to remember this--from the right point of view it's perfectly rigorous--is always to write your PDF explicitly as a product of a length and a height. The height is $f(x)$ and the length is the differential $dx$, so the proper way to write the PDF is $f(x)dx$. Now when we change $x$ to $x/\sigma$ the PDF becomes

$$f(y) dy = f(x/\sigma) d(x/\sigma) = \frac{1}{\sigma} f(x/\sigma) dx,$$

exactly as it should. (Differentials follow the rules of differentiation: for any function $g$, $dg(x) = g'(x)dx$. That's how I determined that $d(x/\sigma) = (dx)/\sigma$.) In short, the $dx$ is a formal reminder to adjust the y-axis in order to keep the total area to unity.

For example, a Gamma$(3)$ distribution has a PDF proportional to $e^{-3x} x^2 dx$, by definition. (The first figure is a portrait of this distribution.) Often we do not need to know the constant of proportionality because it's there only to make sure the total area is unity. To change the scale to $\sigma$, the distribution would now take the form

$$e^{-3x/\sigma} (x/\sigma)^2 d(x/\sigma) = \frac{1}{\sigma^3} x^2 e^{-3x/\sigma} dx.$$

The constant of proportionality goes along for the ride: it multiplies both expressions. In this fashion we can obtain a great deal of information while needing to remember very little: all that is required is knowledge of the basic form of the distribution for one particular nice scale.

Answer 2

I would say that the importance of proportionality and equality depend entirely on what you're trying to say about the distribution or data. Let's think about some standard properties in statistics that people are interested in:

• Mean: Scale parameters alter the mean of most distributions, though many common distributions have separate location parameters that control the mean (e.g., the normal distribution). So then, I wouldn't rely on this property in general. It's pretty easy to think up distributions where the mean and variance are both tightly coupled and would both be changed by any scaling factor (e.g., uniform distribution, gamma distribution).
• Variance: Almost by definition if you change the scaling parameter, you are changing the variance. There are counter-examples to this, but they're basically edge cases (zero variance, infinite variance, undefined variance).
• Shape: The overall shape of the distribution shouldn't change because you change a scaling parameter. You're mainly just stretching it or shrinking it with respect to the CDF.

So then, if you only care about the general shape or family of distributions, sometimes you don't care about the scaling parameter. For example, if you were trying to identify if data came from a normal distribution versus an exponential distribution, you might not care what their exact scaling parameters were.

If you care at all about variances being equal, you care about the scaling parameter. This covers a greater range of cases than you'd think. Different variants of statistical tests need to be used for unequal variances than equal variances. Or, in other words, even if you only care about testing the difference in means for normal distributions... scaling parameters are still important. In the general case, if you care about means, you care about scaling. However, for quite a few popular distributions the two are independent.

So, as others have stated, in general it depends on the distribution you're interested in. Sometimes nearly everything about the distribution (except its general shape), depends on a single parameter that controls the scaling. The standard exponential distribution is a good example of this.