5
$\begingroup$

I'm having difficulty wrapping my head around scale parameters. How exactly do they work? Why are they sometimes ignored? (in other words, when is it important to preserve them in a calculation?)

Thank you for any simple, lay explanations.


UPDATE:
What motivated the question was the Gamma distribution; in reading of it with relation to Poisson priors, I saw it described as simply

$\propto x^{\alpha-1} e ^{-x /\beta }$

However, the question is more of a general nature. While I get the concept of proportionality, I do not yet fully grasp when it is important for items to be equal versus simply proportional; or rather why so often in the literature I see items referred to as proportional with the scaling factor dropped.

$\endgroup$
  • 3
    $\begingroup$ It would be nice to have a specific example you have seen or been confused about. That might help guide people better in their answers. As it stands, the question is a little on the vague side. Cheers. $\endgroup$ – cardinal Feb 10 '13 at 21:07
  • $\begingroup$ I agree w/ @cardinal. Are you referring to the scale parameter for a specific distribution (which?), or from something else? Examples of the cases that confuse you (where people ignore & don't ignore them), are going to necessary to get an answer that will be useful for you. In its present state, I'm not sure this Q is answerable. $\endgroup$ – gung - Reinstate Monica Feb 10 '13 at 21:44
9
$\begingroup$

A scale parameter merely establishes a unit of measurement, such as a foot, inch, angstrom, or parsec.

Without the scale parameter, we still know the shape and location of the distribution but we cannot label the axes, except for showing where the origin is. Here is a distribution (with the origin at its left) shown as a PDF.

Distribution, unlabeled

We may establish its scale by showing one (or more) values along the x-axis. Here is a picture of the same shape with three separate scales shown:

Distribution, labeled

Because a PDF uses area to show probability, it is unnecessary to label the vertical axis: we know the total area must be $1$. For instance, if the scale is set so the ticks are at $1$ and $2$, then I can see that this shape--which is close to triangular--must have a height of about $1$ in order for its total area to be $1$. (In fact, its peak is at $0.812$--close enough.)

Suppose the numbers on this scale are feet and we re-express the values in inches. That does nothing other than relabel the x-axis: the ticks now are labeled $12$ and $24$ inches, respectively, as shown in the middle row of labels. Obviously relabeling does not change the shape (or the origin). The height must change, though: since numerically the base is now $12$ times greater, the height must shrink by the same factor. We deduce the maximum value equals $0.812/12$--but again, there's no need to show this, because we know the total area is unity. If we have to, we draw a y-axis and we merely relabel it.

In general, when the scale is $\sigma \gt 0$ times greater than the original, we label tick $1$ with $\sigma$, tick $2$ with $2\sigma$, and so on. The axis labels may change, but the shape is constant.

In many statistical problems we do not want our conclusions to depend on the units we use to express the measurements. In most cases the units are arbitrary and we don't want the conclusions to be arbitrary! The only exceptions occur when the units are unique; the best example is where the x-axis is a count (but such nice continuous PDFs do not arise in that situation: they are usually shown as bar graphs instead). Therefore, for many purposes we may ignore the scale throughout all calculations and freely introduce it back at the end. Although I'm not sure, I think this might address the concerns behind the question.


Let's end with some remarks about mathematical notation. If the equation of the PDF for the initial labeling (with ticks at $1$ and $2$) is $f(x)$, then the relabeling turns $x$ into $x \sigma$, as is evident from the second figure. The height at $y = x \sigma$ is found by first dividing by $\sigma$ to find the original expression for $x$ and then applying $f$--but don't forget to divide by $\sigma$ to keep the total area to $1$! Therefore, the same distribution using units of $\sigma$--that is, a "scale factor" of $\sigma$--has the PDF

$$\frac{1}{\sigma} f(y/\sigma).$$

An excellent way to remember this--from the right point of view it's perfectly rigorous--is always to write your PDF explicitly as a product of a length and a height. The height is $f(x)$ and the length is the differential $dx$, so the proper way to write the PDF is $f(x)dx$. Now when we change $x$ to $x/\sigma$ the PDF becomes

$$f(y) dy = f(x/\sigma) d(x/\sigma) = \frac{1}{\sigma} f(x/\sigma) dx,$$

exactly as it should. (Differentials follow the rules of differentiation: for any function $g$, $dg(x) = g'(x)dx$. That's how I determined that $d(x/\sigma) = (dx)/\sigma$.) In short, the $dx$ is a formal reminder to adjust the y-axis in order to keep the total area to unity.

For example, a Gamma$(3)$ distribution has a PDF proportional to $e^{-3x} x^2 dx$, by definition. (The first figure is a portrait of this distribution.) Often we do not need to know the constant of proportionality because it's there only to make sure the total area is unity. To change the scale to $\sigma$, the distribution would now take the form

$$e^{-3x/\sigma} (x/\sigma)^2 d(x/\sigma) = \frac{1}{\sigma^3} x^2 e^{-3x/\sigma} dx.$$

The constant of proportionality goes along for the ride: it multiplies both expressions. In this fashion we can obtain a great deal of information while needing to remember very little: all that is required is knowledge of the basic form of the distribution for one particular nice scale.

$\endgroup$
  • 2
    $\begingroup$ Big thumbs up on the graphs. I think they help ground it very well. $\endgroup$ – Namey Feb 13 '13 at 3:39
2
$\begingroup$

I would say that the importance of proportionality and equality depend entirely on what you're trying to say about the distribution or data. Let's think about some standard properties in statistics that people are interested in:

  • Mean: Scale parameters alter the mean of most distributions, though many common distributions have separate location parameters that control the mean (e.g., the normal distribution). So then, I wouldn't rely on this property in general. It's pretty easy to think up distributions where the mean and variance are both tightly coupled and would both be changed by any scaling factor (e.g., uniform distribution, gamma distribution).
  • Variance: Almost by definition if you change the scaling parameter, you are changing the variance. There are counter-examples to this, but they're basically edge cases (zero variance, infinite variance, undefined variance).
  • Shape: The overall shape of the distribution shouldn't change because you change a scaling parameter. You're mainly just stretching it or shrinking it with respect to the CDF.

So then, if you only care about the general shape or family of distributions, sometimes you don't care about the scaling parameter. For example, if you were trying to identify if data came from a normal distribution versus an exponential distribution, you might not care what their exact scaling parameters were.

If you care at all about variances being equal, you care about the scaling parameter. This covers a greater range of cases than you'd think. Different variants of statistical tests need to be used for unequal variances than equal variances. Or, in other words, even if you only care about testing the difference in means for normal distributions... scaling parameters are still important. In the general case, if you care about means, you care about scaling. However, for quite a few popular distributions the two are independent.

So, as others have stated, in general it depends on the distribution you're interested in. Sometimes nearly everything about the distribution (except its general shape), depends on a single parameter that controls the scaling. The standard exponential distribution is a good example of this.

$\endgroup$
  • $\begingroup$ Re the first two bullets. The variance, if it exists, is proportional to the square of the scale. There are no exceptions or "edge cases." Scale parameters do alter the means of many distributions, unless those distributions are carefully parameterized. See the Wikipedia articles for the gamma distribution, which includes your "basic exponential distribution" as a special case, or Weibull distribution, for instance, and notice how the expectation is directly proportional to the scale. $\endgroup$ – whuber Feb 12 '13 at 3:37
  • $\begingroup$ Agreed on all points. With that said, I would say that distributions with variance that is 0, infinity, or undefined to be edge cases. I would also definitely agree that changing scaling parameters not altering the mean is a special case. However, probably because it's convenient to understand, I'd say that distributions parameterized to separate the mean and scale aren't particularly uncommon. I'll try to adjust the wording to clarify that. $\endgroup$ – Namey Feb 13 '13 at 3:22
  • $\begingroup$ Also, I should state that I'm not sure if you can even have a scale parameter for a zero variance distribution (e.g., deterministic value). Basically, the parameter would do nothing at that point. $\endgroup$ – Namey Feb 13 '13 at 3:33
  • $\begingroup$ Also, as a final note the qualifier of basic/standard for "exponential distribution" is meant to distinguish it from an exponential distribution with a location parameter (which is occasionally presented in some contexts). $\endgroup$ – Namey Feb 13 '13 at 3:37
  • $\begingroup$ (+1 for the edits.) A distribution with zero variance has essential support at a point; its scale can be taken to be zero. As my answer shows, it does not matter if a distribution has an infinite variance: a scale can still be defined by reference to a standard member of the family. For instance, we might take the standard Cauchy distribution to have PDF equal to $dx/[\pi(1+x^2)]$, whence the scale of a distribution with PDF $dx/[\sigma\pi(1 + (x/\sigma)^2)]$ is equal to $\sigma$. The scale is well defined, despite the infinite variance of the distribution. $\endgroup$ – whuber Feb 13 '13 at 5:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.