Are there always two degrees of freedom in any probability distribution?

Take any random variable $$X$$ that follows some distribution $$P(X)$$. I was looking at this Wikipedia page and I'm trying to get some intuition for why we choose to define standard moments the way we do. Specifically, we define

\begin{align*} \mu_k &= \int dX P(X) (X - \mu)^k \\ \sigma_k &= \sqrt{\mu_2}^k\\ &=\left(\int dX P(X) (X - \mu)^2\right)^{k/2} \end{align*}

From this, we get the standard moments which are defined as $$\tilde{\mu_i} = \mu_i/\sigma_i$$. These standard moments can be used to make comparisons across distributions.

I'm not sure if my interpretation of what is happening is correct. One can always shift the points such that the mean is 0. This is, we are choosing to look at $$X' = (X - \mu)$$ instead of $$X$$. Next, we scale the values of $$X'$$ such that the variance is unity.

It seems like this is the end of the line - at least the standard moments only seek to ensure that the mean is 0 and variance is 1. There is no other global transformation we can do that preserves the inherent qualities of the distribution. Moreover, when we are comparing unrelated quantities (e.g. the heights of people in a class and the weights say) the first two transformations correspond to unphysical things (namely, the zero of our scale and the units we choose to measure in).

Therefore, the standard moments are simply eliminating the degrees of freedom we have when characterizing arbitrary data. Moreover, there are no additional degrees of freedom i.e. no further transformations we can do to set the higher moments to zero without changing the mean and the variance. Is this correct?

• One way to view the implicit standardization that is going on is to interpret it as a change in units of measurement. Another way is to view it as an affine transformation: by definition, the shape of a distribution is whatever properties are invariant under affine transformations of the variable. Thus, standard moments are properties of the shape but the raw moments are not. There are plenty of transformations available that will change the higher moments--including zeroing out all odd-order moments--but they are necessarily nonlinear. – whuber Jan 11 at 21:45
• @user1936752 What you're effectively doing there is taking a fully specified distribution and then defining a location-scale family in terms of it. That will always leave you with a location parameter and a scale parameter. – Glen_b Jan 12 at 3:33

1 Answer

There are two degrees of freedom in an affine transformation on one dimension (that is, a linear transformation plus an additive constant, $$y=a+bx$$). Therefore, we can tune an affine transformation to change two moments to given values but we can't adjust more than two moments because we would need more than two degrees of freedom.

However, we can use non linear transformations to adjust more than two moments, as whuber pointed in his comment.

In fact, there is a transformation from any continuous probability distribution to any probability distribution. Given two probability distributions $$X$$ and $$Y$$ ($$X$$ continuous) with distribution functions $$F_x$$ and $$F_y$$, we can transform a random variable $$A$$ following distribution $$X$$ into a random variable $$B$$ following $$Y$$ distribution just by $$B=F_y^{-1}(F_x(A))$$.

Therefore, there is a transformation of any continuous distribution that change its moments to the moments of any other possible distribution. Of course, that transformation is usually non linear and has more than two degrees of freedom.