Is it possible that two Random Variables from the same distribution family have the same expectation and variance, but different higher moments? - Cross Validated most recent 30 from stats.stackexchange.com 2019-09-15T16:21:26Z https://stats.stackexchange.com/feeds/question/320175 https://creativecommons.org/licenses/by-sa/4.0/rdf https://stats.stackexchange.com/q/320175 11 Is it possible that two Random Variables from the same distribution family have the same expectation and variance, but different higher moments? Hard Core https://stats.stackexchange.com/users/68313 2017-12-23T12:15:24Z 2019-09-13T12:54:19Z <p>I was thinking about the meaning of location-scale family. My understanding is that for every $X$ member of a location scale family with parameters $a$ location and $b$ scale, then the distribution of $Z =(X-a)/b$ does not depend of any parameters and it's the same for every $X$ belonging to that family.</p> <p>So my question is could you provide an example where two random from the same distribution family are standardized but that does not results in a Random Variable with the same distribution?</p> <p>Say $X$ and $Y$ come from the same distribution family (where with family I mean for example both Normal or both Gamma and so on ..). Define:</p> <p>$Z_1 = \dfrac{X-\mu}{\sigma}$</p> <p>$Z_2 = \dfrac{Y-\mu}{\sigma}$</p> <p>we know that both $Z_1$ and $Z_2$ have the same expectation and variance, $\mu_Z =0, \sigma^2_Z =1$.</p> <p>But can they have different higher moments?</p> <p>My attempt to answer this question is that if the distribution of $X$ and $Y$ depends on more than 2 parameters than it could be. And I am thinking about the generalized $t-student$ that has 3 parameters. </p> <p>But if the number of parameters is $\le2$ and $X$ and $Y$ come from the same distribution family with the same expectation and variance, then does it mean that $Z_1$ and $Z_2$ has the same distribution (higher moments)?</p> https://stats.stackexchange.com/questions/320175/-/320202#320202 14 Answer by Xi'an for Is it possible that two Random Variables from the same distribution family have the same expectation and variance, but different higher moments? Xi'an https://stats.stackexchange.com/users/7224 2017-12-23T17:37:20Z 2017-12-23T22:04:57Z <p>There is an infinite number of distributions with mean zero and variance one, hence take $\epsilon_1$ distributed from one of these distributions, say the $\mathcal{N}(0,1)$, and $\epsilon_2$ from another of these distributions, say the Student's $t$ with 54 degrees of freedom rescaled by $\sqrt\frac{1}{3}$ so that its variance is one, then $$X=\mu+\sigma\epsilon_1\qquad\text{and}\qquad Y=\mu+\sigma\epsilon_2$$ enjoy the properties you mention. The "number" of parameters is irrelevant to the property. </p> <p>Obviously, if you set further rules to the definition of this family, like stating for instance that there exists a fixed density $f$ such that the density of $X$ is $$\frac{1}{\sigma^d} f(\{x-\mu\}/\sigma)$$ you may end up with a single possible distribution.</p> https://stats.stackexchange.com/questions/320175/-/320209#320209 17 Answer by kjetil b halvorsen for Is it possible that two Random Variables from the same distribution family have the same expectation and variance, but different higher moments? kjetil b halvorsen https://stats.stackexchange.com/users/11887 2017-12-23T20:07:54Z 2018-10-28T05:27:39Z <p>If you want an example which is an "officially named parameterized distribution family, you can look into the generalized gamma distribution, <a href="https://en.wikipedia.org/wiki/Generalized_gamma_distribution" rel="nofollow noreferrer">https://en.wikipedia.org/wiki/Generalized_gamma_distribution</a>. This distribution family has three parameters, so you can fix mean and variance and still have freedom to vary higher moments. From the wiki page, the algebra do not look inviting, I would rather to do it numerically. For statistical applications, search this site for gamlss, which is an extension of gam (generalized additive models, in itself a generalization of glm's) which have parameters for "location, scale and shape". </p> <p>Another example is the <span class="math-container">$t$</span>-distributions, extended to be a location-scale family. Then the third parameter will be the degrees of freedom, which will wary the shape for a fixed location and scale. </p> https://stats.stackexchange.com/questions/320175/-/320218#320218 7 Answer by Carl for Is it possible that two Random Variables from the same distribution family have the same expectation and variance, but different higher moments? Carl https://stats.stackexchange.com/users/99274 2017-12-23T22:24:46Z 2019-09-13T12:54:19Z <p>There is apparently some confusion as to what a family of distributions is and how to count free parameters versus free plus fixed (assigned) parameters. Those questions are an aside that is unrelated to the intent of the OP, and of this answer. <strong>I do not use the word <em>family</em> herein because it is confusing.</strong> For example, a <em>family</em> according to <a href="http://www.itl.nist.gov/div898/handbook/eda/section3/eda363.htm" rel="nofollow noreferrer">one source</a> is the result of varying the shape parameter. @whuber states that <em>A "parameterization" of a family is a continuous map from a subset of ℝ<span class="math-container">$^n$</span>, with its usual topology, into the space of distributions, whose image is that family.</em> <strong>I will use the word <em>form</em> which covers both the intended usage of the word <em>family</em> and <em>parameter</em> identification and counting.</strong> For example the formula <span class="math-container">$x^2-2x+4$</span> has the <em>form</em> of a quadratic formula, i.e., <span class="math-container">$a_2x^2+a_1x+a_0$</span> and if <span class="math-container">$a_1=0$</span> the formula is still of quadratic form. However, when <span class="math-container">$a_2=0$</span> the formula is linear and the form is no longer complete enough to contain a quadratic shape term. Those who wish to use the word family in a proper statistical context are encouraged to <a href="https://stats.stackexchange.com/q/320746/99274">contribute to that separate question</a>.</p> <p><strong>Let us answer the question "Can they have different higher moments?".</strong> There are many such examples. We note in passing that the question appears to be about symmetric PDFs, which are the ones that tend to have location and scale in the simple bi-parameter case. The logic: Suppose there are two density functions with different shapes having two identical (location, scale) parameters. Then there is either a shape parameter that adjusts shape, or, the density functions have no common shape parameter and are thus density functions of no common form. </p> <p>Here, is an example of how the shape parameter figures into it. The <a href="https://en.wikipedia.org/wiki/Generalized_normal_distribution" rel="nofollow noreferrer">generalized error density function</a> and <a href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.542.879&amp;rep=rep1&amp;type=pdf" rel="nofollow noreferrer">here</a>, is an answer that appears to have a freely selectable kurtosis.</p> <p><a href="https://i.stack.imgur.com/Mzjce.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/Mzjce.png" alt="enter image description here"></a></p> <p>By Skbkekas - Own work, CC BY-SA 3.0, <a href="https://commons.wikimedia.org/w/index.php?curid=6057753" rel="nofollow noreferrer">https://commons.wikimedia.org/w/index.php?curid=6057753</a></p> <p>The PDF (A.K.A. "probability" density function, note that the word "probability" is superfluous) is <span class="math-container">$$\dfrac{\beta}{2\alpha\Gamma\Big(\dfrac{1}{\beta}\Big)} \; e^{-\Big(\dfrac{|x-\mu|}{\alpha}\Big)^\beta}$$</span></p> <p>The mean and location is <span class="math-container">$\mu$</span>, the scale is <span class="math-container">$\alpha$</span>, and <span class="math-container">$\beta$</span> is the shape. Note that it is easier to present symmetric PDFs, because those PDFs often have location and scale as the simplest two parameter cases whereas asymmetric PDFs, like the <a href="https://en.wikipedia.org/wiki/Gamma_distribution" rel="nofollow noreferrer">gamma PDF</a>, tend to have shape and scale as their simplest case parameters. Continuing with the error density function, the variance is <span class="math-container">$\dfrac{\alpha^2\Gamma\Big(\dfrac{3}{\beta}\Big)}{\Gamma\Big(\dfrac{1}{\beta}\Big)}$</span>, the skewness is <span class="math-container">$0$</span>, and the kurtosis is <span class="math-container">$\dfrac{\Gamma\Big(\dfrac{5}{\beta}\Big)\Gamma\Big(\dfrac{1}{\beta}\Big)}{\Gamma\Big(\dfrac{3}{\beta}\Big)^2}-3$</span>. Thus, if we set the variance to be 1, then we assign the value of <span class="math-container">$\alpha$</span> from <span class="math-container">$\alpha ^2=\dfrac{\Gamma \left(\dfrac{1}{\beta }\right)}{\Gamma \left(\dfrac{3}{\beta }\right)}$</span> while varying <span class="math-container">$\beta&gt;0$</span>, so that the kurtosis is selectable in the range from <span class="math-container">$-0.601114$</span> to <span class="math-container">$\infty$</span>.</p> <p>That is, if we want to vary higher order moments, and if we want to maintain a mean of zero and a variance of 1, we need to vary the shape. This implies three parameters, which in general are 1) the mean or otherwise the appropriate measure of location, 2) the scale to adjust the variance or other measure of variability, and 3) the shape. IT TAKES at least THREE PARAMETERS TO DO IT.</p> <p>Note that if we make the substitutions <span class="math-container">$\beta=2$</span>, <span class="math-container">$\alpha=\sqrt{2}\sigma$</span> in the PDF above, we obtain <span class="math-container">$$\frac{e^{-\frac{(x-\mu )^2}{2 \sigma ^2}}}{\sqrt{2 \pi } \sigma }\;,$$</span></p> <p>which is a normal distribution's density function. Thus, the <a href="https://en.wikipedia.org/wiki/Generalized_normal_distribution" rel="nofollow noreferrer">generalized error density function</a> is a generalization of the normal distribution's density function. There are many ways to generalize a normal distribution's density function. Another example, but with the normal distribution's density function only as a limiting value, and not with mid-range substitution values like the generalized error density function, is the Student's<span class="math-container">$-t$</span> 's density function. Using the Student's<span class="math-container">$-t$</span> density function, we would have a rather more restricted selection of kurtosis, and <span class="math-container">$\textit{df}\geq2$</span> is the shape parameter because the second moment does not exist for <span class="math-container">$\textit{df}&lt;2$</span>. Moreover, <em>df</em> is not actually limited to positive integer values, it is in general real <span class="math-container">$\geq1$</span>. The Student's<span class="math-container">$-t$</span> only becomes normal in the limit as <span class="math-container">$\textit{df}\rightarrow\infty$</span>, which is why I did not choose it as an example. It is neither a good example nor is it a counter example, and in this I disagree with @Xi'an and @whuber. </p> <p>Let me explain this further. One can choose two of many arbitrary density functions of two parameters to have, as an example, a mean of zero and a variance of one. However, they will not all be of the same form. <strong>The question however, relates to density functions of the SAME form, not different forms.</strong> The claim has been made that which density functions have the same form is an arbitrary assignment as this is a matter of definition, and in that my opinion differs. I do not agree that this is arbitrary because one can either make a substitution to convert one density function to be another, or one cannot. In the first case, the density functions are similar, and if by substitution we can show that the density functions are not equivalent, then those density functions are of different form.</p> <p>Thus, using the example of the Student's<span class="math-container">$-t$</span> PDF, the choices are to either consider it to be a generalization of a normal PDF, in which case a normal PDF has a permissible form for a Student's<span class="math-container">$-t$</span>'s PDF, or not, in which case the Student's<span class="math-container">$-t$</span> 's PDF is of a different form from the normal PDF <strong>and thus is irrelevant to the question posed</strong>. </p> <p>We can argue this many ways. My opinion is that a normal PDF is a sub-selected form of a Student's<span class="math-container">$-t$</span> 's PDF, but that a normal PDF is not a sub-selection of a gamma PDF even though a limiting value of a gamma PDF can be shown to be a normal PDF, and, my reason for this is that in the normal/Student'<span class="math-container">$-t$</span> case, the support is the same, but in the normal/gamma case the support is infinite versus semi-infinite, which is the required incompatibility.</p> https://stats.stackexchange.com/questions/320175/-/320234#320234 6 Answer by yyzz for Is it possible that two Random Variables from the same distribution family have the same expectation and variance, but different higher moments? yyzz https://stats.stackexchange.com/users/189523 2017-12-24T03:34:46Z 2017-12-25T23:27:10Z <p>I think you are asking whether two random variables coming from the same location-scale family can have the same mean and variance, but at least one different higher moment. The answer is no.</p> <p><em>Proof</em>: Let $X_1$ and $X_2$ be two such random variables. Since $X_1$ and $X_2$ are in the same location-scale family, there exist a random variable $X$ and real numbers $a_1&gt;0, a_2&gt;0, b_1, b_2$ such that $X_1 \stackrel{d}{=} a_1 X + b_1$ and $X_2 \stackrel{d}{=} a_2 X + b_2$. Since $X_1$ and $X_2$ have the same mean and variance, we have:</p> <ol> <li>$E[X_1] = E[X_2] \implies a_1 E[X] + b_1 = a_2 E[X] + b_2$.</li> <li>$\operatorname{Var}[X_1] = \operatorname{Var}[X_2] \implies a_1^2 \operatorname{Var}[X] = a_2^2 \operatorname{Var}[X]$.</li> </ol> <p>If $\operatorname{Var}[X] = 0$, then $X_1=E[X_1]=X_2=E[X_2]$ with probability $1$, and hence the higher moments of $X_1$ and $X_2$ are all equal. So we may assume that $\operatorname{Var}[X] \neq 0$. Using this, (2) implies that $|a_1|=|a_2|$. Since $a_1&gt;0$ and $a_2&gt;0$, we have in fact that $a_1=a_2$. In turn, (1) above now implies that $b_1=b_2$. We therefore have that: $$E[X_1^k] = E[(a_1X+b_1)^k] = E[(a_2X+b_2)^k] = E[X_2^k],$$ for any $k$, i.e., all moments of $X_1$ and $X_2$ are all equal.</p> https://stats.stackexchange.com/questions/320175/-/364137#364137 1 Answer by Martijn Weterings for Is it possible that two Random Variables from the same distribution family have the same expectation and variance, but different higher moments? Martijn Weterings https://stats.stackexchange.com/users/164061 2018-08-27T11:34:06Z 2018-11-17T22:18:37Z <p>Since the question can be interpreted in multipe ways I will split this answer into two parts. </p> <ul> <li>A: distribution families. </li> <li>B: location-scale distribution families.</li> </ul> <p>The problem with case A can be easily answered/demonstrated by many families with a shape parameter. </p> <p>The problem with case B is more difficult since one and a half parameters seem to be sufficient to specify location and scale (location in <span class="math-container">$\mathbb{R}$</span> and scale in <span class="math-container">$\mathbb{R_{&gt;0}}$</span>), and the problem becomes whether two parameters can be used to encode (multiple) shapes in addition as well. This is not so trivial. We can easily come up with specific two parameter location scale families and demonstrate that you do not have different shapes, but it does not proof that this is a fixed rule for any two parameter location scale family. </p> <h2>A: Can two different distributions from the same 2 parameter distribution family have the same mean and variance?</h2> <p>The answer is <strong>yes</strong> and it can already be shown using one of the explicitly mentioned examples: the normalized Gamma distribution </p> <h3>Family of normalized gamma distributions</h3> <p>Let <span class="math-container">$Z = \frac{X-\mu}{\sigma}$</span> with <span class="math-container">$X$</span> a Gamma distributed variable. The (cumulative) distribution of <span class="math-container">$Z$</span> is as below: </p> <p><span class="math-container">$$F_Z(z;k) = \begin{cases} 0 &amp; \quad \text{if} &amp; z &lt; -\sqrt{k}\\ \frac{1}{\Gamma(k)} \gamma(k, {z\sqrt{k}+k}) &amp; \quad \text{if} &amp; z \geq -\sqrt{k} \end{cases}$$</span></p> <p>where <span class="math-container">$\gamma$</span> is the incomplete gamma function.</p> <p>So here it is clearly the case that different <span class="math-container">$Z_1$</span> and <span class="math-container">$Z_2$</span> (distributions from the family of normalized gamma distributions) can have same mean and variance (namely <span class="math-container">$\mu=0$</span> and <span class="math-container">$\sigma=1$</span>) but differ based on the parameter <span class="math-container">$k$</span> (often denoted 'shape' parameter). This is closely linked to the fact that the family of gamma distributions is not a location-scale family.</p> <h2>B: Can two different distributions from the same 2 parameter <em>location-scale</em> distribution family have the same mean and variance?</h2> <p>I believe that the answer is <strong>no</strong> if we consider only smooth families (smooth: a small change in the parameters will result in a small change of the distribution/function/curve). But that answer is not so trivial and when we would use more general (non-smooth) families then we can say <strong>yes</strong>, although these families only exist in theory and have no practical relevance.</p> <h3>Generating a location-scale family from a single distribution by translation and scaling</h3> <p>From any particular <em>single</em> distribution we can generate a location-scale family by translation and scaling. If <span class="math-container">$f(x)$</span> is the probability density function of the single distribution, then the probability density function for a member of the family will be </p> <p><span class="math-container">$$f(x;\mu,\sigma) = \frac{1}{\sigma}f(\frac{x-\mu}{\sigma})$$</span></p> <p>For a location-scale family that can be generated in such way we have:</p> <ul> <li>for any two members <span class="math-container">$f(x;\mu_1,\sigma_1)$</span> and <span class="math-container">$f(x;\mu_2,\sigma_2)$</span> if their means and variances are equal, then <span class="math-container">$f(x;\mu_1,\sigma_1) = f(x;\mu_2,\sigma_2)$</span></li> </ul> <h3>Can for all two parameter location-scale families their member distributions be generated from a single member distribution by translation and scaling?</h3> <p>So translation and scaling can convert a single distribution into a location-scale family. The question is whether the reverse is true and whether every two parameter location-scale family (where the parameters <span class="math-container">$\theta_1$</span> and <span class="math-container">$\theta_2$</span> do not necessarily need to coincide with the location <span class="math-container">$\mu$</span> and scale <span class="math-container">$\sigma$</span>) can be described by a translation and scaling of a <em>single</em> member from that family. </p> <p>For particular two parameter location-scale families like the family of normal distributions it is not too difficult to show that they can be generated according to the process above (scaling and translating of single example member). </p> <p>One may wonder whether it is possible for <em>every</em> two parameter location-scale family to be generated out of a single member by translation and scaling. Or a conflicting statement: "Can a two parameter location-scale family contain two different member distributions with the same mean and variance?", for which it would be <em>necessary</em> that the family is a union of <em>multiple</em> subfamilies that are each generated by translation and scaling.</p> <h3>Case 1: Family of generalized Students' t-distributions, parameterized by two variables</h3> <p>A contrived example occurs when we make some mapping from <span class="math-container">$R^2$</span> into <span class="math-container">$R^3$</span> (<a href="https://math.stackexchange.com/questions/290019/cardinality-of-mathbbr-and-mathbbr2">cardinality-of-mathbbr-and-mathbbr2</a>) which allows the freedom to use two parameters <span class="math-container">$\theta_1$</span> and <span class="math-container">$\theta_2$</span> to describe a union of multiple subfamilies that are generated by translation and scaling.</p> <p>Let's use the (three parameter) generalized Student's t-distribution: </p> <p><span class="math-container">$f(x;\nu,\mu,\sigma) = \frac{\Gamma \left( \frac{\nu + 1}{2} \right) }{\Gamma \left( \frac{\nu}{2} \right) \sqrt{\pi\nu}\sigma} \left(1 + \frac{1}{\nu} \left( \frac{x-\mu}{\sigma} \right)^2 \right)^{-\frac{\nu+1}{2}}$</span></p> <p>with the three parameters changed as following <span class="math-container">$$\begin{array}{rcl} \mu &amp;=&amp; \tan (\theta_1)\\ \sigma &amp;=&amp; \theta_2\\ \nu &amp;=&amp; \lfloor 0.5+\theta_1/\pi \rfloor \end{array}$$</span></p> <p>then we have </p> <p><span class="math-container">$f(x;\theta_1,\theta_2) = \frac{\Gamma \left( \frac{\lfloor 0.5+\theta_1/\pi \rfloor + 1}{2} \right) }{\Gamma \left( \frac{\lfloor 0.5+\theta_1/\pi \rfloor}{2} \right) \sqrt{\pi\lfloor 0.5+\theta_1/\pi \rfloor}\theta_2} \left(1 + \frac{1}{\lfloor 0.5+\theta_1/\pi \rfloor} \left( \frac{x-\tan(\theta_1)}{\theta_2} \right)^2 \right)^{-\frac{\lfloor 0.5+\theta_1/\pi \rfloor+1}{2}}$</span></p> <p>which may be considered a two parameter location-scale family (albeit not very useful) that can not be generated by translation and scaling of only a single member.</p> <h3>Case 2: Location-scale families generated by negative scaling of a single distribution with nonzero skew</h3> <p>A less contrived example, than using this tan-function, is given by Whuber under the comments of Carl's answer. We can have a family <span class="math-container">$x \mapsto f(x/b + a)$</span> where flipping the sign of <span class="math-container">$b$</span> keeps the mean and variance unchanged but possibly changing the uneven higher moments. So this gives a bit more easily a two parameter location-scale family where members with the same mean and variance can have different higher order moments. This example from Whuber can be split into two subfamilies each of which can be generated out of a single member by translation and scaling.</p> <h3>Smooth families</h3> <p>If we try to make a single smooth two parameter distribution family (smooth: a small change in the parameters will result in a small change of the distribution/function/curve) by somehow making a composition of two or more families that are generated by translation and scaling, then we get into problems to have the two parameters cover both the variation of 'mean' and 'variance', as well as the third parameter 'shape'. A formal proof will have to go along the same lines as the answer to the question: <a href="https://math.stackexchange.com/q/2897100/466748">Is there a smooth surjective function <span class="math-container">$f:\mathbb{R}^2 \mapsto \mathbb{R}^3$</span>?</a> (where the answer is <em>no</em> in the case of <em>smooth</em>, ie. infinitely differentiable, functions although there are <em>continuous</em> functions that would do the job such as Peano curves). </p> <p>Intuition: Imagine there would be some parameters <span class="math-container">$\theta_1$</span>, <span class="math-container">$\theta_2$</span> that describe the distributions in some location-scale distribution family and by which we can change the mean and variance <em>as well</em> as some other moments, then we should be able to express <span class="math-container">$\theta_1$</span>, <span class="math-container">$\theta_2$</span>, in terms of the mean <span class="math-container">$\mu$</span> and variance <span class="math-container">$\sigma$</span></p> <p><span class="math-container">$$\begin{array}{rcl} \theta_1 &amp;= &amp;f_{\theta_1}(\mu,\sigma) \\ \theta_2 &amp;=&amp; f_{\theta_2}(\mu,\sigma)\end{array}$$</span></p> <p>but these need to be multiple valued functions and these can not make continuous transitions, the different values from <span class="math-container">$f_{\theta_1}(\mu,\sigma)$</span> for a particular <span class="math-container">$\mu$</span> and <span class="math-container">$\sigma$</span> are not continuous, and will not be able to model a continuous shape parameter.</p> <p><em>I am actually not so sure about this final part. We could possibly use a space-filling curve (such as the Peano curve, if only we knew how to express coordinates on the curve to coordinates of the hypercube) to have a single parameter <span class="math-container">$\theta_1$</span> completely model multiple features like mean and variance, without giving up the property that a small change of the parameter <span class="math-container">$\theta_1$</span> is equivalent to a small change of the function <span class="math-container">$f(x;\theta_1)$</span> at every <span class="math-container">$x$</span></em> </p>