Hot answers tagged

52

This question provides a nice opportunity to collect some facts on moment-generating functions (mgf). In the answer below, we do the following: Show that if the mgf is finite for at least one (strictly) positive value and one negative value, then all positive moments of $X$ are finite (including nonintegral moments). Prove that the condition in the first ...


30

Kurtosis is certainly not the location of where the peak is. As you say, that's already called the mode. Kurtosis is the standardized fourth moment: If $Z=\frac{X-\mu}{\sigma}$, is a standardized version of the variable we're looking at, then the population kurtosis is the average fourth power of that standardized variable; $E(Z^4)$. The sample kurtosis is ...


29

Yes, examples with skewness and excess kurtosis both zero are relatively easy to construct. (Indeed examples (a) to (d) below also have Pearson mean-median skewness 0) (a) For example, in this answer an example is given by taking a 50-50 mixture of a gamma variate, (which I call $X$), and the negative of a second one, which has a density that looks like ...


25

The general proof of this can be found in Feller (An Introduction to Probability Theory and Its Applications, Vol. 2). It is an inversion problem involving Laplace transform theory. Did you notice that the mgf bears a striking resemblance to the Laplace transform?. For use of Laplace Transformation you can see Widder (Calcus Vol I) . Proof of a special case:...


24

In short: No. There are several properties of a probability distribution that need not affect its mean and variance, but do determine its shape. Skew & Kurtosis For example, a Poisson distribution with $\lambda = 1$ has expected value $\lambda = 1$ and variance $\lambda = 1$. So does a normal distribution with $\mu = 1$ and $\sigma^2 = 1$. An example ...


22

Let me answer in reverse order: 2. Yes. If their MGFs exist, they'll be the same*. see here and here for example Indeed it follows from the result you give in the post this comes from; if the MGF uniquely** determines the distribution, and two distributions have MGFs and they have the same distribution, they must have the same MGF (otherwise you'd have a ...


22

There is something puzzling in those results since the first method provides an unbiased estimator of $\mathbb{E}[X^2]$, namely$$\frac{1}{N}\sum_{i=1}^N X_i^2$$has $\mathbb{E}[X^2]$ as its mean. Hence the blue dots should be around the expected value (orange curve); the second method provides a biased estimator of $\mathbb{E}[X^2]$, namely$$\mathbb{E}[\exp(...


22

Let's assume that an equation-free intuition is not possible, and still insist on boiling down the math to the very essentials to get an idea of what's going on: we are trying to obtain the statistical moments, which, after the obligatory reference to physics, we define as the expected value of a power of a random variable. For a continuous random variable, ...


22

Three moments don't determine a distributional form; if you choose a distribution-famiy with three parameters which relate to the first three population moments, you can do moment matching ("method of moments") to estimate the three parameters and then generate values from such a distribution. There are many such distributions. Sometimes even having all ...


19

$0<s<r \Longrightarrow \forall X \, |X|^s \le \max(1, |X|^r) $


19

First let's address the case $\Sigma = \sigma\mathbb{I}$. At the end is the (easy) generalization to arbitrary $\Sigma$. Begin by observing the inner product is the sum of iid variables, each of them the product of two independent Normal$(0,\sigma)$ variates, thereby reducing the question to finding the mgf of the latter, because the mgf of a sum is the ...


17

Let $U_1, U_2,\dots$ be iid $\mathcal U(-b,b)$ random variables and consider the normalized sum $$ S_n = \frac{\sqrt{3} \sum_{i=1}^n U_i}{b \sqrt{n}} \>, $$ and the associated $\sup$ norm $$ \delta_n = \sup_{x\in\mathbb R} |F_n(x) - \Phi(x)| \>, $$ where $F_n$ is the distribution of $S_n$. Lemma 1 (Uspensky): The following bound on $\delta_n$ holds. $$...


17

It's been a long time since I took a physics class, so let me know if any of this is incorrect. General description of moments with physical analogs Take a random variable, $X$. The $n$-th moment of $X$ around $c$ is: $$m_n(c)=E[(X-c)^n]$$ This corresponds exactly to the physical sense of a moment. Imagine $X$ as a collection of points along the real line ...


17

If you want an example which is an "officially named parameterized distribution family, you can look into the generalized gamma distribution, https://en.wikipedia.org/wiki/Generalized_gamma_distribution. This distribution family has three parameters, so you can fix mean and variance and still have freedom to vary higher moments. From the wiki page, the ...


14

Yes. In fact, you don't even need to know that $E[X]$ is finite: if you know that the $k$-th moment $E[X^k]$ is finite, then all lower moments must be finite. You can see this using Jensen's inequality, which says that for any convex function $\varphi$ and random variable $X$, $$\varphi(E[X]) \leq E[\varphi(X)].$$ Now, suppose we know that $E[X^k]$ is ...


14

There is an infinite number of distributions with mean zero and variance one, hence take $\epsilon_1$ distributed from one of these distributions, say the $\mathcal{N}(0,1)$, and $\epsilon_2$ from another of these distributions, say the Student's $t$ with 54 degrees of freedom rescaled by $\sqrt\frac{1}{3}$ so that its variance is one, then $$X=\mu+\sigma\...


13

In a sense, an MGF is simply a way of encoding a set of moments into a convenient function in a way that you can do some useful things with the function. The variable $t$ in no way relates to the random variable $X$. You could as readily write $M_X(s)$ or $M_X(u)$... it is, in essence a kind of dummy variable. It doesn't stand for anything beyond being the ...


13

I thought I'd throw up some figs showing that both user29918 and Xi'an's plots are consistent. Fig 1 plots what user29918 did, and Fig 2 (based on same data), does what Xi'an did for his plot. Same result, different presentation. What's happening is that as T increases, the variances becomes huge and the estimator $\frac{1}{n} \sum_i x_i^2$ becomes like ...


12

As mentioned in the comments, characteristic functions always exist, because they require integration of a function of modulus $1$. However, the moment generating function doesn't need to exist because in particular it requires the existence of moments of any order. When we know that $E[e^{tX}]$ is integrable for all $t$, we can define $g(z):=E[e^{zX}]$ ...


12

This question comes up a lot in various guises. What is common to them is How can I combine moment-based statistics that have been computed from disjoint subsets of my data? The simplest application concerns data that have been split into two groups. You know the group sizes and the group means. In terms of these four quantities alone, what is the ...


12

I'll give you a few hints that will allow you to compute the mean and variance from your pdf. First of all, remember that the expected value of a univariate continuous random variable $E[X]$ is defined as $E[X] = \int_{-\infty}^{\infty}{x f(x) dx}$ as explained here, where the range of the integral corresponds to the sample space or support (say, $(-\infty, ...


12

The two estimators you are comparing are the method of moments estimator (1.) and the MLE (2.), see here. Both are consistent (so for large $N$, they are in a certain sense likely to be close to the true value $\exp[\mu+1/2\sigma^2]$). For the MM estimator, this is a direct consequence of the Law of large numbers, which says that $\bar X\to_pE(X_i)$. For ...


11

Method 1: Higher-order Pearson systems The Pearson system is, by convention, taken to be the family of solutions $p(x)$ to the differential equation: $$ \frac{d p (x)}{dx} \; = \; -\frac{(a+x) }{c_0 + c_1 x + c_2 x^2} \; p(x) $$ where the four Pearson parameters $(a, c_0, c_1, c_2)$ can be expressed in terms of the first four moments of the population. ...


11

The general form of the covariance depends on the first three moments of the distribution. To facilitate our analysis, we suppose that $X$ has mean $\mu$, variance $\sigma^2$ and skewness $\gamma$. The covariance of interest exists if $\gamma < \infty$ and does not exist otherwise. Using the relationship between the raw moments and the cumulants, you ...


10

By the language on the theorem, he's clearly referring to a random $D$-dimensional vector. This means that each $y_d$ is a random variable; for the sake of notation lets denote it by $Y_d$ (I really hate when authors don't do the distinction). With that said, the $n$-th order moment about $x_0$ of $Y_d$ is defined as: $$\mathbb{E}[(Y_d-x_0)^n]=\int(y_d-x_0)^...


10

We can use an entirely analogous technique to the one typically used to calculate the moments of a lognormal. In particular, note that if $\newcommand{\E}{\mathbb E}X \sim \mathrm{Bin}(n,p)$ and $Y = e^X$, then $Y^k = e^{k X}$. But, $\E e^{kX} = m_X(k)$ where $m_X(t)$ is the moment-generating function of $X$ evaluated at $t$. Hence, $$ \E e^{k X} = m_X(k)...


10

What you think about here is something like a philosopher's stone of statistics. The strict answer is: No, it is impossible to express skewness or kurtosis via the mean and variance. @Macro gave a counterexample of distributions with different skewness and kurtosis. A question of coming up with distributions for the given set of moments has entertained ...


10

Yes. In an exercise, Stuart & Ord (Kendall's Advanced Theory of Statistics, 5th Ed., Ex. 3.12) quote a 1918 result of TJ Stieltjes (which apparently appears in his Oeuvres Completes,): If $f$ is an odd function of period $\frac{1}{2}$, show that $$\int_0^\infty x^r x^{-\log x} f(\log x) dx = 0$$ for all integral values of $r$. Hence show ...


10

Because the characteristic function (cf) of a Bernoulli$(p)$ variate is $$\psi_p(t) = 1 + p(e^{it}-1),$$ the cf of a sum of $n$ independent such variates (which is a Binomial$(p,n)$ variable) is $$\psi_p(t)^n = \left(1 + p(e^{it}-1)\right)^n = \left(1 + \frac{np(e^{it}-1)}{n}\right)^n.$$ It is well known (and easy to show, even for Complex numbers) that ...


9

First, you've mixed up your variables, using $r$ in some cases and $t$ in others. I will use $t$. Next, you should think about what it means to evaluate an MGF at a point. Recall that $$M_X(t) = {\rm E}[e^{tX}].$$ So if $X \sim {\rm Gamma}(2,1)$, then in order for the MGF of $X$ at, say, $t = 2$ to be defined, we would require the integral $${\rm E}[e^{...


Only top voted, non community-wiki answers of a minimum length are eligible