# Tag Info

47

The (right) tail of a distribution describes its behavior at large values. The correct object to study is not its density--which in many practical cases does not exist--but rather its distribution function $F$. More specifically, because $F$ must rise asymptotically to $1$ for large arguments $x$ (by the Law of Total Probability), we are interested in how ...

31

The gamma and the lognormal are both right skew, constant-coefficient-of-variation distributions on $(0,\infty)$, and they're often the basis of "competing" models for particular kinds of phenomena. There are various ways to define the heaviness of a tail, but in this case I think all the usual ones show that the lognormal is heavier. (What the first person ...

16

The two definitions are close, but not exactly the same. One difference lies in the need for the survival ratio to have a limit. For most of this answer I will ignore the criteria for the distribution to be continuous, symmetric, and of finite variance, because these are easy to accomplish once we have found any finite-variance heavy-tailed distribution ...

12

The first thing to do is to formalize what we mean by "heavier tail". One could notionally look at how high the density is in the extreme tail after standardizing both distributions to have the same location and scale (e.g. standard deviation): (from this answer, which is also somewhat relevant to your question) [For this case, the scaling doesn't really ...

11

There are two parts to address here -- 1. what does it mean for something to be heavy-tailed? and 2. does higher kurtosis mean a heavier tail and vice-versa? What's heavy-tailed mean? a. What heavier tail means in a "handwavy" sense -- most people picture it this way: but because the tail is quite small, it's better to look at the log-density (which ...

9

Yes, see Theorem 6.21 of [LT13], Michel Ledoux and Michel Talagrand. Probability in Banach Spaces: isoperimetry and processes, volume 23. Springer Science & Business Media, 2013. For simplicity you may also look at section 8 of my paper. http://arxiv.org/pdf/1507.06370v2.pdf (I just summarized those theorems -- the purpose of the paper is completely ...

9

It means that two things are true. First: $$P(X_1 < t) = P(X_2 < t)$$ for all real numbers $t$ (i.e., $X_1$ and $X_2$ have the same distribution, often the shorthand equidistributed is used to describe this condition). Second: $$P(X_1 < t) = \frac{1}{\sigma \sqrt{2 \pi}}\int_{-\infty}^t e^{\frac{(x - \mu)^2}{2 \sigma^2}} \,\text{d}x$$ for ...

9

Just because they don't have a covariance doesn't mean that the basic $x^t\Sigma^{-1} x$ structure usually associated with covariances can't be used. In fact, the multivariate ($k$-dimensional) Cauchy can be written as: $$f({\mathbf x}; {\mathbf\mu},{\mathbf\Sigma}, k)= \frac{\Gamma\left(\frac{1+k}{2}\right)}{\Gamma(\frac{1}{2})\pi^{\frac{k}{2}}\left|{\... 8 Sort of "both" - the line depends both on the observed quantiles (which define the y-axis of the QQ plot) and the expected/theoretical/reference quantiles (which the define the x-axis). The documentation (which you quote) should always be taken as the canonical reference: ‘qqline’ adds a line to a “theoretical”, by default normal, quantile-quantile ... 8 I think "common" here just means that the marginal distribution \text{N}(0,1) is common to both random variables (i.e., they have the same marginal distribution). Although technically this is insufficient to give a bivariate normal distribution, I think the writer probably intended that form:$$\begin{bmatrix} X \\ Y \end{bmatrix} \sim \text{N} \Big( \...

7

Log-log plot is your best choice. Here's an image of mine from this tutorial. Top is linear scale for both, and bottom image is log-log. Notice how the tails are indistinguishable in the linear plot but very clearly different in the log-log plot. As for R, the following web page describes how to set ggplot2 to use log-log scales: http://docs.ggplot2.org/...

7

Although kurtosis is a related to the heaviness of tails, it would contribute more to the notion of fat tailed distributions, and relatively less to tail heaviness itself, as the following example shows. Herein, I now regurgitate what I have learned in the posts above and below, which are really excellent comments. First, the area of a right tail is the area ...

6

The first plot looks to me quite similar to a mixture of a normal (most of the points) and something with larger variance and somewhat heavier tail (perhaps a 90-10 mixture but it's hard to judge) -- and possibly a slightly lower center. Of course, that doesn't mean the original process is physically a mixture of two original processes; you can get exactly ...

6

Ignoring the tails, the Gaussian and Cauchy (T-dist w/ DF=1) look pretty similar in their meaty center. The MAD only looks at the meaty center (more-or-less). The MAD estimates will be pretty similar, which will give a pretty similar range of "acceptable". The Cauchy, with it's fat tails, will violate that acceptable range more often. I'm not sure what ...

6

While $\text{cov}(X,Y)$ does not exist, for a pair of variates with Cauchy marginals, $\text{cov}(\Phi(X),\Phi(Y))$ does exist for, e.g., bounded functions $\Phi(\cdot)$. Actually, the notion of covariance matrix is not well-suited to describe joint distributions in every setting, as it is not invariant under transformations. Borrowing from the concept of ...

5

That's basically the complete list in your question, (the Pareto and the zeta/Zipf). A power law is one where the pdf/pmf is proportional to $x^{-p}\,$ ($1$). People use power laws for either continuous ($x> k$) or discrete ($x=1,2,...$) data; the continuous case proportional to $x^{-p}$ is the Pareto, the discrete case proportional to $x^{-p}$ is the ...

5

Growth rates must be distributed as some variation of the Cauchy distribution. I have written a series of papers on this. The Cauchy distribution has no mean so it has no variance or covariance. You can find my author page at https://papers.ssrn.com/sol3/cf_dev/AbsByAuth.cfm?per_id=1541471 Start with the paper titled "The distribution of returns," and ...

5

There is no distribution which is more heavy-tailed than any other distribution. Proof: Assume $f$ is any PDF, and its CDF is $F$. We can always construct another distribution $$G(x) = 1 - \sqrt{1 - F(x)}, \quad g(x) = \frac{f(x)}{2\sqrt{1 - F(x)}}$$ which has havier tails, since: $$\int_x^\infty f(t)\, dt = 1 - F(x) < \sqrt{1 - F(x)} = 1 - G(x) = \... 4 Wikipedia is often a reasonable start point for basic definitions. In this case there is an entry for heavy-tailed distributions. The distribution of a random variable X with distribution function F is said to have a heavy right tail if$$\lim_{x\rightarrow\infty} e^{\lambda x}P[X>x]=\infty. for all $\lambda>0$. This can be interpreted as: ...

4

There will never be such a number. Convergence will always depend on properties of your random variables. Having said that, there are some results you may want to use. There is Berry Eseen (note there is a bound for the constant in the theorem $C < 0.4748$). So, if $\mathbb{E}[X_i] = 0, 0 < \mathbb{E}[X_i^2] = \sigma^2 < \infty$ and $\mathbb{E}[|... 4 Your Q-Q plot doesn't look like it has a fat tail. I'll show you how a fat tail looks like: Your tail is like a Victoria Secret's model compared to the above. I wish some of my model residuals had tails like yours has. 4 They are breaking down$\{S_n>u\}$into two disjoint events: either$\max_{i\leq n}X_i>u$or it isn't. Presumably$X_i\geq 0$, so that clearly if the maximum exceeds$u$then so does$S_n$. Alternatively all$X_i$might be less than$u$, in which case you need$S_n>u$. Note that$X_i\geq 0$is required, otherwise this is false. 4 I think it simply adds a line segment between the points (x1, y1) & (x2, y2) for given probabilities (p1, p2) (x1, x2) are the quantiles of the theoretical distribution; (y1, y2) for the data comparison. Function qline has simple code under the hood. This is a simple e.g. in R # sample data set.seed(2) y <- rt(100, df = 5) # get the values probs &... 4 I'll take it that you're happy to take it as a given that the ordinary (symmetric) normal is light tailed and the ordinary (symmetric) Cauchy is heavy tailed. That the indicated skew-normal is not heavy-tailed is easy to see. I'll discuss the 'standard case' of the skew-normal; the scale-location case is not really any more work. Taking$f(x)=2\phi (x)\Phi ...

4

The first thing to note is that the estimators in the linear regression model are not particularly sensitive to heavy tails in the error distribution (so long as the error variance is finite). Fitting a standard linear regression to data with excessively heavy tail will mean that data points in the tails are penalised excessively, but the coefficient ...

3

I'm assuming your derivation there comes from something like the one on this page. I have a distribution with only positive outcomes, and the confidence intervals include negative values. Well, given the normal approximation that makes sense. There is nothing stopping a normal approximation from giving you negative values, which is why it is a bad ...

3

Some comments: There is no "best" multivariate extension of the $t$ distribution. The best choice depends on your needs. The most common in practice is the one provided in wikipedia. Take a look at this link ftp://ftp.ecn.purdue.edu/bethel/kotz_mvt.pdf for a discussion on this and other multivariate t distributions. Another important reference is http://...

3

"A prior that is not very close to a particular value" does not make much sense to me. I guess you meant the prior density does not accumulates most of its mass in a neighbourhood of $x_0$. Maybe you can use a uniform prior on a suitable (large) set, in order to reflect vague/weak prior information. This sort of prior induces a posterior that is ...

3

You can generalise the logistic regression model so that the latent distribution is something other than logistic. Using the t distribution lets you capture relationships where the data are contaminated, meaning observations from the "wrong" class appear unexpectedly far away from the decision boundary -- this would be the binary equivalent of fat tails in ...

3

I think you are asking for 2 different distributions that share some common 'essence', but that differ in how heavy their tails are, so that when you plot them the nature of 'heavy-tailedness' can be demonstrated. Is that correct? If so, why not use the $t$ distribution with $1$ and $\infty$ degrees of freedom? Here is a plot of several $t$ distributions ...

Only top voted, non community-wiki answers of a minimum length are eligible