52

Let $F$ be the CDF of $X_i$. We know that the CDF of $Y$ is $$G(y) = P(Y\leq y)= P(\textrm{all } X_i\leq y)= \prod_i P(X_i\leq y) = F(y)^n$$ Now, it's no loss of generality to take $a=0$, $b=1$, since we can just shift and scale the distribution of $X$ to $[0,\,1]$ and then unshift and unscale the distribution of $Y$. So what does $F$ have to be to get $G(y)...


51

That is a 95% confidence interval for $x$, but not the 95% confidence interval. For any continuous monotonic transformation, your method is a legitimate way to get a confidence interval for the transformed value. (For monotonically decreasing functions, you reverse the bounds.) The other excellent answer by tchakravarty shows that the quantiles match up ...


48

The log of $1$ is just $0$ and the limit as $x$ approaches $0$ (from the positive side) of $\log x$ is $-\infty$. So the range of values for log probabilities is $(-\infty, 0]$. The real advantage is in the arithmetic. Log probabilities are not as easy to understand as probabilities (for most people), but every time you multiply together two probabilities (...


33

In general knowing these 4 properties is not enough to tell you the expectation of the absolute value of a random variable. As proof, here are two discrete distributions $X$ and $Y$ which have mean 0 and the same variance, skew, and kurtosis, but for which $\mathbb{E}(|X|) \ne \mathbb{E}(|Y|)$. t P(X=t) P(Y=t) -3 0.100 0.099 -2 0.100 0.106 -1 0....


32

$F_{X_{(n)}}(x)=[F_X(x)]^n$, so for a standard uniform you need $F_X(x)=x^{1/n}$ for $0<x<1$ (and $0$ to the left and $1$ to the right of that interval), so $f_X(x)=\frac{1}{n}x^{\frac{1}{n}-1}$ on the unit interval and $0$ elsewhere. It's a special case of the beta.


21

You can easily show that this is the case. Let $Y\equiv \log(X)$. Then, the $\alpha$-quantile of $Y$ is $y\in\mathbb{R}$, such that $\mathbb{P}[Y \leq y] = \alpha$. Similarly, the $\alpha$-quantile of $X$ is $x \in \mathbb{R}^+$, such that $\mathbb{P}[X \leq x] = \alpha$, or, $\mathbb{P}[\log(X) \leq y] = \alpha$, or, $\mathbb{P}[X \leq \exp(y)] = \alpha$. ...


21

The video suggests that $\mu=112$ g and $\sigma=9$ g in this particular normal distribution. If that is the case, we can find the probability that the weight is in a given interval, in the video described as the area under the graph for that interval. For example, the probability it is between $119.5$ g and $120.5$ g is $$\Phi\left(\tfrac{120.5-112}{9}\...


20

I. A direct answer to the OP Answer: It depends on what you mean by “heavy tails.” By some definitions of “heavy tails,” the answer is “no,” as pointed out here and elsewhere. Why do we care about heavy tails? Because we care about outliers (substitute the phrase “rare, extreme observation” if you have a problem with the word “outlier.” However, I will use ...


19

The Oxford English Dictionary defines "conjugate" as an adjective meaning "joined together, esp. in a pair, coupled; connected, related." It's not a huge stretch to imagine that a conjugate prior has a special and strong connection to its posterior. It's used in a similar sense in chemistry (conjugate acid/base; conjugate solution), ...


18

I would like to add that taking the log of a probability or probability density can often simplify certain computations, such as calculating the gradient of the density given some of its parameters. This is in particular when the density belongs to the exponential family, which often contain fewer special function calls after being logged than before. This ...


18

If you want a distrbution that looks identically to the exponential distribution, up to a multiplicative constant, you can use a truncated exponential distribution. It is defined by restricting the support of an exponential distribution to the interval of interest and then re-normalizing the density to obtain a distribution. Your case would yield $$f(x) = \...


17

There are two common ways to represent a probability distribution, the probability density function (PDF) and cumulative distribution function (CDF). I suspect you're wondering most about the former. For the latter, the distribution is plotted as cumulative from zero to one, so the y-axis is the sum of the distribution up to a given value of x. For a ...


16

The beta distribution will work if and only if $\alpha<1$ and $\beta>1$ (one of the two inequalities can be replaced by $\leq$ and $\geq$ if you don't mind a flat PDF at $x=0$ or $x=1$.) Its PDF is $$ f(x)=\frac{x^{\alpha-1}(1-x)^{\beta-1}}{B(\alpha,\beta)}, $$ so its derivative is $$ f'(x) = \frac{x^{\alpha-2}(1-x)^{\beta-2}}{B(\alpha,\beta)}\big((\...


16

It depends on what you mean by an "analytical" calculation. In general, this is just $$ E(|X|) = \int |x| f(x)\,dx, $$ so you do have a formula. But I assume that "evaluating a (possibly improper) integral" is not what you had in mind. Then again, probably the simplest non-trivial example would be that of the absolute value of a normal ...


16

In GLM's the exponential family of distributions (not the exponential distribution, https://en.wikipedia.org/wiki/Exponential_family) is used to model various outcomes, Gaussian (or normal) distribution for a real continuous variable, Gamma distribution for a real positive continuous variable, binomial distribution for a discrete variable, and so on. An ...


15

Let me add an example to @develarist's answer. \begin{array}{llcc|r} Y & & y_1 & y_2 \\ \hline X & x_1 & 0.450 & 0.150 & 0.600 \\ & x_2 & 0.167 & 0.233 & 0.400 \\ \hline & & 0.617 & 0.383 & 1.000 \end{array} The table shows the joint distribution of $(X,Y)$: \begin{array}{l} P(X=x_1,Y=y_1)=0.450 \\ ...


15

No, if the individual random variables are continuous and thus their marginal distributions can be described using pdfs, it is not necessarily the case that they enjoy a joint pdf. A standard counterexample to the OP's "claim" is when $X \sim N(0,1)$ and $Z$ is an independent discrete random variable taking on values $\pm 1$ with equal probability. ...


15

I am guessing that you are looking for a positive, continuous probability distribution with infinite mean and with a maximum density away from zero. I thought that by analogy with a Gamma distribution ($p(x) \propto x^a \exp(-x) \, dx$), we could try something with a rational (polynomial) rather than an exponential tail. After a little bit of lazy R and ...


14

No. Many data sets can yield the same mean, SD and n. In the graph above, the three data sets on the left of each graph all share the same mean, SD and n. So do the three data sets on the right of each graph. This is Figure 1 from: Weissgerber, T.L., Milic, N.M., Winham, S.J., and Garovic, V.D. (2015). Beyond bar and line graphs: time for a new data ...


14

Given two samples that have the same mean, standard deviation, and N: are the values in each sample identical? In general, not unless N=2 in both samples. If N is larger than 2, they can differ. You can see this simply by trying it with some simple cases. Perhaps the easiest case is to take an asymmetric sample of size $N=3$ and flip it around its mean ($2,...


14

If $X$ and $Y$ are two random variables, the univariate pdf of $X$ is the marginal distribution of $X$, and the univariate pdf of $Y$ is the marginal distribution of $Y$. Therefore, when you see the word marginal, just think of a single data series' own distribution, itself. don't be tricked into thinking marginal means something different or special than a ...


14

The code for numpy.random.beta is found at legacy-distributions.c at the time of this writing. When $a$ and $b$ are both 1 or less, then Jöhnk's beta generator is used (see page 418 of Non-Uniform Random Variate Generation), with a modification to avoid divisions by zero. Otherwise, it uses the formula $X/(X+Y)$ where $X$ and $Y$ are gamma($a$) and gamma($b$...


13

I will describe every possible solution. This gives you maximal freedom to craft distributions that meet your needs. Basically, sketch any curve you like for the density function $f$ that meets your requirements. Separately scale the heights of the left and right halves of it (on either side of $c$) to make their masses balance, then scale the heights ...


12

Not in general. It's not possible to do it exactly if $\lambda_2>\lambda_1$, since a Poisson variable with mean $\lambda_2$ has higher entropy than one with mean $\lambda_1$, so it takes more information to specify it, even if you are willing to have a crazy non-monotone transformation. For $\lambda_2<\lambda_1$, it is at least not always possible. ...


12

In a classical hypothesis test, you have a test statistic that orders the evidence from that which is most conducive to the null hypothesis to that which is most conducive to the alternative hypothesis. (Without loss of generality, suppose that a higher value of this statistic is more conducive to the alternative hypothesis.) The p-value of the test is the ...


12

No, a very simple counterexample is $(Z, Z)$ where $Z \sim \mathcal{Norm}(0,1)$ where the marginals are standard normal but the joint distribution is concentrated on the diagonal line $y=x$. So the joint distribution do not have a density with respect to the Lebesgue measure on the plane, but it does indeed have a density with respect to the Lebesgue density ...


11

I guess the statement could be made more precise and then it could be easier to understand. First of all, $f(x) = \tfrac{1}{C}$, where $C$ is a constant so that it integrates to unity, is a probability density of an uniform distribution that assigns the same probability density to each point. A normal distribution does not have the same flat shape so that ...


11

I can show you all examples, not just the simple ones. Solution Here they are, schematically: The bottom panels show how the density function $f$ of a distribution $F$ is split into two parts vertically along a nearly arbitrary curve. The cyan portion of the split is a fraction $\lambda$ of $f;$ the upper left plots its graph. The remaining portion (gray) ...


11

The beta distribution can have $\alpha$ and $\beta$ set such that it is: Monotonically decreasing Supported on $[0, 1]$ Have a look at the example on Wikipedia where $\alpha = 1, \, \beta = 3$, for instance. There are also readily available implementations of beta regression in R (e.g. betareg), if that is what you want to use it for.


11

There are infinitely many functions that can generate a distribution that is monotonically decreasing and has a support [0,1] (by integrating a positive function adding an integration constant and properly normalizing) You are looking for a named distribution. That will reduce the options. But you still have many options left and this is a very broad ...


Only top voted, non community-wiki answers of a minimum length are eligible