New answers tagged

0

Take a mean-value expansion around $(\theta_1, \theta_2)$ of $T(\hat \theta_1, \theta_2)$: \begin{align} T(\hat \theta_1, \theta_2) = T(\theta_1, \theta_2) + \frac{\partial T(\bar \theta_1, \bar \theta_2)}{\partial \theta_1}\cdot (\hat \theta_1-\theta_1) \\ + \frac{\partial T(\tilde \theta_1, \tilde \theta_2)}{\partial \theta_2}\cdot (\theta_2-\theta_2) \end{...


0

As said in the comments, we can show that convergence of the CDF. Let's call that discrete uniform variable $X_n$. For any $0\leq t\leq1$, we have: $$P(X_n<t)=\frac{[tn]}{n}$$ Where $[tn]$ means the floor of $tn$. When $n\rightarrow\infty$, this expression converges to $t$. So, we have: $$\lim_{n\rightarrow\infty}P(X_n<t)=P(\text{Unif}(0,1)<t)$$ ...


1

Let $X_1, \dots, X_n$ be a random sample from a continuous distribution with density function $f(x)$ that is continuous and nonzero at the $p$th percentile $x_p$ $(0 < p < 1).$ If $k/n \rightarrow p$ (with $k-np$ bounded), then the sequence of order statistics $x_{k:n}$ is asymptotically normal with mean $x_p$ and variance $c^2/n,$ where $c^2 = p(1-p)/[...


1

Yes, the distributions are always standardized to have zero mean and unit variance. For details, see section 2.3. Conditional Distributions in the vignette of the rugarch package (starting from p. 16).


3

Priors are selected the way you want to penalize your weights. It's not because weights are in general normally or laplacian distributed. L1 regularisation promotes sparsity, while L2 regularisation promotes small weights and go nuts if the absolute value of a weight is high. So, it depends on your purpose. One might use $L_\infty$ norm to limit the maximum ...


0

The negative binomial distribution has also the geometric distribution as waiting time. The geometric distribution can be expressed in two ways: with the domain $k \in \lbrace 1,2,3,\dots \rbrace$ $$f(k) = (1-p)^{k-1}p^k$$ and with the domain $k \in \lbrace 0,1,2,\dots \rbrace$ $$f(k) = (1-p)^kp^k$$ Intuition Say you have the following coin flips result: H,...


1

Although you haven't explicitly stated, it seems that there are three RVs $a,b,c$ with binary values, e.g. $a_1$ meaning $a=1$. Given the last three probabilities, i.e. $P(a_1|c_1), P(a_1|c_0), P(c_1)$ you can find the complete joint distribution of $P(a,c)$. And, when you multiply the first four probabilities with corresponding joint distributions of $a$ ...


1

All scipy distributions have a scale and loc parameter too. So, as scipy arguments: $$a=\gamma, b=\delta, \text{scale}=\lambda, \text{loc}=\eta$$ The constraints look identical to me between scipy and wikipedia, as well.


0

Prediction problems: Assume a situation you can model with some kind of regression(-like) model. An error distribution with long tails will tend to produce many outliers, that is, observations far from the bulk of the data. That makes predictive modelling harder, and could indicate need for robust methods. For some examples Building a predictive model, ...


1

We can explore the Dirichlet family of distributions to check if it contains some distribution satisfying your conditions. So, we have: $$P\sim\text{Dir}(\alpha_1,\alpha_2,\alpha_3)$$ It is reasonable to impose $\mathbb E(P)=(0.50, 0.30, 0.20)$, which implies gives us: $$(\alpha_1,\alpha_2,\alpha_3)=(0.5x,0.3x,0.2x)$$ For some positive number $x$. Now it ...


1

If we call the probabilities as $P,Q,R$ for green, red and blue respectively, and assume the marginals given are correct, we can do the following w/o assuming anything about the joint PDF (i.e. Dirichlet or not): $$P(\text{Green})=\int_{0}^1 P(\text{Green}|P=p)f_P(p)dp=\int_0^1 pf_P(p)dp=\mathbb E[P]$$ So, it is the expected value of $P$. If the marginal is ...


1

In terms of the CDF $F(t)$ or the survival function $S(t) = 1-F(t)$ you have $$p(X>t+s|X>t) = \frac{S(t+s)}{S(t)}$$ You get this fraction to be constant for different $t$ and $s$ when $S(t)$ is an exponential function. (And obviously the relation breaks when $t>1$ or $t+s>1$, because that exponential relation ends above 1. So you only have ...


0

I have a feeling characteristic functions are the best option for handling this question. For basics, kindly see:https://en.wikipedia.org/wiki/Characteristic_function_(probability_theory) now, kindly CTRL+F "independen". you'll see the answer in brief: The characteristic function approach is particularly useful in analysis of linear combinations ...


0

It is not an assumption. The careless statement Pan makes is not an assumption since no other data scientist seems to generally assert it given how large a field ML is. What he is trying to say, and should be corrected, is that ML performs best when the test data is similar to the training data. This makes much more practical sense, and would be corroborated ...


1

We have the equivalence between these events: $$ \left \{ X^2 \in \left ( \frac{i}{4} , \frac{i+1}{4} \right ) \right \} \iff \left \{ X \in \left ( \frac{\sqrt{i}}{2} , \frac{\sqrt{i+1}}{2} \right ) \right \} \bigcup \left \{ X \in \left ( -\frac{\sqrt{i+1}}{2} , -\frac{\sqrt{i}}{2} \right ) \right \} $$ The union on the lhs is disjoint and its probability ...


6

In general, generating a random number from a probability distribution means transforming random numbers so that the numbers fit the distribution. Perhaps the most generic way to do so is called inverse transform sampling: Generate a uniform random number in [0, 1]. Run the quantile function (also known as the inverse CDF or the PPF) on the uniform random ...


17

If you want a distrbution that looks identically to the exponential distribution, up to a multiplicative constant, you can use a truncated exponential distribution. It is defined by restricting the support of an exponential distribution to the interval of interest and then re-normalizing the density to obtain a distribution. Your case would yield $$f(x) = \...


10

There are infinitely many functions that can generate a distribution that is monotonically decreasing and has a support [0,1] (by integrating a positive function adding an integration constant and properly normalizing) You are looking for a named distribution. That will reduce the options. But you still have many options left and this is a very broad ...


15

The beta distribution will work if and only if $\alpha<1$ and $\beta>1$ (one of the two inequalities can be replaced by $\leq$ and $\geq$ if you don't mind a flat PDF at $x=0$ or $x=1$.) Its PDF is $$ f(x)=\frac{x^{\alpha-1}(1-x)^{\beta-1}}{B(\alpha,\beta)}, $$ so its derivative is $$ f'(x) = \frac{x^{\alpha-2}(1-x)^{\beta-2}}{B(\alpha,\beta)}\big((\...


10

The beta distribution can have $\alpha$ and $\beta$ set such that it is: Monotonically decreasing Supported on $[0, 1]$ Have a look at the example on Wikipedia where $\alpha = 1, \, \beta = 3$, for instance. There are also readily available implementations of beta regression in R (e.g. betareg), if that is what you want to use it for.


1

It means that the test data look like the training data. For example, if your facial recognition system was developed in China, it might work well in China, but not if you try to use it in a country where people look different. The "drawn" part means that the data is sampled at random from some population of interest. For example, the population ...


2

In this question/answer I used the following graph: Joint distribution In the left plot you see the joint distribution of disp versus mpg. This is a scatterplot in a 2D-space. Marginal distribution You might be interested in the distribution of all the 'mpg' together. That is depicted by the first (big) histogram. It shows the distribution of 'mpg'. (note ...


1

Based on your Bayesian graph, the joint distribution can be written as $$\begin{align}P(G_1=1,G_2=0,G_3=1,G_4=0)&=P(G_4=0|G_3=1)\times P(G_3=1|G_2=0)\\&\times P(G_2=0|G_1=1)P(G_1=1)\end{align}$$ You have all the necessary probabilities in your table.


0

Comment continued with simulation: Find sample means of 100,000 normal samples of size $n=15.$ $SD(A) = SD(\bar X) = \sigma/\sqrt{n} = 10/\sqrt{15} = 2.5820.$ set.seed(916) a = replicate(10^5, mean(rnorm(15, 100, 10))) mean(a); sd(a) [1] 100.0052 [1] 2.590418 Do the same for sample medians: SD(H) \approx 3.19 > 2.58.$ [Because the same seed is used, ...


0

OK, my review of trigonometry upon drawing a right triangle with the straight distance to the wall being 'b', and the randomly observed bullet (shot at an angle theta) into the wall resting at a distance 'X' from the point on the wall (that is perpendicular from the gun) suggests that tangent of theta equals X/b. If you wanted to simulate a random deviate ...


1

Two unsupervised learning algorithms come to mind that can help derive information of the individual components of a multimodal distribution. They isolate the individual unimodal densities within a multimodal distribution, and from there, the information and statistics of the isolated unimodal densities can be evaluated independently as you would normally do,...


2

Wikipedia gives several summary statistics for bimodality. I will give some useful examples: Sarle's bimodality coefficient Reminiscent of a proposal by Pearson's, it builds on the idea that bimodal distributions present low kurtosis, high skewness, or both at the same time. $\gamma$ is the skewness while $\kappa$ is kurtosis. $\beta \in [0,1]$. $\beta = 5/9$...


1

A typical way to solve it would be applying Total Probability Law: $$P(Y=1)=\sum_{n=1}^\infty P(Y=1|X=n)P(X=n)=\sum_{n=1}^\infty n\left(\frac{1}{4}\right)^n$$


2

If you really need the variables to sum to one, you could "force it" by dividing by the sum. That is, if $X_1, X_2, \cdots X_n$ are random variables, then the RVs $$Z_i = \frac{X_i}{\sum_{i=1}^n X_i}$$ have the property that $\sum_{i=1}^nZ_i = 1$ (so long as $\sum X_i \neq 0$). This is easy to show. $$\sum_{j=1}^n Z_j = \sum_{j=1}^n \frac{X_j}{\...


0

No. In lose sense a random sample can be seen as some sort of instantiation of the random variable. However, the RV itself is not an instance of its distribution in any meaningful context. The distribution function doesn't have an instance.


1

Yes you can --- this is both technically feasible and it can also aid intuition Intuition: Probabilistic intuition is best when it is built on an epistemic foundation that views probability as a belief based on available information. For this reason, it is generally a bad idea to try to build up intuition by thinking about whether a random variable is a ...


0

I'm going through the course, too. The Aha moment came with the distinction that a random variable is a function. Blitzstein isn't the only one who says this, but it was the first time I finally got it. An r.v. is not an algebraic variable. In fact, it even makes sense if you make up privately, for didactic purposes only, a new name for it instead of ...


3

No, this is not possible. The marginals in an elliptical distribution are all scaled versions of one another (this is part of the definition). Thus, whenever one marginal has an absolute moment of order $\kappa$ (which may be a fraction), so does the other. But a $t$ distribution of $\nu$ degrees of freedom, whose PDF decays asymptotically like $|x|^{-\nu+...


4

Autorcorrelated processes A variable in series that 'remembers' its previous values to some degree is not i.i.d.! Any autoregresive value depends on previous values of the variable, and the distribution changes depending on location within the series. For example, the time series variable $y$, where $t$ indicates period of time, $y_t = \beta_0 + \beta_1 y_{t-...


3

If you randomly draw a card from a deck of playing cads, do not put it back, and draw again. Then, the probability distributions for which card will be drawn in each of the two draws are dependent and not identical. Otherwise, if the card of first draw is put back and well shuffled before the second draw, then the distributions of the two draws are ...


3

Some other "real-world"-examples: Let $(M, F)$ be a pair of measurements on an opposite sex married couple, sampled randomly: Measurement is height, will have different means. Measurement is IQ, same mean, different variance. (But maybe for this example, independence is in doubt ...) Paired data in general can be used to make many similar ...


9

Next to the "formal" example by Xi'an, a "real-world" example might be height and weight. Already because the two are measured on different scales will they be distributed differently, but they sure are dependent, as taller people tend to be heavier.


3

If $\varepsilon_1,\varepsilon_2$ are iid $\mathcal N(0,1)$, $$X_1=\mu_1+\sigma_1\epsilon_1\qquad X_2=\mu_2+\varrho \epsilon_1 + \sigma_2 \epsilon_2$$ is a pair of dependent RVs that are not identically distributed for most values of the parameters.


0

Example: A random variable with a standard Cauchy distribution (t distribution with DF=1) has no mean, but median $\eta = 0$ and $f(\eta)=1/\pi.$ So according to @MatthewHolder's Comment (see Ref.), the median $H$ of a sample of size $n=100$ from this distribution should have $H \stackrel{aprx}{\sim} \mathsf{Norm}(\mu=0,\, \sigma=\pi/20).$ set.seed(2020) h =...


6

Scaling a Dirichlet distribution If you want a variable that is distributed like a Dirichlet distributed variable but with a different range then you can scale and shift (transform the variable). This is effectively rescaling the axes. To get from $[0,1]$ to $[-1,1]$ you can multiply by 2 and subtract 1. That is, your new variable $Y$ can be based on a ...


0

I can give you two explanations, one is intuitive and the other is more mathematical: The intuitive explanation: During the training, each data point is seen by model several times (once per epoch for example) and the model is updated through out the training. So every time a data point is mapped to a different point in the latent space (due to changes of ...


0

For the asymptotic tail, search for "sub-Weibull" distributions. They develop concentration properties in a similar manner we cope with sub-exponentials or sub-gaussians.


1

One way of finding the joint pdf of $Y = 2+X_1+X_2+X_3$ and $Z = 5+X_1-X_2+2X_3$ without matrix manipulations is to understand that since $X_1, X_2, X_3$ are jointly normal, linear combinations of $X_1, X_2, X_3$ are also jointly normal. Thus, all we need to do is find the means, variances, and covariance of $Y$ and $Z$ and we can write down an explicit ...


2

Like @whuber said in the comments, a nice way to proceed is defining the matrix $$a=\left[\begin{array}{cc}1&1\\1&-1\\1&2\end{array}\right]$$ Such that $(Y_1.Y_2)=(X_1,X_2,X_3)\cdot a+(2,5)$. Now, since this is a constant vector plus a linear transformation of a normally distributed vector, this also has normal distribution. Its mean is found to ...


2

An expression like $E[X=k]$ doesn't have any commonly understood meaning, as far as I'm aware. $X=k$ is not a random variable, it's an event. So it doesn't have an expected value. This is probably a typo on your assignment and instead of $E[X=k]$ it's supposed to just say $E[X]$.


0

A closed-form expression of the PDF of Y can be obtained by taking derivative on $P_r(Y≤y)$ with respect to y. It turns out that Y is of logit-norma distribution.


2

A very simple example from the Skew normal distribution with density $$ 2\phi(x)\Phi(\alpha x) $$ Choose for the twocomponents $\alpha, -\alpha$ then $$ \frac12 2 \phi(x) \Phi(-\alpha x) + \frac12 2 \phi(x) \Phi(\alpha x) $$ is the standard normal density $\phi(x)$, by using symmetry, since $\Phi(-\alpha x) = 1-\Phi(\alpha x)$, but unfortunately the two ...


10

I can show you all examples, not just the simple ones. Solution Here they are, schematically: The bottom panels show how the density function $f$ of a distribution $F$ is split into two parts vertically along a nearly arbitrary curve. The cyan portion of the split is a fraction $\lambda$ of $f;$ the upper left plots its graph. The remaining portion (gray) ...


1

Well, a copula is just a different way to present the joint distribution. From the copula and the marginals the joint distribution can be reconstructed exactly (by using Sklar's theorem.) Moments is a different story, you cannot in general recontsruct the joint distribution exactly from any finite set of moments, sometimes you can get good approximations. ...


1

My questions are: For this example, can algebra and/or calculus prove that the distribution of Km values is lognormal (or prove it has some other distribution)? More generally, what method can be used to derive the distribution of any parameter fit by nonlinear regression? The Km values can not be exactly lognormal. This is because in your problem ...


Top 50 recent answers are included