Taylor
  • Member for 10 years
  • Last seen this week
What does "marginal" mean as a noun?
5 votes

In this case "marginal" is short for marginal distribution. If you have a distribution for a few random variables, that's usually termined the "joint" (distribution). When you look at individual ...

View answer
Distribution of $X=\binom m Y \left(\frac 1 2\right)^Y \left(\frac 1 2\right)^{m-Y}$ where $Y$ is a Binomial variable?
Accepted answer
5 votes

If $Y \sim \text{Binomial}(m,p)$, then its pmf is $f_Y(y;p) = {m \choose y} p^y(1-p)^{m-y}$. You are transforming $Y$ by applying this pmf to it, and you're using the value $p=1/2$. So $$ X = f_Y(Y;.5)...

View answer
Functions of independent random variables are independent
5 votes

Yes to (1) and no to (2). Let me explain. The reasoning is from the transformation theorem. This is it generally. Assume you have two original random variables $X_1$ and $X_2$, along with their joint ...

View answer
What is a Highest Density Region (HDR)?
5 votes

A highest posterior density [interval] is basically the shortest interval on a posterior density for some given confidence level. A highest density region is probably the same idea applied to any ...

View answer
ARIMA, what is the interpretation for the sum of AR coefficients?
Accepted answer
4 votes

Just rearrange your $\text{AR}(p)$ polynomial $\phi(z) = 1 - \phi_1 z - \cdots - \phi_p z^p$. If $z'$ is a root, then $1 - \phi_1 z' - \cdots - \phi_p (z')^p = 0$. If $z'$ is a unit root, then you can ...

View answer
Importance sampling estimation of power function
Accepted answer
4 votes

I will mention three ways to approximate the following $$ \mathrm{E} \left[ \phi(X_1, \cdots, X_{100}) \right] = Pr\left(\frac{\bar{X} - 0.1}{\sqrt{0.1/100}} < -1.645 \right). $$ Use the CLT to ...

View answer
Does $\text{cov}(a_1' X, a_2' X) = 0$ imply $a_1 \cdot a_2 = 0$?
Accepted answer
4 votes

Consider a random walk model, $X_t = X_{t-1} + Z_t$ where $Z_t \overset{iid}{\sim} \text{Normal}(0,1)$. Also, assume that $X_1 = Z_1$ and has variance $1$. Take the random vector $X = (X_1,X_2,X_3)'$, ...

View answer
What is the distribution of $\frac{(Y_1 - Y_2)^2}{2},$ where $Y_i$ are standard Normal and independent.
4 votes

I'll answer this question as if it had been tagged with the self-study tag. Hint: consider $Z = \frac{Y_1 - Y_2}{\sqrt{2}} \sim \text{Normal}(0,1)$. You are looking for the distribution of $Z^2$.

View answer
Is a GMM-HMM equivalent to a no-mixture HMM enriched with more states?
4 votes

No you are not wrong thinking that. If $Y \mid X_1 \sim \alpha f_1(y) + (1-\alpha)f_2(y)$, then you can also let $X_2 \sim \text{Bernoulli}(\alpha)$ independently and say $$ Y \mid X_1, X_2 = 1 \...

View answer
Calculating joint density function of Brownian motion
4 votes

The other answer doesn't really give the full story. It's the transformation theorem. I just had an answer involving it here. Call $$X(t_1) = Y_1,~X(t_2) = Y_2 + Y_1, ...,~X(t_n) = Y_n + \cdots + ...

View answer
Is an event a subspace of the sample space?
Accepted answer
4 votes

When I asked after class, I was told that events are not subsets of the sample space. No you're correct. Events are subsets of the sample space. There could be a few sources of confusion, though. ...

View answer
How does Bayesian Sufficiency relate to Frequentist Sufficiency?
Accepted answer
4 votes

If a statistic $T$ is sufficient in the frequentist way, then $p(\mathbf{x} \mid \theta, t) = p(\mathbf{x} \mid t)$, so \begin{align*} p(\theta \mid \mathbf{x}, t) &= \frac{p(\mathbf{x}\mid t,\...

View answer
Posterior latent $t_p$-distribution
Accepted answer
4 votes

Just use Bayes' rule. You're assuming these: $f_{Y|Z}(y \mid z) \propto z^{p/2}\det(\Sigma)^{-1/2} \exp\left[-\frac{z}{2}(y - \mu)^t\Sigma^{-1}(y-\mu) \right]$ $f_Z(z) \propto z^{\nu/2 - 1}\exp\left[...

View answer
Distribution of extreme values, case of uniform
Accepted answer
4 votes

For $z \in [0,n]$ \begin{align*} P(Z_n \le z) &= 1-P\left(U_{(n)} \le 1 - \frac{z}{n}\right) \\ &= 1 - \prod_{i=1}^nF_{U_i}\left( 1 - \frac{z}{n}\right) \\ &= 1 - \left(1 - \frac{z}{n}\...

View answer
If $X \sim {\rm Binomial}(N,p)$, what is the distribution of $X/N$?
Accepted answer
4 votes

Adding a little bit to the hint above: $$ P(X=k) = P(X/n = k/n). $$

View answer
What is the difference between converges in probability and distribution
4 votes

Intuitively, convergence in probability means the random variables get close to a nonrandom constant, and convergence in distribution means that it gets close to another random variable. Closeness ...

View answer
How can the AIC or BIC be used instead of the train/test split?
Accepted answer
4 votes

In chapter 5.5 of this book, they discuss how a lot of these model selection criteria arise. They start with Akaike's FPE criterion for AR models, and then go on to discuss AIC, AICc and BIC. They ...

View answer
Total sum of squares(TSS) is not equal ESS +RSS, when the model doesn't include intercept of ones
4 votes

To expand on @hxd1011's linked-to answer in the comments, \begin{align*} \text{TSS} &= \sum_i(y_i - \bar{y})^2 \\ &= \sum_{i}(y_i - \hat{y}_i + \hat{y}_i - \bar{y})^2\\ &= \sum_{i}(y_i - \...

View answer
Law of total variance and feature selection
Accepted answer
4 votes

If you predict $y$ by taking the conditional mean of it (with respect to one or both of the $x_i$s), then it will always be true that $$ \text{Var}[E(y \mid x_1)] \le \text{Var}[E(y \mid x_1, x_2)], $...

View answer
Covariance of $a \otimes b$ for independent random variables $a$ and $b$
Accepted answer
4 votes

First $$ E[\mathbf{c}] = E[\mathbf{a}\otimes \mathbf{b}] = E[\mathbf{a}] \otimes E[\mathbf{b}] = \mu_a \otimes \mu_b $$ by independence. Then \begin{align*} \operatorname{Var}[\mathbf{c}] &= E\...

View answer
Why does an (Augmented) Dickey Fuller test apply in an ARMA situation when it (seems to) assume an $AR(p)$ situation?
4 votes

The null hypothesis for DF test is a type of non-stationarity. $H_0: \phi_1 = 1$ where $X_t - \mu = \phi_1(X_{t-1}-\mu) + Z_t$. The null hypothesis for ADF test is a type of non-stationarity as well, ...

View answer
ACF and differencing order of operations
Accepted answer
4 votes

You look at ACF and PACFs of the differenced series. This is because these are tools for looking at stationary processes. You mention that the undifferenced values have a mean increasing over time......

View answer
Proof that the floor of an exponential random variable is a geometric variable
Accepted answer
4 votes

Here are a couple hints: \begin{align*} P([X] = x) &= P(x \le X < x+1) \tag{logic} \\ &= P(x < X \le x+1) \tag{$X$ is a continuous rv} \end{align*} for any non-negative integer $x$.

View answer
Differencing a random walk process in time series
4 votes

A random walk process is defined as $$ X_t = X_{t-1} + Z_t, $$ where $\{Z_t\}$ is white or IID noise. Using just pencil and paper, you can see that subtracting $X_{t-1}$ from both sides tells you the ...

View answer
Can we think of a Random Variable as an instantiation of its distribution?
4 votes

Yes it is a value, but no it doesn't necessarily have to be realized. A random variable can be realized or unrealized. Just as a house can be built or unfinished. The analogy is meant to emphasize ...

View answer
Variance of Minimum and Maximum of 2 iid Normal
4 votes

If you can convince yourself that $$ \max(X,Y) \overset{d}{=} -\min(X,Y), $$ then taking the variance on both sides will give you your answer. Regarding the other part, you'll probably have to ...

View answer
Maximum Likelihood as an Expectation
4 votes

The answer by Dougal does a good job explaining the empirical distribution. This answer just highlights more the connection between when expectations are sums, and when they are integrals, addressing "...

View answer
Conditioning a biased estimator on a sufficient statistic
Accepted answer
4 votes

If you Rao-Blackwellize a biased estimator, the bias does not change, and the variance potentially shrinks. The result doesn't change much. Let $U$ be the estimator for $h(\theta)$. Let $b = E[U]-h(\...

View answer
Expected value of continuous probability distribution
Accepted answer
4 votes

I'm not quite familiar with statsmodels, but if you have data values, $x_1, \ldots, x_n$, and probabilities of these values $p_1, \ldots, p_n$ given to you by the function distribution.pdf(), then an ...

View answer
Method of moment estimates for n Bernoulli trials
Accepted answer
4 votes

Hint: "method of moments" means you set sample moments equal to population/theoretical moments. For example, the first sample moment is $\bar{X} = n^{-1}\sum_{i=1}^n X_i$, and the second sample ...

View answer
1 2
3
4 5
17