rightskewed
  • Member for 9 years, 6 months
  • Last seen this week
6 answers
21 votes
5k views
Completing a 3x3 correlation matrix: two coefficients of the three given
Accepted answer
24 votes

We already know $\gamma$ is bounded between $[-1,1]$ The correlation matrix should be positive semidefinite and hence its principal minors should be nonnegative Thus, \begin{align*} 1(1-\gamma^2)-0.6(...

View answer
2 answers
5 votes
180k views
How Can I Calculate Standard Deviation (step-by-step) in R?
Accepted answer
17 votes

> a <- c(179,160,136,227) > sd(a) [1] 38.57892 > sqrt(sum((a-mean(a))^2/(length(a)-1))) [1] 38.57892 ```

View answer
1 answers
15 votes
14k views
Difference between multilevel modelling and mixed effects models?
14 votes

Section 2.2.2.1 from lme4 book Because each level of sample occurs with one and only one level of batch we say that sample is nested within batch. Some presentations of mixed-effects models, ...

View answer
1 answers
10 votes
9k views
Random variable with zero variance
13 votes

$E[(X-E[X])^2] =0 \implies X = E[X]$ Thus $X$ is almost surely constant. A better description for such random variables is that it follows a degenerate distribution.

View answer
1 answers
8 votes
1k views
np package kernel density estimation with Epanechnikov kernel
Accepted answer
7 votes

EDIT This is explained in the FAQ: I use plot() (npplot()) to plot, say, a density and the resulting plot looks like an inverted density rather than a density This can occur when the ...

View answer
2 answers
13 votes
3k views
Optimizing a Support Vector Machine with Quadratic Programming
Accepted answer
7 votes

HINT: Quadprog solves the following: $$ \begin{align*} \min_x d^T x + 1/2 x^T D x\\ \text{such that }A^T x \geq x_0 \end{align*} $$ Consider $$ x = \begin{pmatrix} w\\ b \end{pmatrix} \text{and } ...

View answer
1 answers
2 votes
570 views
Find the posterior density of theta, given prior with exponential density and samples from normal distribution
Accepted answer
6 votes

HINT: \begin{align*} \exp\big(-\frac{\theta}{\kappa_0}-\frac{(\bar{t}-\theta)^2}{2\frac{\sigma^2}{n}}\big) &= \exp\big( -\frac{n}{2\sigma^2} \big((\bar{t}-\theta)^2 + 2\frac{\sigma^2 \theta}{n \...

View answer
2 answers
5 votes
633 views
Recommendations for books regarding statistical consulting
6 votes

Statistical Sleuth does not describe the consulting process, but teaches methods using case studies. To quote: The Sleuth was written to train graduate students in disciplines other than Statistics ...

View answer
3 answers
9 votes
349 views
What's higher, $E(X^2)^3$ or $E(X^3)^2$
5 votes

Lyapunov's Inequality (See: Casella and Berger, Statistical Inference 4.7.6): For $1 < r < s < \infty$: $$ \mathbb{E}[|X|^r]^\frac{1}{r} \leq \mathbb{E}[|X|^s]^\frac{1}{s} $$ Proof: By ...

View answer
2 answers
2 votes
3k views
R How to find the secondary peak of a distribution
Accepted answer
5 votes

You can use mixture models to capture the biomodality library(flexmix) set.seed(42) D <- c(rnorm(100,1,1), rnorm(100,5,1)) kde <- density(D) m1 <- FLXMRglm(family = "gaussian") m2 <- ...

View answer
1 answers
0 votes
1k views
What is meant by "scale family of densities"?
5 votes

Definition 3.5.4 from Casella & Berger: Let $f(x)$ be any pdf. Then for any $\sigma >0$, the family of pdfs $(1/\sigma) f(x/\sigma)$, indexed by the parameter $\sigma$, is called the scale ...

View answer
1 answers
2 votes
474 views
UMVUE of $p^4$ when $X_1,\ldots,X_n$ is a sample from Bernoulli$(p)$
Accepted answer
5 votes

Define $T=\sum X_i$ T is a complete sufficient statistic for $p$. Now, consider indicator $I_{X_1=1,X_2=1,X_3=1,X_4=1}$ which is an unbiased estimator of $p^4$(As you proved in the first part) Rao-...

View answer
1 answers
7 votes
275 views
Runs of the same type within a deck of cards - distribution of runs of different length
4 votes

Let $I_i^c$ be an indicator variable such that: $$ I_i^c = \begin{cases} 1 & \text{run of length $c$ starts at $i^{th}$ position}, \\ 0 & \text{otherwise} \end{cases} $$ To find $\mathbb{E}[...

View answer
1 answers
0 votes
1k views
Missing values imputation of time series using na.kalman command
Accepted answer
4 votes

You need to pass it the model and not a string: ts1=arima(TP, order = c(1,1,0), xreg=F) na.kalman(TP, model = ts1$model)

View answer
1 answers
2 votes
87 views
Alternative definitions of posterior probability
Accepted answer
4 votes

$$P(D|S) \quad \underline{deff} \quad ∑_iP(D|M_i)·P(M_i|S) \tag{1}$$ is just a consequence of law of total probability: $\begin{align*} P(D|S) &= \sum_{i}P(D,M_i | S)\\ &= \sum_{i} \frac{P(D,...

View answer
2 answers
4 votes
55 views
Is there an equivalent to Fourier decomposition using normal distributions instead of sin / cos?
4 votes

You might want to try out Gaussian Mixture models for your data. For example, to decompose a mixture of $\mathcal{N}(10, 5), \mathcal{N}(22, 3)$, using flexmix package library(flexmix) set.seed(42) ...

View answer
1 answers
1 votes
4k views
EM algorithm for a binomial distribution
Accepted answer
4 votes

Let $\pi_A = \pi_B = \frac{1}{2}$ represent the probability of selecting coin A and B respectively. Observed data: $\{\mathbf{X_1}, \mathbf{X_2}, \mathbf{X_3}, \mathbf{X_4}, \mathbf{X_5} \}$ ...

View answer
1 answers
9 votes
165 views
Distribution of quadratic form of normals
4 votes

Here is an attempt: Consider $Z=X-Y$ such that $X \sim \chi^2(\alpha)$ and $Y \sim \chi^2(\beta)$ with $\alpha \geq \beta$ $$ \mathcal{M}_X(t) = \left(1-2 \, t\right)^{-\alpha/2} $$ $$ \...

View answer
1 answers
0 votes
68 views
Can anyone identify this distribution?
Accepted answer
4 votes

$$ \begin{align} f(x) = c\ \exp({-x^2-\frac{x}{4}}) &= c\ \exp({-(x+\frac{1}{8})^2+\frac{1}{64}})\\ &= c' \exp(-\frac{y^2}{2})\ \forall -\infty < y < \infty \end{align} $$ where $c'=c\ \...

View answer
1 answers
3 votes
441 views
Displaying large number of text labels on a scatter plot
Accepted answer
3 votes

Using ggvegan and ggrepel library("vegan") library("ggvegan") library("cowplot") library("ggrepel") data(varespec, varechem) vare.cca <- cca(varespec, varechem) obj <- fortify(vare.cca) want &...

View answer
1 answers
3 votes
27 views
Deviation due to conditioning
3 votes

$E[E[A|B]]=E[A]$ and so the expression is simply the variance of $E[A|B]$ \begin{align*} \mathrm{Var}(A) &= E[\mathrm{Var}(A|B)] + \mathrm{Var}(E[A|B])\\ \mathrm{Var}(E[A|B]) &= \mathrm{Var}(A)...

View answer
1 answers
1 votes
33 views
$X_k$ are $\mathcal U(0,2\theta)$ distributed, and $Y_n=\max_{1\leq k < n}X_k$, how is $F_{Y_n}(t)=(\frac{t-\theta}{\theta})?$
Accepted answer
3 votes

It is indeed incorrect. $F_{Y_n}(t) = P( \max_{i} X_i \leq t) = P(X_1,X_2, \dots X_n \leq t) $ Assuming $X_i$ are independent and not simply identically distributed: $P(X_1,X_2, \dots X_n \leq t) =...

View answer
2 answers
3 votes
2k views
A good step by step second graduate course textbook for a mathematical statistics textbooks
3 votes

Statistical Inference by Casella et al. is often used as a primary textboook.

View answer
1 answers
2 votes
161 views
Can I approximate the variance of a ratio to the ratio of two variances?
Accepted answer
3 votes

$Var( X_i\epsilon_i)= E[(X_i\epsilon_i)^2]-(E[X_i\epsilon_i])^2 =EX_i^2 \times E\epsilon_i^2-0=(\mu^2+\tau^2)\sigma^2$ Corrected formula: $$Var(\frac{U}{V}) \approx (\frac{E[U]}{E[V]})^{2}\cdot(\frac{...

View answer
2 answers
5 votes
191 views
Assessing the relationship between continuous variables
Accepted answer
3 votes

There are examples of using Multiple Linear Regression for similar studies[1] Here is an notebook example of doing this in R. [1] Genomic ancestry and somatic alterations correlate with age at ...

View answer
1 answers
1 votes
85 views
How can I find the distribution of the quantity $T(X,Y)$ given a sample from a bivariate normal distribution?
Accepted answer
2 votes

Consider $D_i = X_i - Y_i$. Then \begin{align*} D_i &\sim \mathcal{N}\left(\mu_1-\mu_2, 2\sigma^2(1-\rho)\right)\\ \implies \bar{D} &\sim \mathcal{N}\left(\mu_1-\mu_2, \frac{2\sigma^2}{n}(1-\...

View answer
1 answers
0 votes
556 views
UMVUE explanations
Accepted answer
2 votes

Naive way to solve $ii$: Observation 1: $\sum X_i$ is complete and sufficient statistic for $\theta$ Observation 2: $\sum X_i \sim Poisson(n\theta)$ We need to look for an unbiased estimator of $\...

View answer
1 answers
6 votes
926 views
Critical region of likelihood ratio test
Accepted answer
2 votes

$\chi^2_1(0.95) = 3.841$ $C = \{\mathbf{Y}: (Y_1+3Y_2) \log{\frac{p_0}{\hat{p}}} + (Y_1+3Y_0) \log(\frac{1-p_0}{1-\hat{p}}) \geq \frac{-\chi^2_1(0.95)}{2} \}$ When $Y_0 =Y_2$, $\hat{p} = 1/2$ Thus, ...

View answer
1 answers
3 votes
675 views
EM with Binomial
Accepted answer
2 votes

Define $p = P(Z_i > u) = P( \frac{Z-\mathcal{\epsilon}}{\sqrt{\sigma^2+1}} > \frac{u-\mathcal{\epsilon_i}}{\sqrt{\sigma^2+1}}) = \phi(\frac{\mathcal{\epsilon_i}-u}{\sigma{\sigma^2+1}})$ The ...

View answer
2 answers
2 votes
612 views
Calculate similarity between assortment of grocery shopping basket
2 votes

Jaccard Index is often used to calculate similarity of such sample sets. Let's assume there are 4 products $P_1,P_2,P_3,P_4$ that can be bought offline or online. So if $S_{off} = [1,0,1,1]$ and $S_{...

View answer