The tag has no wiki summary.

learn more… | top users | synonyms

0
votes
1answer
17 views

Proof that points change clusters less often as iterations proceed in k means

Is there a way that to prove the following: In k-means clustering, as the iterations proceed, the data points tend to stay in their existing clusters, overall, because the replacement of the centroid ...
4
votes
0answers
38 views

“Only if” and “if” direction in Kolmogorov's Existence Theorem

The title of this question sounds somewhat grand, but all I'm asking is whether there's a typo in the book I'm reading. The book is "A First Look at Rigorous Probability Theory," by Jeffrey Rosenthal ...
1
vote
1answer
58 views

Time-series and autocorrelation inequality

I am having problems proving for a weakly stationary process $\{X_t : t\in T\}$: $\rho_X(2)\geq 2 (\rho_X(1))^2-1$ where $\rho_X(j)=corr(X_t, X_{t+j})$. So far I have shown that $-1\leq ...
2
votes
0answers
34 views

Issue with the proof of PCA

I found a very nice PCA proof over here PCA_proof and I'm trying to understand it (I don't know what Langrange multipliers are so I'm trying my best). From the second page of the previous link, ...
7
votes
3answers
75 views

Proof that the probability of one RV being larger than $n-1$ others is $\frac{1}{n}$

This is a follow-on from my previous question about samples from a distribution. Suppose $X_1 \ldots X_{n-1}, X_n$ are random variables all following some fixed distribution $D$. How do I prove that ...
2
votes
0answers
110 views

What's the Mode of a Bivariate Poisson Distribution?

I have been looking at the bivariate Poisson distribution and I am wondering if there is close form expression for the mode of this distribution. I know the mode of the univariate Poisson distribution ...
0
votes
0answers
10 views

Are these three different ways of expressing the optimal value function $V^*$ the same? (reinforcement learning)

My question didn't really fit on the title but its the following are the three following equations actually the same: $$V^*(s) = \underset{\pi}{max}V^{\pi}(s)$$ and $$V^*(s)=R(s)+ \underset{a \in ...
0
votes
1answer
14 views

prove that SVM chooses the bisecting line of nearest support vectors?

I have trouble solving the problem 3.18 from "pattern recognition" by "Sergios Theodoridis, ‎Konstantinos Koutroumbas" the problem is : Show that for the case of two linearly separable classes the ...
5
votes
1answer
68 views

Explanation of formula for median closest point to origin of N samples from unit ball

In Elements of Statistical Learning, a problem is introduced to highlight issues with k-nn in high dimensional spaces. There are $N$ data points that are uniformly distributed in a $p$-dimensional ...
2
votes
0answers
74 views

proving regression with dummy variables gives same estimates as separate models

Let ($x_{i1}$, $x_{i2}$, ..., $x_{id}$, $y_i$), $i = 1,..., n$ be an i.i.d. multivariate sample and furthermore assume each observation belongs to one of possible $K$ categories. Assume for each ...
1
vote
1answer
46 views

How to prove Bernoulli distribution belongs to the exponential family

According to a book, a distribution belongs to the exponential family if it can be written in the form of I wrote the Bernoulli distribution as $\exp\Big(y \log\,[{\mu}/{(1-\mu)}] + ...
13
votes
3answers
352 views

Uniform random variable as sum of two random variables

Taken from Grimmet and Stirzaker: Show that it cannot be the case that $U=X+Y$ where $U$ is uniformly distributed on [0,1] and $X$ and $Y$ are independent and identically distributed. You should not ...
9
votes
2answers
237 views

Is there an elegant/insightful way to understand this linear regression identity for multiple $R^2$?

In linear regression I have come across a delightful result that if we fit the model $$E[Y] = \beta_1 X_1 + \beta_2 X_2 + c,$$ then, if we standardize and centre the $Y$, $X_1$ and $X_2$ data, ...
2
votes
1answer
20 views

Gradient of expected transformed outcome with respect to distribution parameters

If $x$ has density $p(x \mid \theta)$ why is it true that for some function $f$, $$\begin{align} \nabla_{\theta}\mathbb{E}[f(x)] &= \mathbb{E}[f(x) \nabla_\theta p(x)] \end{align}$$
1
vote
1answer
35 views

Sample covariance mean-corrected vector proof

Prove that $$(n-1)S = X^TX -{1\over{n}}(X^T\vec1)(\vec1^TX) = X^TX-n\vec{\bar x}\vec{\bar x}^T$$ My attempt so far goes like this $$S = {1\over{n-1}}X_m^TX_m$$ Edit: Where $X_m$ is the ...
2
votes
1answer
64 views

Rewrite instrumental variables estimator into formula with covariances?

In the book Microeconometrics of Cameron and Trivedi, they write the IV estimator as $\widehat{\beta}_{IV} = \frac{Cov[z,y]}{Cov[z,x]}$, formula (4.49) on p. 99. They say that they derived this from ...
3
votes
2answers
54 views

Poisson as a limiting case of negative binomial

I was reading "Maximum Likelihood Estimation for the Negative Binomial Dispersion Parameter" by Walter W. Pieogorsch, and in the intro it says the Poisson distribution is a limiting case of negative ...
8
votes
1answer
74 views

Is the negative binomial not expressible as in the exponential family if there are 2 unknowns?

I had a homework assignment to express the negative binomial distribution as an exponential family of distributions given that the dispersion parameter was a known constant. This was fairly easy, but ...
6
votes
2answers
224 views

Proving the LATE Theorem of Angrist and Imbens 1994

Assume we have a binary instrument $Z_i$ which can be used to estimate the effect of the endogenous variable $D_i$ on the outcome $Y_i$. Suppose the instrument has a significant first stage, it is ...
1
vote
1answer
59 views

Why constrain mean and standard deviation when proving Gaussian is maximum differential entropy pdf?

I'm reading Bishop's Pattern Recognition and Machine Learning. In chapter 1.6: Information Theory (page 53) when trying to derive the maximum differential entropy pdf from the definition of continuous ...
1
vote
0answers
60 views

Average within-cluster distance using divisive clustering

I have to prove that the average within-cluster distance for 10 data points cannot increase when going from 1 cluster to 2 clusters (divisive clustering). Intuitively, it seems obvious that this is ...
1
vote
3answers
80 views

Proving Linear Estimator (beta) is BLUE?

In the book Statistical Inference pg 570 of pdf, There's a derivation on how a linear estimator can be proven to be BLUE. I got all the way up to 11.3.18 and then the next part stuck me. After ...
3
votes
1answer
42 views

Equivalence of random effects via likelihood and smoothed splines

Some fake data: X = runif(1000) ff = rep(1:10,100) E = rnorm(1000) y = x+e+f f = as.factor(ff) When you fit a model like ...
0
votes
2answers
39 views

Stuck on a proposition

Suppose four numbers $\{a,b,c,d\}$ where $a$ and $c$ random variables from a continuous distribution with support on $\mathbb{R}$. Does $b\neq d$ imply $|a−b|−|c−d|+a−c\neq 0$ almost surely? I ...
29
votes
6answers
2k views

How can I analytically prove that randomly dividing an amount results in an exponential distribution (of e.g. income and wealth)?

In this current article in SCIENCE the following is being proposed: Suppose you randomly divide 500 million in income among 10,000 people. There's only one way to give everyone an equal, 50,000 ...
4
votes
0answers
320 views

Cantelli's inequality proof

I am trying to prove the following inequality: EDIT: Almost immediately after I posted this question, I discovered that the inequality I am being asked to prove is called Cantelli's inequality. When ...
1
vote
1answer
80 views

Proof of asymptotic variance

How do you prove that $X_n - E[X_n] = O_p(\sqrt{Var(X_n)})$ It's used in my textbook and I don't know where they get it from.
8
votes
1answer
492 views

Proof / derivation of skewness and kurtosis formulas

Can anyone explain to me where the formula of skewness or kurtosis comes from? (I mean its derivation.) What's the logic behind it? Who proved it?
1
vote
2answers
156 views

Multivariate Bayesian formula

I got there example graphs bishop's PRML (8.2.1) 1. a <- c -> b $$ p(a,b,c) = p(a|c)p(b|c)p(c) --(1)\\ p(a,b) = \sum_c p(a|c)p(b|c)p(c) --(2) $$ Q1: Can I use a new graph to represent the ...
1
vote
1answer
51 views

[Revised]Proving the expected \bold{density} of being the Nth order statistics is decreasing in sample size

(Sorry that I've previously formulated the question in a wrong way, which confused everyone including myself. This is a better version of the question. Thanks!) Here's another order statistics ...
3
votes
2answers
142 views

Proving some properties of expected first order statistics with respect to sample size

Question: Consider $n$ random variables $x_1, x_2,\cdots x_n\sim \mathcal{N}(0,1)$. The expected value of the $i$th order statistic (the maximum) can be written as $E(\mathcal{O}^n_1)= ...
1
vote
1answer
55 views

Differentiating Shannon's entropy [closed]

Can somebody please show the steps of how differentiation of Shannon's entropy yields the following result? $H = -\sum_{l=0}^{L-1} p(l)\log_2[p(l)]$ The result of differentiating is $H_m = ...
7
votes
2answers
598 views

Deriving the bivariate Poisson distribution

I've recently encountered the bivariate Poisson distribution, but I'm a little confused as to how it can be derived. The distribution is given by: $P(X = x, Y = y) = ...
0
votes
1answer
78 views

Derive the mean of a discrete probability distribution

I am reading inference statistics by casella and berger. They are deriving the general formula for the probability distribution like that: ...
5
votes
2answers
102 views

How would one formally prove that the OOB error in random forest is unbiased?

I have read this statement many times but have never come across a proof. I would like to try to produce one myself but I'm not even sure on what notation to use. Can anyone help me with this?
0
votes
0answers
71 views

PCA proof needed for proportion of variance explained by L PCs = mean R-square from regression on PC scores

I observed the following relation and would like to know where I can find a general proof for this: Assume a data matrix $A = [a_{ij}]_{t x k}$. 1) Perform principal component analysis (PCA) using ...
2
votes
1answer
130 views

If a random variable V is independent of two independent random variables X and Y, how to prove that V is independent of X + Y?

This is question 3.8.4 of An Introduction to Mathematical Statistics and Its Applications, 5th Edition, by Larsen and Marx. This is not homework for a class I am taking now, but might someday be for ...
2
votes
1answer
98 views

Justification for the Bootstrap Percentile Interval

The following is a proof of the validity of the (bootstrap) percentile interval taken from Larry Wasserman's "All of Statistics". $\theta^*_{\alpha/2}$ and $\theta^*_{1-\alpha/2}$ denote the ...
2
votes
0answers
27 views

Independent event proof

I'm doing a stats past paper for my first year exam, and I'm having some trouble answering the following question. Could someone please help me? Thanks 'Show that if 3 events $\{A, B, C\}$ are ...
24
votes
3answers
2k views

How exactly did statisticians agree to using (n-1) as the unbiased estimator for population variance without simulation?

The formula for computing variance has $(n-1)$ in the denominator: $s^2 = \frac{\sum_{i=1}^N (x_i - \bar{x})^2}{n-1}$ I've always wondered why. However, reading and watching a few good videos about ...
4
votes
2answers
339 views

Independence of a linear and a quadratic form

How can I prove the following lemma? Let $\mathbf{X}^ \prime$ = $ \left[ X_1 , X_2 , \ldots, X_n \right]$ where $ X_1, X_2, \ldots X_n $ are observations of a random sample from a distribution which ...
3
votes
1answer
72 views

Proof that the weighted sum of $n$ PDFs is a valid PDF

Let $f_i(y)$ for $i = 1, \ldots, n$ be valid PDF’s, and let $a_i ∈ (0, 1)$ be constants, such that $\sum_{i=1}^n a_i= 1$. Show that the function $f(y) = \sum_{i=1}^n a_i\, f_i(y)$ is a valid PDF. If ...
0
votes
0answers
33 views

Conditional independence proof seems too obvious, am I looking at it incorrectly?

I'm given the following question about conditional independence: "Suppose we have four random variables a, b, c, d. Prove that, if a is conditionally independent of b and c given d, then a is ...
0
votes
1answer
235 views

Why does eigenvalue decomposition of a correlation matrix maximizes possible variance?

I was reading up on principal component analysis and I was wondering how does eigenvalue decomposition of the correlation matrix maxmimizes the possible variance that is captured? Can someone refer to ...
2
votes
0answers
135 views

Prove that this exponential kernel is positive definite

Let $x,y\in R^d$ and $d:R^d\times R^d \rightarrow R$ a metric on $R^d$ be given. The exponential kernel is defined by: $k(x,x')=e^{−αd(x,x')}$ where $α>0$. The kernel matrix is defined as the ...
7
votes
1answer
107 views

Coverage Proof of Confidence Intervals

The confidence interval for the mean of a random variable $Y$ has coverage $1-\alpha$ which I am trying to show. Starting from $$\widehat{E(Y)} - ...
0
votes
0answers
72 views

Finding parameter bias under omitted variable, with variance covariance notation

Dear CrossValidated community, Can anyone help me to prove the bias in a given parameter of a regression when there is omitted variable? I know to do it using matrices and matrix algebra. For ...
7
votes
4answers
660 views

Prove the equivalence of the following two formulas for Spearman correlation

From wikipedia, Spearman's rank correlation is calculated by converting variables $X_i$ and $Y_i$ into ranked variables $x_i$ and $y_i$, and then calculating Pearson's correlation between the ranked ...
1
vote
0answers
210 views

Proof that omitted variable bias may lead to endogeneity

I am looking for a proof that omitted variable bias (OVB) in OLS regression may lead to endogeneity. I have found many examples here and out there on how to prove that a given parameter $b_{j}$ (where ...
6
votes
1answer
141 views

Quantile regression estimator formula

I have seen two different representations of the quantile regression estimator which are $$Q(\beta_{q}) = \sum^{n}_{i:y_{i}\geq x'_{i}\beta} q\mid y_i - x'_i \beta_q \mid + \sum^{n}_{i:y_{i}< ...