For mathematical proofs or derivations of results.

learn more… | top users | synonyms

0
votes
0answers
20 views

Can any distribution be fully described by the [infinite] set of all its moments? $\{E[x^n]\}_{n\in N}$ [duplicate]

Is it possible describe any distribution uniquely by the infinite set of all it's moments $\{E[x^n]\}_{n\in N}$ If yes, does that include discrete, truncated, etc. distributions? If not, is this ...
0
votes
1answer
28 views

Regression Tree Impurity

It seems pretty obvious that if I currently have a tree with a certain total impurity, then by splitting it again optimally, I can never end up with a greater total impurity. It seems similar to the ...
0
votes
1answer
19 views

Variance of Wilcoxon Rank-Sum Test (no ties)

$\newcommand{\E}[1]{\mathbb{E}\left[#1\right]} \newcommand{\r}[1]{r\left(#1\right)} \newcommand{\P}[1]{\mathbb{P}\left(#1\right)} \newcommand{\Var}[1]{\text{Var}\left[#1\right]}$ Given two sets of ...
0
votes
1answer
29 views

Where can I find a clear derivation of backpropagation through a Convolutional Neural Network?

Any links to books, articles or papers would be appreciated, or even a written explanation.
-1
votes
1answer
37 views

Expectation of a random variable and the indicator random variable proof

I need to show that $E[T 1_A] = E[T|A]P(A)$. What I've got so far is $E[T1_A] = \int_{\Omega}T.1_A dP = \int_AT dP$. Now I know that $\int_AdP = P(A)$ however I'm lost as to how do I get the ...
6
votes
4answers
814 views

How to Prove that an Event Occurs Infinitely Often (Almost Surely)?

Exercise: There is a fair 6-sided die and a biased coin that has probability p > 0 of coming up heads on each toss. The die gets rolled infinitely often, and whenever you roll a 6, you then ...
4
votes
1answer
42 views

Partition data into two sets such that the difference of their variance is minimal

Suppose there are $n$ data values $x_1<x_2<\ldots<x_{n-1}<x_n$,and I've found a partition number $k$, such that $$ ...
2
votes
1answer
33 views

Conceptual proof that conditional of a multivariate Gaussian is multivariate Gaussian

I understand the arithmetic derivation of the PDF of a conditional distribution of a multivariate Gaussian, as explained here, for example. Does anyone know of a more conceptual (perhaps, co-ordinate ...
0
votes
4answers
46 views

Variance of linear combinations of correlated random variables

I understand the proof that $$(aX+bY) = a^2Var(X) +b^2Var(Y) + 2abCov(X,Y), $$ but I don't understand how to prove the generalization to arbitrary linear combinations. Let $a_i$ be scalars for ...
3
votes
1answer
40 views

How to prove the properties of penalized likelihood estimator in Fan and Li (2001) paper

I'm reading through Fan and Li's paper "Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties". In Page 2 near bottom right corner, they proposed three properties that a ...
1
vote
0answers
64 views

Mathematics / Statistics [closed]

Today I was thinking how I'm bad at making statements, I know who does statistical cares more about the practical part, but knowing how to do demonstrations in the theoretical part is important? I ...
4
votes
1answer
51 views

Generalized inverse solution to system of linear equations proof

I'm going through a set of course notes for an introduction to the theory of linear models class at my university. Unfortunately, the professor who wrote this note set is no longer at this school, and ...
4
votes
2answers
88 views

Is $\text{Cov}(|a|,|b|)\geq \text{Cov}(a,b)$?

The above seems intuitively true, (where $|a|$ refers to the absolute value of $a$), but I'm struggling to prove it - would be very grateful for either a proof or a reference.
1
vote
1answer
41 views

Proof for the sampling variance of the Neyman Estimator

I'm going through Imbens and Rubin's new book and I just for the life of me can't figure out 1 minor detail in their proof for the sampling variance of the Neyman estimator $\bar{Y}^{obs}_{t} - ...
3
votes
1answer
106 views

Transformation of a random vector

I am very familiar with the formula $$f_{Y}(y) = f_{X}(x(y))\left|\dfrac{\text{d}}{\text{d}y}[x(y)]\right|$$ where $X$ and $Y = g(X)$ is a one-to-one transformation of $X$. (I use $x(y)$ to mean "$x$ ...
6
votes
2answers
131 views

A strange step on a proof about the distribution of quadratic forms

The following theorem comes from the 7th edition of "Introduction to Mathematical Statistics" by Hogg, Craig and Mckean and it concerns the necessary and sufficient condition for the independence of ...
2
votes
0answers
43 views

Proving a “well-known” result regarding the distribution of a normally distributed random variable

In an important project work, I would like to include a "proof" of the following, but have unfortunately been unable to readily compute it myself. I am aware that this is a flaw on my part, but ...
3
votes
1answer
45 views

Is a covariance matrix composed of matrixes derived from separate samples guaranteed to be positive definitive?

I have two samples that partially overlap on the variables they describe. The samples are taken from more or less the same population, and show similar values on the overlapping variables. Based on ...
1
vote
1answer
59 views

Don't understand identity in proof for unbiased sample variance

Wikipedia gives the following proof why to use Bessel's correction for the unbiased sample variance: \begin{align} E[\sigma_y^2] & = E\left[ \frac 1n \sum_{i=1}^n \left(y_i - \frac 1n ...
0
votes
1answer
23 views

Mini-batch Stochastic Gradient Descent: Question about proof in paper

I'm going through the JMLR paper on distributed mini-batch Stochastic Gradient Descent and I have a question about a part of the proof. They use the notation: $$ F(w) = \mathop{\mathbb{E}}_{z}[f(w, ...
0
votes
1answer
31 views

Random variables, inequalities involving them and medians thereof (II)

I have a continuous function $A$ for which it holds that: $$x\in S\implies A(x)<0$$ $$x\in T\implies A(x)>0$$ Of course, $S$ and $T$ don't intersect. Now consider two continuous ...
1
vote
1answer
57 views

Random variables, inequalities involving them and medians thereof

I have three facts which I know to be true and I am wondering if together they logically imply a fourth one (or not!). Consider $F_x$ and $G_y$ two continuous, uni-modal distributions on ...
0
votes
0answers
19 views

Comparing regressions: usual regressor vs regressed-out regressor

I'm comparing the regression coefficients between 2 models: Model 1: $$ Y = \beta_1X_1 + \beta_2X_2 + u $$ Model 2: $$ Y = \beta_1'X_1' + \beta_2'X_2 + v $$ where $X_1' = ...
3
votes
0answers
93 views

Proof of Kolmogorov-Smirnov test

Could someone provide me a reference, preferably a book, where I can find detailed proofs and explanations of the Kolmogorov-Smirnov test (including the two-sample variant) and the derivation of the ...
0
votes
1answer
30 views

Proof that points change clusters less often as iterations proceed in k means

Is there a way that to prove the following: In k-means clustering, as the iterations proceed, the data points tend to stay in their existing clusters, overall, because the replacement of the centroid ...
5
votes
1answer
57 views

“Only if” and “if” direction in Kolmogorov's Existence Theorem

The title of this question sounds somewhat grand, but all I'm asking is whether there's a typo in the book I'm reading. The book is "A First Look at Rigorous Probability Theory," by Jeffrey Rosenthal ...
1
vote
1answer
74 views

Time-series and autocorrelation inequality

I am having problems proving for a weakly stationary process $\{X_t : t\in T\}$: $\rho_X(2)\geq 2 (\rho_X(1))^2-1$ where $\rho_X(j)=corr(X_t, X_{t+j})$. So far I have shown that $-1\leq ...
3
votes
1answer
116 views

Proving a distribution is a member of the simple exponential family

Does anyone have any tips/ideas/method for proving that a distribution is a member of the simple exponential family (SEF)? Or is the process unique to each distribution? For example, I am trying to ...
2
votes
0answers
68 views

Issue with the proof of PCA

I found a very nice PCA proof over here PCA_proof and I'm trying to understand it (I don't know what Langrange multipliers are so I'm trying my best). From the second page of the previous link, ...
7
votes
3answers
90 views

Proof that the probability of one RV being larger than $n-1$ others is $\frac{1}{n}$

This is a follow-on from my previous question about samples from a distribution. Suppose $X_1 \ldots X_{n-1}, X_n$ are random variables all following some fixed distribution $D$. How do I prove that ...
2
votes
0answers
137 views

What's the Mode of a Bivariate Poisson Distribution?

I have been looking at the bivariate Poisson distribution and I am wondering if there is close form expression for the mode of this distribution. I know the mode of the univariate Poisson distribution ...
0
votes
0answers
22 views

Are these three different ways of expressing the optimal value function $V^*$ the same? (reinforcement learning)

My question didn't really fit on the title but its the following are the three following equations actually the same: $$V^*(s) = \underset{\pi}{max}V^{\pi}(s)$$ and $$V^*(s)=R(s)+ \underset{a \in ...
3
votes
1answer
483 views

Expectation, Variance and Correlation of a bivariate Lognormal distribution

If $Y \sim N(\mu,\sigma^2)$ is normally distributed, then $X=\mathrm{e}^Y$ is lognormally distributed. To get the log-$\mu$ and log-$\sigma$ of this lognormal distribution you calculate $$\sigma^2 = ...
0
votes
1answer
21 views

prove that SVM chooses the bisecting line of nearest support vectors?

I have trouble solving the problem 3.18 from "pattern recognition" by "Sergios Theodoridis, ‎Konstantinos Koutroumbas" the problem is : Show that for the case of two linearly separable classes the ...
5
votes
1answer
179 views

Explanation of formula for median closest point to origin of N samples from unit ball

In Elements of Statistical Learning, a problem is introduced to highlight issues with k-nn in high dimensional spaces. There are $N$ data points that are uniformly distributed in a $p$-dimensional ...
2
votes
0answers
108 views

proving regression with dummy variables gives same estimates as separate models

Let ($x_{i1}$, $x_{i2}$, ..., $x_{id}$, $y_i$), $i = 1,..., n$ be an i.i.d. multivariate sample and furthermore assume each observation belongs to one of possible $K$ categories. Assume for each ...
10
votes
1answer
131 views

Show that if $X \sim Bin(n, p)$, then $E|X - np| \le \sqrt{npq}.$

Currently stuck on this, I know I should probably use the mean deviation of the binomial distribution but I can't figure it out.
2
votes
1answer
80 views

How to prove Bernoulli distribution belongs to the exponential family

According to a book, a distribution belongs to the exponential family if it can be written in the form of I wrote the Bernoulli distribution as $\exp\Big(y \log\,[{\mu}/{(1-\mu)}] + ...
13
votes
2answers
416 views

Uniform random variable as sum of two random variables

Taken from Grimmet and Stirzaker: Show that it cannot be the case that $U=X+Y$ where $U$ is uniformly distributed on [0,1] and $X$ and $Y$ are independent and identically distributed. You should not ...
9
votes
2answers
278 views

Is there an elegant/insightful way to understand this linear regression identity for multiple $R^2$?

In linear regression I have come across a delightful result that if we fit the model $$E[Y] = \beta_1 X_1 + \beta_2 X_2 + c,$$ then, if we standardize and centre the $Y$, $X_1$ and $X_2$ data, ...
2
votes
1answer
23 views

Gradient of expected transformed outcome with respect to distribution parameters

If $x$ has density $p(x \mid \theta)$ why is it true that for some function $f$, $$\begin{align} \nabla_{\theta}\mathbb{E}[f(x)] &= \mathbb{E}[f(x) \nabla_\theta p(x)] \end{align}$$
2
votes
1answer
43 views

Sample covariance mean-corrected vector proof

Prove that $$(n-1)S = X^TX -{1\over{n}}(X^T\vec1)(\vec1^TX) = X^TX-n\vec{\bar x}\vec{\bar x}^T$$ My attempt so far goes like this $$S = {1\over{n-1}}X_m^TX_m$$ Edit: Where $X_m$ is the ...
2
votes
1answer
77 views

Rewrite instrumental variables estimator into formula with covariances?

In the book Microeconometrics of Cameron and Trivedi, they write the IV estimator as $\widehat{\beta}_{IV} = \frac{Cov[z,y]}{Cov[z,x]}$, formula (4.49) on p. 99. They say that they derived this from ...
3
votes
2answers
135 views

Poisson as a limiting case of negative binomial

I was reading "Maximum Likelihood Estimation for the Negative Binomial Dispersion Parameter" by Walter W. Pieogorsch, and in the intro it says the Poisson distribution is a limiting case of negative ...
8
votes
1answer
118 views

Is the negative binomial not expressible as in the exponential family if there are 2 unknowns?

I had a homework assignment to express the negative binomial distribution as an exponential family of distributions given that the dispersion parameter was a known constant. This was fairly easy, but ...
7
votes
2answers
305 views

Proving the LATE Theorem of Angrist and Imbens 1994

Assume we have a binary instrument $Z_i$ which can be used to estimate the effect of the endogenous variable $D_i$ on the outcome $Y_i$. Suppose the instrument has a significant first stage, it is ...
1
vote
1answer
74 views

Why constrain mean and standard deviation when proving Gaussian is maximum differential entropy pdf?

I'm reading Bishop's Pattern Recognition and Machine Learning. In chapter 1.6: Information Theory (page 53) when trying to derive the maximum differential entropy pdf from the definition of continuous ...
1
vote
0answers
105 views

Average within-cluster distance using divisive clustering

I have to prove that the average within-cluster distance for 10 data points cannot increase when going from 1 cluster to 2 clusters (divisive clustering). Intuitively, it seems obvious that this is ...
2
votes
3answers
171 views

Proving Linear Estimator (beta) is BLUE?

In the book Statistical Inference pg 570 of pdf, There's a derivation on how a linear estimator can be proven to be BLUE. I got all the way up to 11.3.18 and then the next part stuck me. After ...
3
votes
1answer
53 views

Equivalence of random effects via likelihood and smoothed splines

Some fake data: X = runif(1000) ff = rep(1:10,100) E = rnorm(1000) y = x+e+f f = as.factor(ff) When you fit a model like ...