New answers tagged self-study
0
votes
Difference between Bayes classifier, KNN classifier and Naive Bayes Classifier
Is there any difference between Bayes Classifier and Naive Bayes Classifier ? Is there any fundamental difference ?
Yes, not a fundamental though. Naive Bayes Classifier simplifies the Bayes rule by ...
- 243
3
votes
Roll 4 Dice, What's the Expected Value of the Sum of the highest 3?
For clarity, allow me to re-introduce the following notation: let $X_i$ be the outcome of the $i$-th roll, $i = 1, 2, 3, 4$. You are interested in determining the pmf (to be more rigorous, it is pmf, ...
- 6,097
1
vote
Accepted
Derive the ML-estimation for the parameter $\beta$
The derivation is basically fine until here. After this step, you introduce $\bar x, \bar y$ and the result is not correct.
$$0=-\frac{1}{2\sigma^2}\sum_{i=1}^{n} - 2y_i x_i + 2 x_i^2\beta.$$
It's ...
- 82.8k
0
votes
Conditional expectation of dependent variable provided relationship
Since $Y$ is a function of $h(X)$ then in your case you have some 'knowledge' on $Y$, hence it would not be extraordinary to be able to calculate the conditional expectation of $Y$, because it is ...
- 1
3
votes
How to determine the limiting distribution of $Y=(\bar{X})^2$, where $X=(X_1,...,X_n)$ and $X_i \overset{\mathrm{iid}}{\sim} \textrm{Unif}(0,1)$?
In the comments you are close to a correct answer. You already have that $\sqrt{n}\left(\bar{X}_n-\mu\right) \overset{d}{\rightarrow} \mathop{\mathcal{N}}\left(0, \sigma^2\right)$ by the Lindeberg–...
- 3,460
4
votes
How to determine the limiting distribution of $Y=(\bar{X})^2$, where $X=(X_1,...,X_n)$ and $X_i \overset{\mathrm{iid}}{\sim} \textrm{Unif}(0,1)$?
Hint: Have a look at the mean and variance of the uniform distribution and use this to define the statistic:
$$Z_n \equiv \sqrt{12n} (\bar{X}_n - \tfrac{1}{2}),$$
giving you:
$$\bar{X}_n^2 = \frac{(...
- 104k
3
votes
Accepted
Estimating the parameters of a Poisson times a constant
Gaussian iid errors: If the errors are independent and normally distributed, the likelihood can be expressed as
\begin{align}
L(\lambda,\sigma,a)
&=\prod_{i=1}^n f_{Y_i}(y_i),
\\&=\prod_{i=1}^...
- 8,987
1
vote
How to determine the limiting distribution of $Y=(\bar{X})^2$, where $X=(X_1,...,X_n)$ and $X_i \overset{\mathrm{iid}}{\sim} \textrm{Unif}(0,1)$?
This is just an extended comment in that a display is produced showing how close the normal approximation is to the true probability density function. The density for $Y$ for any sample size can be ...
- 3,031
2
votes
Binomial hypothesis test - Coin flip
The test statistic and the sample mean are not the same thing. Recall the test statistic is what we compare to a known distribution to obtain inference - parts 1 and 2 of the question are essentially ...
- 55.2k
2
votes
Confusing Question for Stats
In my opinion that's a silly decision because Type 1 error, Type 2 error and a correct decision all assume you know both the "True" hypothesis and the stat that you are measuring. But in ...
- 284
0
votes
Estimating the parameters of a Poisson times a constant
EDIT: this was an answer to the original form of the question.
A simple answer was staring me in the face although I am not sure if this is likely not the maximum likelihood estimator of $a$ (will ...
- 145
2
votes
Probability of successful scenarios where you run a trial until there are N more successes than failures
Let $p$ be the chance of $X$ winning a point. Fix $N \ge 1$ and for $i$ any integer between $-N$ and $N$ define $f(i)$ to be the chance of $X$ winning the game given $X$ is $i$ points ahead of $Y.$
...
- 299k
1
vote
Probability of successful scenarios where you run a trial until there are N more successes than failures
See the formula for $h_i^0$ the bottom of this answer. For your formulation, the probability of $X$ "winning" corresponds to $h_i^0$, $N$ corresponds to $i$, $2N$ corresponds to $N$, and $P$ ...
- 711
0
votes
Calculation of residual standard deviation and r-squared
I think that making the error term $\epsilon$ explicit would help. In log terms we would have:
$$\log(y) = \alpha + \beta \log(x) + \epsilon.$$
Taking exponents, the actual model would look like the ...
1
vote
Accepted
Deriving the asymptotic distribution using delta method
This time using the Delta method:
$$E(y)=\mu$$
$$V(y)=\mu^3$$
$$r=\mu^2$$
$$\hat{r}=\left(\sum_{i=1}^n y_i/n\right)^2$$
Now the function of interest is
$$f=\sqrt{n} (\hat{r}-r)$$
The asymptotic ...
- 3,031
1
vote
Deriving the asymptotic distribution using delta method
If from the comments that knowing the variance of $\hat{\mu}^2$ is desired, then that can be determined exactly for any sample size.
The mean of $\hat{r}=\hat{\mu}^2$ is $\mu^2 (1+\mu/n)$ and the ...
- 3,031
0
votes
Is this difference of two kernels also a kernel?
Writing it out, we have $$
k_1 - k_2 = x^\top \begin{bmatrix}3 & 1 \\ 1 & 5\end{bmatrix} y - x^\top y \\
=x^\top (A - I)y
$$
and we can verify that $A - I$ is positive definite.
If we form a ...
- 82.8k
1
vote
Deriving the asymptotic distribution using delta method
Consider $\mathop{g}\left(x\right)\mathrel{:=x^2}$ with derivative $\mathop{g'}\left(x\right)=2x$. What does the Wikipedia article on the delta method tell you about the asymptotic distribution of $\...
- 3,460
2
votes
Accepted
Show that $\min_{a \in \mathbb{R}} E \left[ \max \left( (1-a) V, a Z \right) \right]$ is minimized by $a$ such that $0<a<1$
It is not possible to limit the solution to the open interval $a \in (0,1),$ because when for instance $(X,Y)$ is standard Normal, the global minimum is attained on the entire closed interval $[0,1]$ ...
- 299k
0
votes
Accepted
Fisher information for the negative binomial distribution
Ok, now I got it.
$$p(y) = {y-1 \choose r-1} \theta^r(1-\theta)^{y-r}$$
Let's denote $X$ as the random variable, then:
$$\ell(\theta) = C + r \ln(\theta) + (X-r) \ln(1-\theta)$$
$$\ell'(\theta) = \...
- 53
1
vote
The bayes estimator of Bernoulli model with constant prior
[In my notation, $\theta=p$ and $k=\sum_{i=1}^nX_i$.]
It is well-known that the posterior in this problem is
\begin{eqnarray}
\pi(\theta|y) &= & \frac{\theta ^{\alpha _{0}+k-1}\left( 1-\theta \...
- 28.5k
1
vote
Distribution of Ratio of Two Dependent Binomial Random Variables
Your initial problem description does not follow entirely the description in the second part. But based on that description I get to the following:
You have a pool of $N$ People of which a fraction $g$...
- 55.2k
5
votes
Accepted
Minimax estimator for geometric distribution
The weighted quadratic risk of an estimator $\delta$ is given by
$$R(p,\delta)=\sum_{x=0}^\infty (p-\delta(x))^2\times (1-p)^{x-1}$$
Since the first term of this sum is $(p-\delta(0))^2\times (1-p)^{-...
- 95.9k
2
votes
Accepted
UMP test for hypergeometric distribution
Here $X\sim \operatorname{HyperGeo}(N,D,n) . $
We need to find (if it exists) the UMP test of $\mathcal H_0: D\leq D_0$ vs $\mathcal H_1: D> D_0.$
What should be the approach to tackle such problem,...
- 2,539
1
vote
A linear regression exercise based on the relationship between $2017$ and $2019$ Math SAT scores
Going purely off the summary here without knowing the data it is derived from, we can probably answer the questions here as following. Keep in mind that I am generalizing here so you can answer the ...
- 1,199
2
votes
Accepted
Generating random variables from a mixture of Normal distributions and Exponential distribution using composition method
Let me rephrase the problem as follows:
Question In order to sample $X$ from $$0.3\,\mathcal Exp(1)+0.5\, \mathcal N(0,1)+0.2\,\mathcal N(4,1)\tag{1}$$
a. Write $\text{Prob}(X\le x)$ as
$$0.3\,\text{...
- 95.9k
2
votes
Alternative way of finding the Distribution of $Y_{3} = (X_{1}X_{2}X_{3})^{1/3}$
The simplest method I can think of in this case exploits the symmetry (equal probabilities of 2 and 3) to evaluate the distribution of $$\log Y = \left(\log X_1 + \log X_2 +\log X_3\right)/3$$ by ...
- 299k
9
votes
Accepted
Calculating functions of truncated and censored normal variables
The first result simulates the mean of a truncated normal distribution with truncation point 1, whose expected value is, from here and standard normality with $\mu=0$, $\sigma=1$,
$$
\mu_T=\frac{\phi(...
- 28.5k
2
votes
Number of subset from a sample space S is $2^n$?
Here are two combinatorial non-inductive proofs.
Suppose $S = \{a_1, \ldots, a_n\}$, where $a_1, \ldots, a_n$ are distinct. The goal is to count the number of all the subsets of $S$. In other words, ...
- 6,097
2
votes
Accepted
Number of subset from a sample space S is $2^n$?
Thanks everyone for the comments, so I tried to answer it myself.
Each element of $S$ can either be in the subet or not. For each elements we only have two options, all the subsets will therefore be ...
- 49
2
votes
Estimating the parameter $\beta$
$
L(\beta) = P(\mathrm{Data} \mid \beta) = \prod_i f(x_i)
$ and $\lambda = \frac{1}{\mu} = \frac{s}{\beta}$
Log is increasing so maximizing log is same as maximizing likelihood:
$\ell(\beta) = \sum \...
- 1,386
2
votes
Estimating the parameter $\beta$
Assise from using a direct derivation, leading to a closed form expression, you can do this with a generalized linear model a description is given in the question
Fitting exponential (regression) ...
- 55.2k
8
votes
Accepted
Estimating the parameter $\beta$
To estimate the $\beta$ by the maximum likelihood method, let $Y_1,\ldots, Y_n$ be the sample of lifetimes, with $Y_i\sim \text{Exp}(\beta/s_i)$, s.t. $E(Y_i) = \beta/s_i$ and independently for each $...
- 4,094
2
votes
Probability of sample means being equal
By some properties of the normal distribution (make sure you understand which)
$$\bar X \sim N(70, 16/4), \quad \bar Y \sim N(70, 9/9)$$
If $X$ and $Y$ are independent, $\bar X - \bar Y = W \sim N(0, ...
- 4,094
0
votes
Accepted
Bayes error and classifier error with 0-1 loss
Reading the original Friedman paper this question comes from clarified it. The classifiers are trained on a training set $T$ and the distributions of the $\hat{G}$ are conditional on $T$.
For the 0-1 ...
- 219
4
votes
Consistency of a simple estimator for $y_i = \beta_1 x_i + u_i$
Hint: Consider the possibility that your conjecture may be false. As a simple illustration, consider the case where $x_1+x_n \neq 0$ and find the resulting variance for your estimator. Show that ...
- 104k
1
vote
Is the sum of a discrete and a continuous random variable continuous or mixed?
Assume that $X$ takes values in a countable set $\{n_i\}_{i=1,2,\dots}$. If $Y$ is continuous, for every real number $t$
$$
{\rm P}(X+Y=t) = \sum_i{\rm P}(X=n_i,Y=t-n_i) =0,
$$
since for all $i$ we ...
6
votes
Accepted
Prove that the sum is sufficient using using the definition of sufficiency
You are almost there. Note that after $X_1,\ldots, X_n$ has been fixed at $x_1,\ldots, x_n$, $Y$ is automatically fixed. Therefore, your numerator becomes
$$
P_\theta(X_1=x_1,\ldots, X_n=x_n, x_1+\...
- 4,094
2
votes
How to work with distributions that are discrete in some variables and continuous in others?
As many comments pointed out, the information given by "$f(t, j)$" needs more clarification, because it is neither a simple pdf/pmf nor a standard representation of mixture model. My ...
- 6,097
1
vote
Accepted
Probability that x-axis perfectly separates clusters
Yes, you are exactly correct.
Here is an illustration, which also shows you can easily simulate questions like this one to sharpen your intuition:
R code:
...
- 105k
0
votes
How to show that $X$ has a Cauchy distribution?
From the problem description we know that
$$
\tan\left(\alpha\right)=\frac{X}{b} \iff X = \tan\left(\alpha\right) \cdot b,\\
\alpha = \arctan\left(\frac{X}{b}\right) \sim \mathop{\mathrm{Unif}}\left(\...
- 3,460
0
votes
Accepted
A question about conditional expectation
Because $\sum_{j = 1}^Y E[X_j|Y]$ is $\sigma(Y)$-measurable, to show it is the conditional expectation as desired, it is sufficient to show for any $n \in \{1, 2, \ldots\}$, it holds that (this is ...
- 6,097
1
vote
Accepted
Mean (or lower bound) of Gaussian random variable conditional on sum, $E(X^2| k \geq|X+Y|)$
Let's simplify a little. Define
$$(U,V) = \frac{1}{\sqrt{\sigma_X^2+\sigma_Y^2}}\left(X+Y,\ \frac{\sigma_Y}{\sigma_X}X - \frac{\sigma_X}{\sigma_Y}Y\right).$$
You can readily check that $U$ and $V$ ...
- 299k
1
vote
Derive posterior density function with Jeffrey's prior for theta
A gamma distribution for $\theta$ with shape $\alpha$ and rate $\beta$ has density $f(\theta)=\frac{\beta^\alpha}{\Gamma(\alpha)}\theta^{\alpha-1} \exp(-\beta \theta)$ which, by dropping ...
- 32.9k
3
votes
Accepted
How to prove $\mathcal{I}_{1}(\eta) = \mathcal{I}_{1}(\theta)[h'(\eta)]^{2}$ where $\mathcal{I}_{1}$ is the Fisher information and $\theta = h(\eta)$?
It's just a consequence of chain rule. The density of $X$ is $f(x; \theta) = f(x; h(\eta))$. Therefore, viewing it as a function of $\eta$, we have
\begin{align}
\frac{d\ln f(x; h(\eta))}{d\eta} = \...
- 6,097
4
votes
How to prove $\mathcal{I}_{1}(\eta) = \mathcal{I}_{1}(\theta)[h'(\eta)]^{2}$ where $\mathcal{I}_{1}$ is the Fisher information and $\theta = h(\eta)$?
Start out from likelihood function $L(\theta;Y_1)$ (for a single observation) and make the reparametrization $\theta = h(\eta)$ to get the reparametrized log-likelihood $L(\eta;Y_1) = L(h(\eta);Y_1)$. ...
- 4,094
2
votes
How to find the expected value and median of a chi square distribution with $12$ dof in R?
The mean of a $\chi^2$ random variable is equal to the degrees of freedom. The median has no close form afaik, you can get it in R from the inverse cumulative distribution function (aka the quantile ...
- 17.8k
10
votes
Expected value of random variable that is defined by another
Your answer is okay and is justified below.
You can write:$$X=B+(1-B)Y$$where $B\sim\text{Bernoulli}(p)$ and $Y\sim\text{Unif}(k)$ are independent random variables.
Then:$$\mathbb EX=\mathbb EB+\...
- 819
11
votes
Accepted
Expected value of random variable that is defined by another
Your random variable takes the value $1$ with probability $p+\frac{1-p}{k}$, and takes each value $j\in\{2, \dots, k\}$ with probability $\frac{1-p}{k}$. So the expectation is simply
$$ \begin{align*} ...
- 105k
2
votes
Accepted
How to combine probablities in the following example?
If e.g. the sets $A,B,C$ form a partition of index set $[11]$ then you must be able to find expressions for probabilities like: $$P(X_A=a,X_B=b,X_C=c)$$ where $n:=a+b+c$ equals the total number of ...
- 819
Top 50 recent answers are included