New answers tagged

0 votes

Difference between Bayes classifier, KNN classifier and Naive Bayes Classifier

Is there any difference between Bayes Classifier and Naive Bayes Classifier ? Is there any fundamental difference ? Yes, not a fundamental though. Naive Bayes Classifier simplifies the Bayes rule by ...
  • 243
3 votes

Roll 4 Dice, What's the Expected Value of the Sum of the highest 3?

For clarity, allow me to re-introduce the following notation: let $X_i$ be the outcome of the $i$-th roll, $i = 1, 2, 3, 4$. You are interested in determining the pmf (to be more rigorous, it is pmf, ...
  • 6,097
1 vote
Accepted

Derive the ML-estimation for the parameter $\beta$

The derivation is basically fine until here. After this step, you introduce $\bar x, \bar y$ and the result is not correct. $$0=-\frac{1}{2\sigma^2}\sum_{i=1}^{n} - 2y_i x_i + 2 x_i^2\beta.$$ It's ...
  • 82.8k
0 votes

Conditional expectation of dependent variable provided relationship

Since $Y$ is a function of $h(X)$ then in your case you have some 'knowledge' on $Y$, hence it would not be extraordinary to be able to calculate the conditional expectation of $Y$, because it is ...
3 votes

How to determine the limiting distribution of $Y=(\bar{X})^2$, where $X=(X_1,...,X_n)$ and $X_i \overset{\mathrm{iid}}{\sim} \textrm{Unif}(0,1)$?

In the comments you are close to a correct answer. You already have that $\sqrt{n}\left(\bar{X}_n-\mu\right) \overset{d}{\rightarrow} \mathop{\mathcal{N}}\left(0, \sigma^2\right)$ by the Lindeberg–...
  • 3,460
4 votes

How to determine the limiting distribution of $Y=(\bar{X})^2$, where $X=(X_1,...,X_n)$ and $X_i \overset{\mathrm{iid}}{\sim} \textrm{Unif}(0,1)$?

Hint: Have a look at the mean and variance of the uniform distribution and use this to define the statistic: $$Z_n \equiv \sqrt{12n} (\bar{X}_n - \tfrac{1}{2}),$$ giving you: $$\bar{X}_n^2 = \frac{(...
  • 104k
3 votes
Accepted

Estimating the parameters of a Poisson times a constant

Gaussian iid errors: If the errors are independent and normally distributed, the likelihood can be expressed as \begin{align} L(\lambda,\sigma,a) &=\prod_{i=1}^n f_{Y_i}(y_i), \\&=\prod_{i=1}^...
  • 8,987
1 vote

How to determine the limiting distribution of $Y=(\bar{X})^2$, where $X=(X_1,...,X_n)$ and $X_i \overset{\mathrm{iid}}{\sim} \textrm{Unif}(0,1)$?

This is just an extended comment in that a display is produced showing how close the normal approximation is to the true probability density function. The density for $Y$ for any sample size can be ...
  • 3,031
2 votes

Binomial hypothesis test - Coin flip

The test statistic and the sample mean are not the same thing. Recall the test statistic is what we compare to a known distribution to obtain inference - parts 1 and 2 of the question are essentially ...
  • 55.2k
2 votes

Confusing Question for Stats

In my opinion that's a silly decision because Type 1 error, Type 2 error and a correct decision all assume you know both the "True" hypothesis and the stat that you are measuring. But in ...
  • 284
0 votes

Estimating the parameters of a Poisson times a constant

EDIT: this was an answer to the original form of the question. A simple answer was staring me in the face although I am not sure if this is likely not the maximum likelihood estimator of $a$ (will ...
  • 145
2 votes

Probability of successful scenarios where you run a trial until there are N more successes than failures

Let $p$ be the chance of $X$ winning a point. Fix $N \ge 1$ and for $i$ any integer between $-N$ and $N$ define $f(i)$ to be the chance of $X$ winning the game given $X$ is $i$ points ahead of $Y.$ ...
  • 299k
1 vote

Probability of successful scenarios where you run a trial until there are N more successes than failures

See the formula for $h_i^0$ the bottom of this answer. For your formulation, the probability of $X$ "winning" corresponds to $h_i^0$, $N$ corresponds to $i$, $2N$ corresponds to $N$, and $P$ ...
  • 711
0 votes

Calculation of residual standard deviation and r-squared

I think that making the error term $\epsilon$ explicit would help. In log terms we would have: $$\log(y) = \alpha + \beta \log(x) + \epsilon.$$ Taking exponents, the actual model would look like the ...
1 vote
Accepted

Deriving the asymptotic distribution using delta method

This time using the Delta method: $$E(y)=\mu$$ $$V(y)=\mu^3$$ $$r=\mu^2$$ $$\hat{r}=\left(\sum_{i=1}^n y_i/n\right)^2$$ Now the function of interest is $$f=\sqrt{n} (\hat{r}-r)$$ The asymptotic ...
  • 3,031
1 vote

Deriving the asymptotic distribution using delta method

If from the comments that knowing the variance of $\hat{\mu}^2$ is desired, then that can be determined exactly for any sample size. The mean of $\hat{r}=\hat{\mu}^2$ is $\mu^2 (1+\mu/n)$ and the ...
  • 3,031
0 votes

Is this difference of two kernels also a kernel?

Writing it out, we have $$ k_1 - k_2 = x^\top \begin{bmatrix}3 & 1 \\ 1 & 5\end{bmatrix} y - x^\top y \\ =x^\top (A - I)y $$ and we can verify that $A - I$ is positive definite. If we form a ...
  • 82.8k
1 vote

Deriving the asymptotic distribution using delta method

Consider $\mathop{g}\left(x\right)\mathrel{:=x^2}$ with derivative $\mathop{g'}\left(x\right)=2x$. What does the Wikipedia article on the delta method tell you about the asymptotic distribution of $\...
  • 3,460
2 votes
Accepted

Show that $\min_{a \in \mathbb{R}} E \left[ \max \left( (1-a) V, a Z \right) \right]$ is minimized by $a$ such that $0<a<1$

It is not possible to limit the solution to the open interval $a \in (0,1),$ because when for instance $(X,Y)$ is standard Normal, the global minimum is attained on the entire closed interval $[0,1]$ ...
  • 299k
0 votes
Accepted

Fisher information for the negative binomial distribution

Ok, now I got it. $$p(y) = {y-1 \choose r-1} \theta^r(1-\theta)^{y-r}$$ Let's denote $X$ as the random variable, then: $$\ell(\theta) = C + r \ln(\theta) + (X-r) \ln(1-\theta)$$ $$\ell'(\theta) = \...
  • 53
1 vote

The bayes estimator of Bernoulli model with constant prior

[In my notation, $\theta=p$ and $k=\sum_{i=1}^nX_i$.] It is well-known that the posterior in this problem is \begin{eqnarray} \pi(\theta|y) &= & \frac{\theta ^{\alpha _{0}+k-1}\left( 1-\theta \...
1 vote

Distribution of Ratio of Two Dependent Binomial Random Variables

Your initial problem description does not follow entirely the description in the second part. But based on that description I get to the following: You have a pool of $N$ People of which a fraction $g$...
5 votes
Accepted

Minimax estimator for geometric distribution

The weighted quadratic risk of an estimator $\delta$ is given by $$R(p,\delta)=\sum_{x=0}^\infty (p-\delta(x))^2\times (1-p)^{x-1}$$ Since the first term of this sum is $(p-\delta(0))^2\times (1-p)^{-...
  • 95.9k
2 votes
Accepted

UMP test for hypergeometric distribution

Here $X\sim \operatorname{HyperGeo}(N,D,n) . $ We need to find (if it exists) the UMP test of $\mathcal H_0: D\leq D_0$ vs $\mathcal H_1: D> D_0.$ What should be the approach to tackle such problem,...
  • 2,539
1 vote

A linear regression exercise based on the relationship between $2017$ and $2019$ Math SAT scores

Going purely off the summary here without knowing the data it is derived from, we can probably answer the questions here as following. Keep in mind that I am generalizing here so you can answer the ...
2 votes
Accepted

Generating random variables from a mixture of Normal distributions and Exponential distribution using composition method

Let me rephrase the problem as follows: Question In order to sample $X$ from $$0.3\,\mathcal Exp(1)+0.5\, \mathcal N(0,1)+0.2\,\mathcal N(4,1)\tag{1}$$ a. Write $\text{Prob}(X\le x)$ as $$0.3\,\text{...
  • 95.9k
2 votes

Alternative way of finding the Distribution of $Y_{3} = (X_{1}X_{2}X_{3})^{1/3}$

The simplest method I can think of in this case exploits the symmetry (equal probabilities of 2 and 3) to evaluate the distribution of $$\log Y = \left(\log X_1 + \log X_2 +\log X_3\right)/3$$ by ...
  • 299k
9 votes
Accepted

Calculating functions of truncated and censored normal variables

The first result simulates the mean of a truncated normal distribution with truncation point 1, whose expected value is, from here and standard normality with $\mu=0$, $\sigma=1$, $$ \mu_T=\frac{\phi(...
2 votes

Number of subset from a sample space S is $2^n$?

Here are two combinatorial non-inductive proofs. Suppose $S = \{a_1, \ldots, a_n\}$, where $a_1, \ldots, a_n$ are distinct. The goal is to count the number of all the subsets of $S$. In other words, ...
  • 6,097
2 votes
Accepted

Number of subset from a sample space S is $2^n$?

Thanks everyone for the comments, so I tried to answer it myself. Each element of $S$ can either be in the subet or not. For each elements we only have two options, all the subsets will therefore be ...
  • 49
2 votes

Estimating the parameter $\beta$

$ L(\beta) = P(\mathrm{Data} \mid \beta) = \prod_i f(x_i) $ and $\lambda = \frac{1}{\mu} = \frac{s}{\beta}$ Log is increasing so maximizing log is same as maximizing likelihood: $\ell(\beta) = \sum \...
  • 1,386
2 votes

Estimating the parameter $\beta$

Assise from using a direct derivation, leading to a closed form expression, you can do this with a generalized linear model a description is given in the question Fitting exponential (regression) ...
8 votes
Accepted

Estimating the parameter $\beta$

To estimate the $\beta$ by the maximum likelihood method, let $Y_1,\ldots, Y_n$ be the sample of lifetimes, with $Y_i\sim \text{Exp}(\beta/s_i)$, s.t. $E(Y_i) = \beta/s_i$ and independently for each $...
  • 4,094
2 votes

Probability of sample means being equal

By some properties of the normal distribution (make sure you understand which) $$\bar X \sim N(70, 16/4), \quad \bar Y \sim N(70, 9/9)$$ If $X$ and $Y$ are independent, $\bar X - \bar Y = W \sim N(0, ...
  • 4,094
0 votes
Accepted

Bayes error and classifier error with 0-1 loss

Reading the original Friedman paper this question comes from clarified it. The classifiers are trained on a training set $T$ and the distributions of the $\hat{G}$ are conditional on $T$. For the 0-1 ...
  • 219
4 votes

Consistency of a simple estimator for $y_i = \beta_1 x_i + u_i$

Hint: Consider the possibility that your conjecture may be false. As a simple illustration, consider the case where $x_1+x_n \neq 0$ and find the resulting variance for your estimator. Show that ...
  • 104k
1 vote

Is the sum of a discrete and a continuous random variable continuous or mixed?

Assume that $X$ takes values in a countable set $\{n_i\}_{i=1,2,\dots}$. If $Y$ is continuous, for every real number $t$ $$ {\rm P}(X+Y=t) = \sum_i{\rm P}(X=n_i,Y=t-n_i) =0, $$ since for all $i$ we ...
6 votes
Accepted

Prove that the sum is sufficient using using the definition of sufficiency

You are almost there. Note that after $X_1,\ldots, X_n$ has been fixed at $x_1,\ldots, x_n$, $Y$ is automatically fixed. Therefore, your numerator becomes $$ P_\theta(X_1=x_1,\ldots, X_n=x_n, x_1+\...
  • 4,094
2 votes

How to work with distributions that are discrete in some variables and continuous in others?

As many comments pointed out, the information given by "$f(t, j)$" needs more clarification, because it is neither a simple pdf/pmf nor a standard representation of mixture model. My ...
  • 6,097
1 vote
Accepted

Probability that x-axis perfectly separates clusters

Yes, you are exactly correct. Here is an illustration, which also shows you can easily simulate questions like this one to sharpen your intuition: R code: ...
0 votes

How to show that $X$ has a Cauchy distribution?

From the problem description we know that $$ \tan\left(\alpha\right)=\frac{X}{b} \iff X = \tan\left(\alpha\right) \cdot b,\\ \alpha = \arctan\left(\frac{X}{b}\right) \sim \mathop{\mathrm{Unif}}\left(\...
  • 3,460
0 votes
Accepted

A question about conditional expectation

Because $\sum_{j = 1}^Y E[X_j|Y]$ is $\sigma(Y)$-measurable, to show it is the conditional expectation as desired, it is sufficient to show for any $n \in \{1, 2, \ldots\}$, it holds that (this is ...
  • 6,097
1 vote
Accepted

Mean (or lower bound) of Gaussian random variable conditional on sum, $E(X^2| k \geq|X+Y|)$

Let's simplify a little. Define $$(U,V) = \frac{1}{\sqrt{\sigma_X^2+\sigma_Y^2}}\left(X+Y,\ \frac{\sigma_Y}{\sigma_X}X - \frac{\sigma_X}{\sigma_Y}Y\right).$$ You can readily check that $U$ and $V$ ...
  • 299k
1 vote

Derive posterior density function with Jeffrey's prior for theta

A gamma distribution for $\theta$ with shape $\alpha$ and rate $\beta$ has density $f(\theta)=\frac{\beta^\alpha}{\Gamma(\alpha)}\theta^{\alpha-1} \exp(-\beta \theta)$ which, by dropping ...
  • 32.9k
3 votes
Accepted

How to prove $\mathcal{I}_{1}(\eta) = \mathcal{I}_{1}(\theta)[h'(\eta)]^{2}$ where $\mathcal{I}_{1}$ is the Fisher information and $\theta = h(\eta)$?

It's just a consequence of chain rule. The density of $X$ is $f(x; \theta) = f(x; h(\eta))$. Therefore, viewing it as a function of $\eta$, we have \begin{align} \frac{d\ln f(x; h(\eta))}{d\eta} = \...
  • 6,097
4 votes

How to prove $\mathcal{I}_{1}(\eta) = \mathcal{I}_{1}(\theta)[h'(\eta)]^{2}$ where $\mathcal{I}_{1}$ is the Fisher information and $\theta = h(\eta)$?

Start out from likelihood function $L(\theta;Y_1)$ (for a single observation) and make the reparametrization $\theta = h(\eta)$ to get the reparametrized log-likelihood $L(\eta;Y_1) = L(h(\eta);Y_1)$. ...
  • 4,094
2 votes

How to find the expected value and median of a chi square distribution with $12$ dof in R?

The mean of a $\chi^2$ random variable is equal to the degrees of freedom. The median has no close form afaik, you can get it in R from the inverse cumulative distribution function (aka the quantile ...
10 votes

Expected value of random variable that is defined by another

Your answer is okay and is justified below. You can write:$$X=B+(1-B)Y$$where $B\sim\text{Bernoulli}(p)$ and $Y\sim\text{Unif}(k)$ are independent random variables. Then:$$\mathbb EX=\mathbb EB+\...
  • 819
11 votes
Accepted

Expected value of random variable that is defined by another

Your random variable takes the value $1$ with probability $p+\frac{1-p}{k}$, and takes each value $j\in\{2, \dots, k\}$ with probability $\frac{1-p}{k}$. So the expectation is simply $$ \begin{align*} ...
2 votes
Accepted

How to combine probablities in the following example?

If e.g. the sets $A,B,C$ form a partition of index set $[11]$ then you must be able to find expressions for probabilities like: $$P(X_A=a,X_B=b,X_C=c)$$ where $n:=a+b+c$ equals the total number of ...
  • 819

Top 50 recent answers are included