# Tag Info

### Difference between Bayes classifier, KNN classifier and Naive Bayes Classifier

Is there any difference between Bayes Classifier and Naive Bayes Classifier ? Is there any fundamental difference ? Yes, not a fundamental though. Naive Bayes Classifier simplifies the Bayes rule by ...
• 243

### Roll 4 Dice, What's the Expected Value of the Sum of the highest 3?

For clarity, allow me to re-introduce the following notation: let $X_i$ be the outcome of the $i$-th roll, $i = 1, 2, 3, 4$. You are interested in determining the pmf (to be more rigorous, it is pmf, ...
• 6,097
1 vote
Accepted

### Derive the ML-estimation for the parameter $\beta$

The derivation is basically fine until here. After this step, you introduce $\bar x, \bar y$ and the result is not correct. $$0=-\frac{1}{2\sigma^2}\sum_{i=1}^{n} - 2y_i x_i + 2 x_i^2\beta.$$ It's ...
• 82.8k

### Conditional expectation of dependent variable provided relationship

Since $Y$ is a function of $h(X)$ then in your case you have some 'knowledge' on $Y$, hence it would not be extraordinary to be able to calculate the conditional expectation of $Y$, because it is ...

### How to determine the limiting distribution of $Y=(\bar{X})^2$, where $X=(X_1,...,X_n)$ and $X_i \overset{\mathrm{iid}}{\sim} \textrm{Unif}(0,1)$?

In the comments you are close to a correct answer. You already have that $\sqrt{n}\left(\bar{X}_n-\mu\right) \overset{d}{\rightarrow} \mathop{\mathcal{N}}\left(0, \sigma^2\right)$ by the Lindeberg–...
• 3,460

• 53
1 vote

### The bayes estimator of Bernoulli model with constant prior

[In my notation, $\theta=p$ and $k=\sum_{i=1}^nX_i$.] It is well-known that the posterior in this problem is \begin{eqnarray} \pi(\theta|y) &= & \frac{\theta ^{\alpha _{0}+k-1}\left( 1-\theta \...
• 28.5k
1 vote

### Distribution of Ratio of Two Dependent Binomial Random Variables

Your initial problem description does not follow entirely the description in the second part. But based on that description I get to the following: You have a pool of $N$ People of which a fraction $g$...
• 55.2k
Accepted

• 1,386

### Estimating the parameter $\beta$

Assise from using a direct derivation, leading to a closed form expression, you can do this with a generalized linear model a description is given in the question Fitting exponential (regression) ...
• 55.2k
Accepted

• 4,094
Accepted

### Bayes error and classifier error with 0-1 loss

Reading the original Friedman paper this question comes from clarified it. The classifiers are trained on a training set $T$ and the distributions of the $\hat{G}$ are conditional on $T$. For the 0-1 ...
• 219

### Consistency of a simple estimator for $y_i = \beta_1 x_i + u_i$

Hint: Consider the possibility that your conjecture may be false. As a simple illustration, consider the case where $x_1+x_n \neq 0$ and find the resulting variance for your estimator. Show that ...
• 104k
1 vote

### Is the sum of a discrete and a continuous random variable continuous or mixed?

Assume that $X$ takes values in a countable set $\{n_i\}_{i=1,2,\dots}$. If $Y$ is continuous, for every real number $t$ $${\rm P}(X+Y=t) = \sum_i{\rm P}(X=n_i,Y=t-n_i) =0,$$ since for all $i$ we ...
Accepted

• 3,460
Accepted

### A question about conditional expectation

Because $\sum_{j = 1}^Y E[X_j|Y]$ is $\sigma(Y)$-measurable, to show it is the conditional expectation as desired, it is sufficient to show for any $n \in \{1, 2, \ldots\}$, it holds that (this is ...
• 6,097
1 vote
Accepted

### Mean (or lower bound) of Gaussian random variable conditional on sum, $E(X^2| k \geq|X+Y|)$

Let's simplify a little. Define $$(U,V) = \frac{1}{\sqrt{\sigma_X^2+\sigma_Y^2}}\left(X+Y,\ \frac{\sigma_Y}{\sigma_X}X - \frac{\sigma_X}{\sigma_Y}Y\right).$$ You can readily check that $U$ and $V$ ...
• 299k
1 vote

### Derive posterior density function with Jeffrey's prior for theta

A gamma distribution for $\theta$ with shape $\alpha$ and rate $\beta$ has density $f(\theta)=\frac{\beta^\alpha}{\Gamma(\alpha)}\theta^{\alpha-1} \exp(-\beta \theta)$ which, by dropping ...
• 32.9k
Accepted

### How to prove $\mathcal{I}_{1}(\eta) = \mathcal{I}_{1}(\theta)[h'(\eta)]^{2}$ where $\mathcal{I}_{1}$ is the Fisher information and $\theta = h(\eta)$?

It's just a consequence of chain rule. The density of $X$ is $f(x; \theta) = f(x; h(\eta))$. Therefore, viewing it as a function of $\eta$, we have \begin{align} \frac{d\ln f(x; h(\eta))}{d\eta} = \...
• 6,097

### How to prove $\mathcal{I}_{1}(\eta) = \mathcal{I}_{1}(\theta)[h'(\eta)]^{2}$ where $\mathcal{I}_{1}$ is the Fisher information and $\theta = h(\eta)$?

Start out from likelihood function $L(\theta;Y_1)$ (for a single observation) and make the reparametrization $\theta = h(\eta)$ to get the reparametrized log-likelihood $L(\eta;Y_1) = L(h(\eta);Y_1)$. ...
• 4,094

### How to find the expected value and median of a chi square distribution with $12$ dof in R?

The mean of a $\chi^2$ random variable is equal to the degrees of freedom. The median has no close form afaik, you can get it in R from the inverse cumulative distribution function (aka the quantile ...
• 17.8k

Your answer is okay and is justified below. You can write:$$X=B+(1-B)Y$$where $B\sim\text{Bernoulli}(p)$ and $Y\sim\text{Unif}(k)$ are independent random variables. Then:$$\mathbb EX=\mathbb EB+\... • 819 11 votes Accepted ### Expected value of random variable that is defined by another Your random variable takes the value 1 with probability p+\frac{1-p}{k}, and takes each value j\in\{2, \dots, k\} with probability \frac{1-p}{k}. So the expectation is simply$$ \begin{align*} ...
• 105k
If e.g. the sets $A,B,C$ form a partition of index set $[11]$ then you must be able to find expressions for probabilities like: $$P(X_A=a,X_B=b,X_C=c)$$ where $n:=a+b+c$ equals the total number of ...