New answers tagged

0

Let $X_n \ : \ \Omega \to \mathbb R$. The implication $$ |g(x)-g(a)|\ge \varepsilon \implies |x-a|\ge \delta \tag{1}. $$ means that the set $$ A=\big \{ \omega \in \Omega : |g\left (X_n(\omega) \right)-g(a)|\ge \varepsilon \big \} $$ is a subset of $$ B=\big \{ \omega \in \Omega : |X_n(\omega)-a|\ge \delta \big \} $$ since $\omega \in A \Rightarrow \omega \...


0

Just an idea (maybe not so good): since about 70% of your values are zeroes, you are actually interested in comparing the last few deciles between groups. One possible tool for that can be the shift function: https://github.com/GRousselet/rogme But you should indeed clarify your research question, as others said.


2

Like Three Diag in his very good answer let's consider this a binomial problem with p being the probability of each single member of the list being dead and having cast a vote. The Bayesian question would be how to estimate our best guess of that probability and for binomial problems that is expecially easy because we can use a conjugated, in this case a ...


2

Let us say you have a large list of supposedly dead voter, you pick one at random and check whether they are actually dead and actually voted, in which case you mark a one, otherwise there must be a mistake in the list and you mark it with a zero. This is a bernoulli variable: $$ X_i = \begin{cases} 0, \text{with prob. }\ p \\ 1, \text{ with prob.}\ 1-p \end{...


1

Let $X\sim\mathcal N_n(\mu,\Sigma)$ and suppose we have a unit vector $u \in \mathbb R^n$ and we want to see what the distribution of the projection of $X$ onto the subspace given by $u$ is. This is $$ u(u^Tu)^{-1}u^TX = (u^TX) u $$ so $u^TX$ is the coordinate in this one dimensional subspace and $(u^TX)u$ is the corresponding element of $\mathbb R^n$. This ...


0

The central limit theorem applies only to a sum of mutually independent random variables with the same distribution and finite mean and variance. If the variables are merely pairwise independent (meaning any two of them are independent of each other, but more than two are not necessarily independent), the theorem need not hold true, and Avanzi et al. (2020) ...


2

You'd want two separate dataframes in this case for a more efficient computation. Say your dataframe is split into two lists, list1 and list2. Then: sd = math.sqrt((np.std(list1) ** 2 + np.std(list2) ** 2)/len(df)) print((np.mean(list1)-np.mean(list2))/sd) You could of course simplify it using notations, but these are the basic gist.


1

While you could use different periods for different stocks, results that are extrapolated from this data could be misleading, since the data does not account for one-off events in different time periods. For example, if one of your time periods is $2006-2012$ while another time period is $2011-2015$, the latter time period may account less of the $2008$ ...


1

This is only true if the location parameter is zero. (In which case you can use the characteristic function of the stable distribution $e^{-|ct|^\alpha (1-i\beta sign(t) tan(\pi\alpha/2))}$ to proof it for $\alpha\neq 1$) As a counter example consider the normal distribution (for which $\alpha=2$) $$Y = \frac{1}{\sqrt{n}}\sum_{i=1}^n X_i$$ In this case you ...


3

This result is not due to the positivity of $X,$ nor to the convexity of the function $x\to 1/x,$ nor to any particular property of this function apart from that it decreases. It would be less than satisfactory, then, to rely on the standard convexity inequalities such as Jensen's Inequality. Consider this characterization of the covariance of random ...


3

One proof is to note that \begin{align} \mathbf{Cov} (X, X^{-1}) &= \frac{1}{2}\mathbf{E} \left[ \left( X_1 - X_2 \right) \cdot \left( X_1^{-1} - X_2^{-1} \right) \right] \\ &= -\frac{1}{2}\mathbf{E} \left[ \frac{\left( X_1 - X_2 \right)^2}{X_1\cdot X_2} \right] \leq 0. \end{align}


4

Using the formula for covariance that you gave, you can reexpress the covariance as follows: $$\begin{aligned} \text{Cov}\left(X, \frac{1}{X}\right) &= E \left[ X\frac{1}{X}\right]-E[X]E\left[\frac{1}{X}\right] \\ &= 1 - E[X]E\left[\frac{1}{X}\right] \end{aligned}$$ Let $\varphi(Y) = \frac{1}{Y}$, which is a convex function for positive values of $...


4

Via, Jensen's inequality, you'll have $$\frac{1}{E[X]}\leq E\left[\frac{1}{X}\right]$$ because $f(x)=1/x$ is a convex function for positive $x$. If you substitute this into the covariance definition, you'll reach the desired result.


1

First, regarding this comment by the OP It is my understanding that $\pi_i$ here is considered a set of fixed parameters along with $k$ and $m$ (i.e. this is not a hierarchical model), where $\mathbb E (\pi_i)$ and $\text{var}(\pi_i)$ refer to the sample mean and variance of the set of fixed numbers $\{\pi_1,\dots,\pi_{m/k}\}$. Mathematically, there is no ...


1

We have $$ \hat\beta = (X^TX + \lambda I)^{-1}X^Ty $$ with $X \in \mathbb R^{n\times p}$ so, as @passerby51 commented, $$ \|\hat\beta\|^2 = y^TX(X^TX + \lambda I)^{-2}X^Ty. $$ To interpret this, let $X = UDV^T$ be the SVD of $X$. Then $$ \|\hat\beta\|^2 = y^T UD(D^2 + \lambda I)^{-2}DU^Ty = \sum_{i=1}^p \frac{d_i^2}{(d_i^2 + \lambda)^2} z_i^2 $$ with $z = U^...


1

I think you may be using a two-sided 90% confidence interval along with a one-sided test at the 10% level. If so, your result is not surprising. Your data must have been something like my 10-vector y below. [Computations in R.] y [1] 51.5 48.3 49.7 54.8 60.3 53.7 51.3 48.8 49.3 69.3 mean(y); sd(y) [1] 53.7 # sample mean [1] 6.559641 # sample ...


1

As Therneau and Grambsch say in "Modeling Survival Data: Extending the Cox Model," Springer (2000), defining the covariates as a function of time is tricky (p. 278): Examples where the covariate path is guaranteed are the exception, however. A major concern ... is whether the hypothetical path represents any patient at all. Survival curves based ...


0

For many applications, the natural logarithm of the likelihood function, called the log-likelihood, is more convenient to work with in our case." In statistics we often work with likelihood function, it is usually the ln that is considered. However, the two are related: log(x) = ln(x) / ln(10) = ln(x) / 2.303, and the ln-likelihood function reaches the ...


0

Method 1 $(X_{(1)},X_{(n)})$ is not complete because we can find $g\neq0$ but $\mathbb{E}\left[g(X_{(1)},X_{(n)})\right]=0,\forall\theta$. $g$ is $(t_1,t_2)\rightarrow\frac{n+1}{n-1}t_2-\frac{n+1}{1-n}t_1$. This is because $\mathbb{E}(X_{(n)})=\frac{n-1}{n+1}\theta$ and $\mathbb{E}(X_{(1)})=\frac{1-n}{n+1}\theta$. Thus $\mathbb{E}\left[g(X_{(1)},X_{(n)})\...


1

It is not exactly correct to say the blue curves represent the 95% CIs. For a given sample size, the 95% CI would be different for every different observed mean. But, it is correct that any mean will fall within the blue curves with 95% probability.


0

It's to get back the Poisson form. The second equality comes from noticing you don't need the $i=0$ term because it's zero. The fourth inequality comes from relabelling to get back the Poisson form in $j$ instead of $i$ Yes, but no, but it doesn't matter. The last equality goes from a sum over $j$ to the expectation of a Poisson random variable, and it ...


1

This definition of sufficiency by Koller & Friedman is both restrictive and incorrect: A sufficient statistic for a sample of size $M$ is not always the sum of statistics of the elements of the sample. A counter-example is the Uniform $\mathcal U(0,\theta)$ distribution where a sufficient statistic is $\max(\xi_1,\ldots,\xi_M)$ When two samples of the ...


4

Your own attempted answer incorrectly treats $k$ as a fixed value, rather than an index. This gives you an incorrect likelihood function, which means that your subsequent work is also incorrect. We first observe the event equivalence: $$-\theta k + k \leqslant x_k \leqslant \theta k + k \quad \quad \quad \iff \quad \quad \quad \Big| \frac{x_k}{k} - 1 \Big|...


0

First, check out what independent increments means. You need to consider a general sequence of points say $t_0 < t_1 < \cdots < t_n$. Hints: $\bar B(t_1) - \bar B(t_0) = B(t_1 +s) - B(t_0+s)$. You know something about the distribution of an increment of $B(\cdot)$, that is, you know the distribution of the right-hand side of the equation just ...


0

If you look at the Appendix of this article you might find a clearer explanation of the transformation steps in order to get the Markov Field representation equivalent to a factor graph. The appendix even generalizes to higher-order cliques using additional auxiliary variables. You can apply it to your example... But maybe for pairwise cliques, the ...


1

The point where you eliminate $\sum \epsilon_i x_i + \sum \epsilon_i \bar {x} = 0+0$ you are using expectation. This equation is not true for specific cases. Therefore your conclusion at the end that $\hat \beta = \beta$ should have been $E (\hat \beta) = E (\beta) = \beta$ which makes more sense. You can not have (in general, in all cases) that $\hat \beta =...


0

$\newcommand{\A}{\mathcal{A}} \newcommand{\B}{\mathcal{B}} \newcommand{\C}{\mathbb{C}} \newcommand{\E}{\mathbf{E}} \newcommand{\F}{\mathcal{F}} \newcommand{\G}{\mathcal{G}} \renewcommand{\H}{\mathcal{H}} \renewcommand{\L}{\mathcal{L}} \newcommand{\M}{\mathcal{M}} \newcommand{\N}{\mathbf{N}} \renewcommand{\P}{\mathbf{P}} \newcommand{\Q}{\mathbb{Q}} \...


0

I think that Cassella and Berger are choosing their conditions to match the narrative of the chapter. They are covering Convergence Concepts in that chapter, and so they moving through Convergence in Probability, Consistency, the weak law of large numbers (WLLN), the central limit theorem, almost sure convergence, the strong law of large number (SLLN), etc. ...


0

There are 3 problems with the explanations: For testing, you mention that you calculate "the" average score. You are looking to compare 2 computer programs, so unless you know the actual expectancy for one of them (which you don't in this setting), you'll have to calculate two averages: one for the old program, one for the new and test if the ...


1

'Significance' in statistics means 'low probability of obtaining results at least as extreme as actually obtained'. You need to make some assumptions before you can proceed. Here is one example: You can require that in a valid course a certain fraction of students needs to pass, say 90%. You can also fix the so-called ’significance level’ $\alpha$ to some ...


0

Thank you fblundun and thank you Adam k. $Since$ $Q$ $is$ $countable$, $let$ {$x_n$}$_{n≥1}$ $=$ $($$0$, $1$$]$ $∩$ $Q$ $∈$ $A$ $for$ $some$ $countable$ $sequence$ $of$ $x_1$, $x_2$, $....$ $Define$ $A_n$ = $(x_n$ − $1/2^{n+1}$, $x_n$] $∩$ $Q$ $∈$ $A$, $∀n$; $Define$ $B_1$ = $A_1$, $B_i$ = $A_i$ \ $∪^{i−1}_{j=1}$ $A_j$, $i$ $=$ $2,$ $...$ $,$ $n$. $Since$ $...


0

After skimming some Bayesian Statistics textbooks, I came to the conclusion that the standard Bayesian approach would be to take :$$𝐸[𝜋_𝑛(𝜃)]≥2/3$$ Still, I am not sure about this. Correct. We want the probability of the next toss being heads greater than or equal to $2/3$. This means $$P(H|\mathcal D)=\int P(H|\theta)\pi(\theta|\mathcal D)d\theta=\int \...


3

The key points here are that we are told (1) that the coin is fair and (2) that each flip is completely random and independent from the previous flip. If these two conditions are met, then the probability of the next flip being tails is 1/2. If the conditions are NOT met, then the probability is unlikely to be 1/2. If we didn't know if the coin was fair, and ...


1

Classical statistical process control (SPC) operates only operates on a single sequence of data points (ranked time) without taking into account their temporal distance. That is, it does not matter whether the data points are seconds or years apart. In many cases, this is an impermissible simplifaction of much richer data. If you have content knowledge ...


0

I assume the OP means a Complex normal distribution whose covariance and relation matrices are identity matrices. To get at the result we use a few properties; first $\mathbb{V}(X) = \mathbb{E}(X^2) - \mathbb{E}(X)^2$, second that diagonal covariance and relation matrices imply independence of random variables, third is the geometric series. Start with: $$\...


1

To illustrate the point that the answer depends on the underlying statistical model: If $Y\sim\mathcal E(1/X)$, an exponential variable, then $$\mathbb E[Y^k]= X^k\Gamma(k+1)$$ meaning that $Y^k/\Gamma(k+1)=Y^k/k!$ is an unbiased estimator of $X^k$, based on a single observation. This extends to Gamma variables, obviously.


0

Update on this in case anyone has the same question. The sample size was not big enough. After I increased the sample size, hyperparameter C selected would always be small enough to keep the coefficients at 0.


-1

I hAVE tried to find the conditional distribution for two cases only...you should try to find the conditional distribution for other two cases.It will clear your doubt.


2

The Confidence Interval (CI) definition the OP has provided is more or less the working definition - perhaps not the most rigourous one if we were to involve samples, but this form is used in general. However, one should note that The CI is random, not the parameter The population parameter $\theta$ is a fixed, unknown constant. It is by definition the true ...


1

R simulation mentioned in my Comment: set.seed(2020) L = rnorm(10^6,12, sqrt(.06)) W = rnorm(10^6,8, sqrt(.05)) H = rnorm(10^6,6, sqrt(.04)) V = L*W*H summary(V); var(V); sd(V) Min. 1st Qu. Median Mean 3rd Qu. Max. 451.9 557.1 575.6 576.0 594.4 727.0 [1] 766.321 [1] 27.6825 hist(V, prob=T) curve(dnorm(x,576,27.7), add=T)


2

The obvious analogy here is to use independent estimators $Y_1,...,Y_c$ with expectations: $$\mathbb{E}(Y_i) = X^{p_i} \quad \quad \quad \sum p_i = k.$$ In theory this is possible so long as you have a method to construct the required estimators for this problem. The independence requirement will generally make it impractical, since in most cases it will ...


1

You could try a regression model where the par and the distance are both covariates. You're right that a regular linear regression model wouldn't work, since our response of interest is an integer, but check out generalized linear models (see glm in R). These allow for responses to be other data types. In your case, you could investigate Poisson regression, ...


0

But, when I do the math on a $58/42$ split, it seems massively unfavorable to me. Over $100$ games, I would get ($58$ wins * $\$580$ per win) + ($42$ losses * $\$420$ per loss) = $\$51,280$. I would expect this number to also be $\$58,000$, but clearly I am wrong. Effectively this is like making a smaller bet. You win \$420 no matter what the outcome is. ...


1

The total pot over all 100 games is \$100,000. With a 58% chance of winning, you would therefore expect an average gain of \$58,000 if the winner of a round gets the whole pot. In some sense, you might consider this to be fair. On the other hand, if you get half of the pot whether you win or not, you'd have an average gain of $50,000. You might consider this ...


1

The CLT allows to talk about the asymptotic distribution of the mean, which is useful for providing inference on it. For example, that an experiment has led to a change in the mean of group A vs group B. It is true that for right-tailed distribution it is more convenient (IMHO) to talk about the median instead, but making inference on the median is harder (e....


0

In general, the second moment of $X$ about 0 means $\mathbb{E}((X-0)^2),$ i.e. $\mathbb{E}(X^2)$, whereas the second moment about some other value, such as 2, means $\mathbb{E}((X-2)^2)$.


0

$f(a)=a^2\cdot E[X^2] - 2\cdot E[X] +\frac{1}{a^2}$ $f'(a) = 2a\cdot E[X^2] -\frac{2}{a^3}$ $f''(a) = 2\cdot E[X^2] +\frac{6}{a^4} > 0$ Therefore, wherever $f'(a) = 0$ , we may say that $f$ will be minimized, as $f''(a) > 0$ for all $a\in\mathbb{R}$. Now, $$f'(a) = 0\quad\implies\quad a=\pm \frac{1}{\sqrt[4]{E[X^2]}}$$ As it's given that $a>0$ , ...


1

You can use both because they're the same: $$\begin{align}\hat{\operatorname{var}(\bar X)}&=\frac{\sum (x_i-\bar x)^2}{n^2}=\frac{\sum x_i^2-2\sum x_i\bar x+\sum\bar x^2}{n^2}\\&=\frac{\sum x_i-2\hat p\sum x_i+n\hat p^2}{n^2}=\frac{n\hat p-2\hat p n \hat p+n\hat p^2}{n^2}\\&=\frac{\hat p-\hat p^2}{n}=\frac{\hat p (1-\hat p)}{n}\end{align}$$ Note ...


2

Comment: Just a reality check with a particular example: Let $X \sim\mathsf{Pois}(\lambda=3),$ so that $\mu = \sigma^2= 3.$ Let $Y = 3X-5X^2 + 1,$ and find $E(Y).$ With 10 million realizations of $Y,$ one can expect three significant digits of accuracy. set.seed(2020) x = rpois(10^7, 3) y = 3*x - 5*x^2 + 1 mean(x); var(x) [1] 3.00072 # aprx E(X) = 3 [1]...


4

Given a unknown disttibution with $E(X)=\mu$ and $V(x)=\sigma^2$: $$E(3X-5X^2+1)=3E(X)-5E(X^2)+E(1)$$ $$E(3X-5X^2+1)=3E(X)-5[V(x)+E(X)^2]+E(1)$$ $$E(3X-5X^2+1)=3\mu-5[\sigma^2+\mu^2]+1$$ $$E(3X-5X^2+1)=3\mu-5\sigma^2-5\mu^2+1$$ I consider the full expression of @corey979 is the answer but it needs $\mu$ and $\sigma^2$.


Top 50 recent answers are included