13
votes
Accepted
How to calculate average length of adherence to vegetarianism when we only have survey data about current vegetarians?
Let $f_X(x)$ denote the pdf of adherence length $X$ of vegetarianism in the population. Our objective is to estimate $EX=\int_0^\infty x f_X(x)\;dx$.
Assuming that the probability of being included ...
- 11.7k
9
votes
Minimum expectation
When $p$ increases, that means its cumulative distribution function
$$P(x) = \int_0^x p(x)\,\mathrm{d}x$$
is convex. Since $P(0)=0$ and $P(1)=1,$ the convexity implies the graph of $P$ lies on or ...
- 334k
9
votes
Accepted
How can I use Propensity Scores to adjust for survey non-response bias?
The answer to this depends on whether you have a probability sample versus a nonprobability sample, where a probability sample refers to a sample selected using random sampling from the population.
If ...
- 506
9
votes
Win Percentage Question
This is a simple weighted average, call your next $N$ games' win percentage $p$, an solve for the following:
$$\frac{1500 \times 0.57 + p \times N}{1500+N}=0.6$$
- 58.2k
8
votes
Accepted
How to estimate the (approximate) variance of the weighted mean?
You can get general answer to this question (and the specific answer) from just considering the variances of sums. Suppose there are $N$ individuals in the population and you sample $n$ of them. The $...
- 46.8k
7
votes
Will the mean of a set of means always be the same as the mean obtained from the entire set of raw data?
In general, if you have a set of $m$ groups with respective sizes $n_1,...,n_m$ and means $\bar{x}_1,...,\bar{x}_m$ then the overall sample mean of all the data is:
$$\bar{x} = \sum_{k=1}^m \frac{n_k}{...
- 133k
7
votes
Are there better approaches than the weighted mean?
There seems to be a smooth dependence of variance on observation index, so you could try a joint modeling approach, see for instance Articles that work with covariates for mean, variance, and ...
- 82.8k
7
votes
Win Percentage Question
@gunes has covered the answer.
Probably worth pointing out that there are constraints on the values $(p, N)$ that satisfy that equation (assuming $N>0$ and $0\leq p\leq 1$).
Regardless of your ...
- 2,911
7
votes
Accepted
Simpson's paradox in Freedman, Pisani and Purves book
This is explained on same page (p. 19):
Technical note. Table 2 is hard to read because it compares 12 admission rates. A statistician might summarize table 2 by computing one overall admissions rate ...
- 14.3k
6
votes
Downweight outliers in mean
Rounding up the comments that has value of an answer, several methods can be used here.
1. Trimmed mean (by @Bernhard)
Calculates the average of data that lies between the 5th and 95th percentile ...
- 810
5
votes
Accepted
Can an optimal weighted average ever have negative weights?
In addition to the other excellent answer. Yes, it can give negative weights, in some cases, even if that can look counterintuitive. Let us see. First, I will go through solving the minimization ...
- 82.8k
5
votes
Can an optimal weighted average ever have negative weights?
In abstract theory, yes, because of the correlation structure.
If you had uncorrelated measurements (with positive variance), then the weights could only be positive.
Example:
Let $\mu$ denote the ...
- 23k
5
votes
Inverse Probability Weighting for Binary Outcomes
If you use the Hajek estimator, the most commonly used estimator for IPW, the expected potential outcomes are bounded between 0 and 1 as long as the weights are non-negative, which they will be in ...
- 36.8k
5
votes
Accepted
Recognize extreme weights in inverse probability treatment weighting
The problem with extreme weights is that they yield high variability in the weights which decreases the effective sample size. You don't have to check for extreme weights; you just need to check for ...
- 36.8k
4
votes
Accepted
Calculate classifier accuracy from per label accuracy
I fail to see why you are trying to use per-label accuracy for the calculation of the overall accuracy. For any problem, you can compute the overall accuracy as:
$$Acc =\sum \frac{\#CorrectPredictions}...
- 1,586
4
votes
Accepted
Show posterior mean can be written as a weighted average of the prior mean and MLE
Continuing from where you left,
$$\begin{align}
\bar\lambda_{post}^{-1} &= \frac{\sum_{i}y_i+b}{n+a} \\
&= \frac{n}{n+a}\frac{\sum_{i}y_i}{n} + \frac{a}{n+a}\frac{b}{a} \\
&= \rho\lambda_{...
- 779
4
votes
How to calculate the mean from bin endpoints and frequencies?
Because your data is binned into intervals, you cannot really calculate the original sample mean because you should not make up information that you don't have access to. However, you have a couple of ...
- 9,680
4
votes
Accepted
How to interpret the size of weighted mean difference (WMD) in a meta analysis?
A (weighted) mean difference is the difference between effect estimates for intervention and control on a specific scale. Assuming this is the study you are referring to this, your scale is time ...
- 1,290
4
votes
Are there coventions on reporting weighted sample sizes?
I assume you will report weighted averages or proportions. Usually weights are based on a reference population,eg, the United States or New York City. This basis for weights should be reported. ...
- 1,338
3
votes
Accepted
Log response ratio of means not agreeing with linear mixed model output
There are two extra points:
Because you’re using a mixed effects Poisson regression with a log (nonlinear) link function, the coefficients you obtain have an intepretation conditional on the random ...
- 21.5k
3
votes
Accepted
What's a good measure of spread when using a weighted average
In that scenario you would use the weighted standard deviation:
$$
s=\sqrt{\frac{1}{\sum_iw_i}\sum_i w_i(x_i-\bar{x})^2}
$$
where $x_i$ is the $i$-th data point and $w_i$ the weight on that data point,...
- 7,752
3
votes
SE of weighted mean
Well, under i.i.d. assumptions for the $x_i$ the variance of the weighted sample mean relative to the population variance would have a factor $\left(\sum_iw_i^2\right)/\left(\sum_iw_i\right)^2$, which ...
- 13.1k
3
votes
Better confidence intervals for weighted average
An estimator that is obviously better in some ways is
$$\hat\mu= \frac{\sum_{\textrm{observed }k} n_kx_k}{\sum_{\textrm{observed }k} n_k}$$
In particular, if $|J|$ is large enough that all $K$ ...
- 46.8k
3
votes
How can I identify outliers?
Welcome to CV.SE!
There is no such thing as a general method to identify outliers. Before you think about identifying outliers, you want to think about what is the generator for your dataset.
The ...
- 826
3
votes
Minimum expectation
First, change the problem to say that the density is non-decreasing and piecewise constant. Solve that problem first and then return to this problem.
Suppose there is a density $f(x)$ that minimizes $...
- 2,652
2
votes
Computing standard error in weighted mean estimation
@Ming K 's equation is not working for me. @Hugh mentioned Hmisc::wtd.var(x, w), but this is for variance, if you are wondering about weighted standard error, this ...
- 121
2
votes
Meta-analysis: How to find the mean age of all studies?
So this does not go unanswered I copy my comment here as an answer.
If you are willing to assume that age has a symmetrical distribution then the median also estimates the mean and vice versa. ...
- 18.4k
2
votes
Accepted
Combine mutliple predictions
there is more information in a classifier that tells you there is a 90% or a 1% change of event compared to one that says 50%
Why do you think so?
If one classifier outputs more extreme ...
- 131k
2
votes
What does it mean by "approach the performance of the Bayesian gold standard"?
The "Bayesian gold standard" is to "regularize" predictions by computing the posterior predictive distribution $P(y|x,D) = \int p(y|x,w) p(w|D) dw$, where $D$ is our dataset and $w$ is the weights in ...
- 26.5k
2
votes
Combining two means and SDs of one group
Rules to pool mean and variance of two groups can be found in O'Neill (2014) (Result 1). The formulas are:
$$\begin{equation} \begin{aligned}
\bar{x}_\text{pooled} &= \frac{1}{n_1+n_2} \Bigg[ ...
- 133k
Only top scored, non community-wiki answers of a minimum length are eligible