11,143 reputation
11331
bio website
location
age 51
visits member for 4 years, 3 months
seen 1 hour ago

Apr
14
comment Why small values produce undulating densities when ploting logarithm of a loguniform prior (in R)?
If I try x <- 100^runif(1000000); plot(density(log(x))) then I get something which looks sensible between $0$ and about $4.6 \approx \log_e(100)$, so what did you run to get that oscillation?
Apr
10
comment Roll a 6-sided die until the total $\geq M$. Mean amount by which $M$ is exceeded?
I suspect that $M=300$ could be read as "very large $M$" as I believe that $M=301$ or $M=999$ would give almost exactly the same result. What I would do is find the distribution of the sum minus $M$.
Apr
9
comment Is there a name for this sort of plot? Is there any reason not to use it?
Difficult to find the $5$th percentile of $9$ observations. I think I would call what you are describing as a band chart: this is one (in fact two) looking at daily high and low temperature across the year in London.
Apr
8
comment Basic stats question - How to tell if my data distribution is symmetric
$-70,-63,-56,-49,-42,-35,-28,-21,-14,-7,0,1,4,9,16,25,36,49,64,81,100$ is deliberately not symmetric (uniform in the lower half but not in the upper half) and a box plot would put the median (equal to the mean) nearer the upper quartile than the lower quartile but also nearer the minimum than the maximum.
Apr
5
comment correlation between spelling ability and frequency of use of text messaging
You could start with a scatter plot to see what the results look like (though with a large sample you should be aware of overlapping points)
Mar
28
comment Closed form solution for t-stats and p-values in multiple regression
Excel has the TTEST function
Mar
28
comment Why do we need confidence intervals?
Yes. They might be equal by chance, but probably would not be.
Mar
28
comment Why do we need confidence intervals?
The expected value of the sample mean may be the population mean, but more often than not the sample mean will not be exactly equal to the population mean, and different samples are likely to have different sample means.
Mar
28
comment Why do we need confidence intervals?
It takes effort to look at every single account to get the exact answer with certainty, so taking a sample saves effort at the cost of some possible sampling error. The bigger the sample is, the smaller the sampling error is likely to be, but requires greater effort.
Mar
20
comment The abundance of P values in absence of a hypothesis
That table is actually a good example of what happens with large sample sizes (even the small differences in average age appear to be significant, suggesting perhaps that average waistlines may widen with old age or perhaps that larger waists increase average life expectancy very slightly). But the $P$-values are not dominating the table and at best this is exploratory analysis which could provide hypotheses for future study (e.g. a longitudinal study seeing whether people's waists widen or whether they die). It also suggests some factors which might be worth controlling for in future work.
Mar
19
comment Why shouldn't the denominator of the covariance estimator be n-2 rather than n-1?
The denominator of your definition of sample variance is $n−1$ probably because this makes it an unbiased estimate of the population variance. The same is true of the sample covariance using the same denominator. But there are other definitions of sample statistics, with different denominators.
Feb
21
comment When does replication reveal fraud?
"Statistics means never having to say you're certain"
Jan
25
comment How to tell the probability of failure if there were no failures?
If there are few or no failures then $\Theta \sim U(0,0.1)$ will produce almost the same result as $\Theta \sim U(0,1)$ i.e. Beta$(1,1)$, and the latter is easier to handle
Jan
12
comment Density estimation for streams of Data
You could store your data as counts in intervals, where the intervals are substantially narrower than the bandwith you are using for smoothing your kernel. Then you just need to remember as many numbers as there are intervals.
Jan
10
comment Measuring “almost the same” when significantly different
@Glen_b: I am looking for a summary measurement of "closeness" which is insensitive to scale (e.g. multiplying all the data by $10$) and can be used for meaningful comparisons with other splits of the same population by different factors and for comparisons with other cases for example when the success rates change.
Jan
10
comment Measuring “almost the same” when significantly different
There are other similar tables (such as gender, age, and ethnic origin) which show different disparities as well has having a variety of distributions between the size of the largest group and of the others. I was wondering if there was some sensible summary statistic that I had missed which like the $\chi^2$ statistic gave a greater weight to disparities in the larger groups but which was not so dependent on the total number of individuals as the $\chi^2$ statistic. For example the $\chi^2$ statistic divided by the total number of individuals would be unitless.
Jan
9
comment How to find $\displaystyle \dfrac{d}{dt} \left [\int_t^\infty xf(x)~dx \right ] $ (when $f(x)$ is a probability density function)
Use the fundamental theorem of calculus
Jan
9
comment How to find $\displaystyle \dfrac{d}{dt} \left [\int_t^\infty xf(x)~dx \right ] $ (when $f(x)$ is a probability density function)
Do you mean $\displaystyle \dfrac{d}{dt} \left [\int_t^\infty xf(x)~dx \right ]$? Perhaps $-tf(t).$ Or do you mean $\displaystyle \dfrac{d}{dt} \left [\dfrac{\int_t^\infty xf(x)~dx}{1-F(t)} \right ]$?
Jan
3
comment Explanation of formula for median closest point to origin of N samples from unit ball
I suspect it may be suggesting that in high dimensions, points to predict are effectively a long way from the training data, as if on the edge of a sphere, so you are not really interpolating but rather extrapolating, and so uncertainties are much greater. But I do not really know.
Dec
25
comment When to use a normal approximation of a Bernoulli distribution
@Victor - I used the MathJax version of TeX. See meta.math.stackexchange.com/questions/5020/… for some tips