David Robinson
  • Member for 10 years
  • Last seen more than a week ago
What is the intuition behind beta distribution?
Accepted answer
753 votes

The short version is that the Beta distribution can be understood as representing a distribution of probabilities, that is, it represents all the possible values of a probability when we don't know ...

View answer
How to understand the drawbacks of K-means
507 votes

What a great question- it's a chance to show how one would inspect the drawbacks and assumptions of any statistical method. Namely: make up some data and try the algorithm on it! We'll consider two ...

View answer
Mean squared error vs. mean squared prediction error
Accepted answer
30 votes

The difference is not the mathematical expression, but rather what you are measuring. Mean squared error measures the expected squared distance between an estimator and the true underlying parameter: ...

View answer
Bayesian batting average prior
Accepted answer
27 votes

Notice that: \begin{equation} \frac{\alpha\cdot\beta}{(\alpha+\beta)^2}=(\frac{\alpha}{\alpha+\beta})\cdot(1-\frac{\alpha}{\alpha+\beta}) \end{equation} This means the variance can therefore be ...

View answer
How is Poisson distribution different to normal distribution?
Accepted answer
26 votes

A Poisson distribution is discrete while a normal distribution is continuous, and a Poisson random variable is always >= 0. Thus, a Kolgomorov-Smirnov test will often be able to tell the difference. ...

View answer
R Language what is difference between rnorm and runif
23 votes

rnorm generates a random value from the normal distribution. runif generates a random value from the uniform.

View answer
Why are 0.05 < p < 0.95 results called false positives?
Accepted answer
15 votes

Your question is based on a false premise: isn't the null hypothesis still more likely than not to be wrong when p < 0.50 A p-value is not a probability that the null hypothesis is true. For ...

View answer
Mean and standard deviation of Gaussian Distribution
Accepted answer
12 votes

You can estimate them. The best estimate of the mean of the Gaussian distribution is the mean of your sample- that is, the sum of your sample divided by the number of elements in it. $$\bar{x} = \...

View answer
Q-Value Less than P-Value
Accepted answer
10 votes

Yes, this is possible, if the proportion of null hypotheses (which is estimated by the qvalue package based on your p-value distribution) is small and your test is powerful. Here's an example. Let's ...

View answer
Which independent variables are most important in predicting the response variable?
10 votes

First, note that your understanding of PCA is slightly off. PCA doesn't "group" variables into principal components. Each principal component is, rather, a new variable (a "new metabolite") of the ...

View answer
Arithmetic for updating likelihoods using Bayes theorem
Accepted answer
10 votes

I'll start by answering your question about updating events with the "fourth and fifth extensions." As you suspected, the arithmetic is indeed quite simple. First, recall how Bayes theorem is derived ...

View answer
Misbehavior of `summary.lm`
Accepted answer
8 votes

The reason for the difference is that you have + 0 in your formula, setting an intercept of 0. This means it is comparing the model not to one with y equal to mean(Elapsed), but one with y equal to 0. ...

View answer
how to apply hypothesis testing in this case?
Accepted answer
7 votes

First, let's rephrase your alternative hypothesis. You phrased it as "less than 50% of the individuals tested..." But by talking specifically about the individuals tested, we know whether it's true or ...

View answer
How can I convert a q-value distribution to a p-value distribution?
Accepted answer
6 votes

You can convert from a q-value distribution to a p-value distribution rather simply (indeed, it's easier than the other way around!). The way to do this in R is (explanation is in the comments): ...

View answer
Coupon collecting problem
5 votes

The only way that the product $X_iX_j=1$ is if both coupons i and j are selected. (Otherwise, either $X_i$ or $X_j$ will be 0, and the product will as well). Thus, $E[X_iX_j]$ is the probability ...

View answer
Make the Average smaller then the Median
Accepted answer
4 votes

There are no solutions without dropping one of the numbers below 0. Besides your arithmetic solution to show this, you can run a simulation in R. The following simulation tries adding various numbers ...

View answer
Logistic regression and discrepant sample sizes between 0/1 groups
Accepted answer
4 votes

It sounds like your problem is analogous to "Predict who will win the lottery based on whether they buy a lottery ticket." Now, whether someone buys lottery tickets is a very significant predictor of ...

View answer
How to compare variability within and between groups?
Accepted answer
4 votes

You can do this with an ANOVA analysis: my.locs$cluster = factor(rep(c(1, 2), each=5)) anova(lm(attribute ~ cluster, my.locs)) # Analysis of Variance Table # # Response: attribute # Df ...

View answer
Changing attributes from nominal to binary
Accepted answer
4 votes

Here's how the binary values for Apple, Orange and Pear are calculated. You know it has two binary digits (since k=3 and it has k-1 digits), so you compute each of those two digits: Apple is one of ...

View answer
How many possibilities is there to choose 10 boxes of given size?
Accepted answer
4 votes

Imagine that this person is planning out his choices in advance. He has 10 slots and has to assign each a letter- A, B, C, or D. First, he must put an A somewhere. He has 10 choices. Second, he must ...

View answer
Why does bigger $n$ mean higher chance to reject the null hypothesis?
Accepted answer
3 votes

Why does all of this mean that the bigger n is, the more easier it will be to reject the null hypothesis? That's true only when the null hypothesis is false (where $\mu_0$ is not the true mean). ...

View answer
Measure to use in experiment
3 votes

You didn't get the same information: you got a confidence interval of a certain width from measuring 40 people, and then after measuring 100 people you got a confidence interval of a (probably) much ...

View answer
Logistic regression in R with large amounts of data
Accepted answer
2 votes

Based on Very small p-values (less than 2e-16) Large effect size estimations (log-odds ratios ranging from .68 to 1.06) Small standard errors (around .015) All evidence indicates there is a ...

View answer
How much should the sample size be increased to reject the null hypothesis with a t-test?
2 votes

You're interested in the power of the t-test, given a certain effect size and standard deviation (that you've estimated with a smaller experiment). In R, this can be calculated with the power.t.test ...

View answer
Binomial regression asymptotes
2 votes

I would use the maximum of the X vector as the total possible number of successes. (This is a biased estimate of the true maximum number of successes, but it should work fairly well if you have enough ...

View answer
What is the distribution of q-values under the null
Accepted answer
1 votes

Under the null distribution, your true false discovery rate will always be equal to 100% (no matter what threshold you set, every hypothesis you accept will be a false discovery). Therefore, the true ...

View answer
p-value for hypothesis test with given correlation, sample size
Accepted answer
1 votes

As described here, you can use the formula to get a test statistic t, where n is the sample size and r is the calculated correlation. This test statistic t will follow a Student's t-distribution in ...

View answer