Reputation
11,686
Top tag
Next privilege 15,000 Rep.
Protect questions
Badges
1 13 33
Impact
~398k people reached

Aug
29
comment Monty Hall (goat) problem w. elementary events
@left4bread: in summary, a major issue with your counting argument is that you seem to think that $(a,a,c,?)$ is as likely to happen as $(a,b,c,?)$. This is not true
Aug
29
comment Monty Hall (goat) problem w. elementary events
... Meanwhile the case of (a,b,?,?) also has probability $\frac{1}{9}$. The moderator can only choose $c$. So the case $(a,b,c,?)$ has probability $\frac{1}{9}$: overall there are $6$ three-state cases with probability $\frac{1}{9}$ and in these the user will win by switching. So the total probability that the user wins given that the user switches is $\dfrac{6}{9}=\dfrac23$
Aug
29
comment Monty Hall (goat) problem w. elementary events
@left4bread: The case of (a,a,?,?) has probability $\frac{1}{9}$. The moderator can choose $b$ or $c$. So the case $(a,a,b,?)$ has probability $\frac{1}{18}$ and the case $(a,a,c,?)$ also has probability $\frac{1}{18}$: overall there are $6$ three-state cases with probability $\frac{1}{18}$ and in these the user will lose by switching (the user has a strategy at the end and is not picking at random). So the total probability that the user loses given that the user switches is $\dfrac{6}{18}=\dfrac13$
Aug
26
comment Why splitting the data into the training and testing set is not enough
I would reverse the wording in your second and third paragraphs: I would use the validation set to find the best model and tune its hyperparameters (doing this with several validation sets which partition the training set makes this cross validation) and, once the model has been finalised, then apply it to the test set to see an example of the model's out-of-sample performance.
Aug
26
comment Why splitting the data into the training and testing set is not enough
Personally, I would not use the phrase "external cross validation", as I would see cross validation as the repeated splitting off of different validation sets from the training set for model selection and tuning purposes. You cannot meaningfully do this repeatedly with the test set, as that is as a one-off proxy for future as-yet-unknown data used to judge the performance of the final model.
Aug
4
comment Derivation of the equation of ridge regression
@bill: here you need the $I$ to get a matrix of the correct dimension so the addition works with $X^TX$: $\lambda$ is just a scalar
Jul
28
comment what does linear regression actually mean?
@AlecTeal: It is not my graph, but heights are in inches and the graph type is a sunflower plot with strokes coming out of each point (strictly speaking a range) representing the number of observations there. Look at cran.r-project.org/web/packages/HistData/HistData.pdf under "Galton" for some example code
Jul
15
comment Is there a general/golden rule for appropriate binning in a histogram?
Wikipedia list some different approaches but with over a million data points you could just choose bin widths of $0.01$ starting at $0$, so about $60$ bins and see if that causes an issue with the histogram
Jul
14
comment How to solve this problem on Curse of Dimensionality problem - Nearest Neighbours
@MarkL.Stone: Call a hypersphere an $n$-sphere if you prefer. For me volume is 3-dimensional while hypervolume is the measure at the highest dimension being considered.
Jul
3
comment Statistical Significance
If you are going to use the future perfect then perhaps you should move it out of the indicative, e.g. to "would have been rejected". You may also need to say the the "1%" represents the probability of erroneously rejecting the null hypothesis when it is in fact true, so "5%" involves that possibility and more.
Jun
19
comment What can we say about population mean from a sample size of 1?
This seems to give confidence intervals covering the mean with probability about $95\%$ when $\sigma \approx | \mu | \gt 0$ but with much higher probabilities otherwise. If $\mu = 0$ then clearly the probability is $100\%$ as the confidence intervals always contain $0$.
Jun
13
comment Percentile vs quantile vs quartile
A deeper question is whether quantiles etc. are intervals or points.
Jun
12
comment Estimating median survival times from Kaplan-Meier plot inspection
The data might show you that 16 out of 29 of the "mutation present" cases survived more than 4000 days (or whatever the numbers are), which is over 10 years. How would you work out the median from this?
Jun
11
comment Why is a symmetric distribution sufficient for the sample mean and variance to be uncorrelated?
@Glen_b: I am well aware that zero correlation does not imply independence. In fact my two bullet points were designed to show this difference. By the way, looking at your deleted answer, I thought about the Cauchy example in your second half and so said "if they have a correlation", while your first half deals with the "only if" point which my mine did not (the question seemed confused about this); so you might think about undeleting it.
Jun
10
comment Correlation coefficient is very small
Visually the relationship looks very weak on the right-hand side and if the regression line had been horizontal (something like $y=4$) I would not have been surprised
May
25
comment Standard Error of the ratio of Binomial variates
$Y$ has a positive probability of being $0$ (so too does $X$) which is going to cause issues with $\frac {X-Y}{Y}$
May
17
comment Clarify probability solution re. birthdays
Remember that (in some countries at least) births on Saturday and Sunday are less common than on other days because doctors like not working at weekends.
May
17
comment Combine several days of time series into one
Yes you can, though whether a simple linear regression would be meaningful is another matter (e.g. if you were measuring tiredness, there might be a cyclical pattern)
May
12
comment Poker and the Birthday Problem
You can make order matter in your poker hands and have $\dfrac{52!}{47!}$ possible hands, i.e. $5!$ as many. You could easily adjust the rest of your analysis to fit this. But this option to choose whether to make order matter or not does not work so well with sampling with replacement especially if you want to use counting arguments.
May
10
comment Should I use A/B testing to find evidence that click-through rate has changed over four months?
I would have thought the rate was $35/100$. You can certainly do something like what you are suggesting, but I would not all it A/B testing, which I would have thought meant showing different ads to different people in the same time period and comparing response rates