89,748 reputation
8150312
bio website quantdec.com
location Northeastern US
age 14
visits member for 4 years, 2 months
seen 20 mins ago

Consultant (environmental and spatial stats a specialty), expert witness, and teacher. I can be reached through (outdated but still valid) links posted on my web site.

Twitter: @WilliamAHuber // ASA-P website: http://amstatphilly.org/


Why waste time learning, when ignorance is instantaneous?

--T(iger) Hobbes.

For any complex problem there is a simple solution. And it's always wrong.

--[Mis?]attributed to H.L. Mencken by Dava Sobel, Longitude.


15m
comment References and Best practices for setting seeds in pseudo-Random Number Generation
Sure they are worthwhile--but that does not justify making them into definite "dos" and "don'ts" as expressed by that manual page. The problem with such uncategorical dicta is that others--such as lawyers--will be led to think that any contrary practice is inherently wrong, regardless of purpose or circumstances. It is important to leave room for judgment in the practice of statistics! In particular, let us please not confound recommendations for the use of software with "best practices."
24m
comment 5 defectives rule of thumb
The question concerns estimating the rate of defects rather than making sure that a defect is observed. Why not, then, take it at face value and examine the variance of an estimator of the rate? When the expected number is $5$, the variance of any reasonable estimator will be approximately $5$, making two standard deviations (a rough "error bar") equal to almost $100\%$ of the estimate. If that's OK for one's application, then this rule of thumb may be OK. But if one would like to obtain a more accurate estimate, then clearly far more items have to be sampled. BTW, $99.32605\%= 1-e^{-5}$.
32m
comment Minimum variance for sum of three random variables
Yes, applying the determinant restriction is correct. In fact, if you write the covariance matrix in the form $$\pmatrix{1&0&0\\0&2&0\\0&0&5}\pmatrix{1&\rho&\sigma\\ \rho&1&\tau\\ \sigma&\tau&1}\pmatrix{1&0&0\\0&2&0\\0&0&5}$$ you can directly apply the formulas from the other thread. They will show that you need only consider the cases $\pm\rho=\pm\sigma=\pm\tau=1$, from which you easily obtain the minimum of $4$ in your example (as well as a general formula for any three variables). Solutions for more than three variables are more challenging to obtain!
38m
comment Peak Hours for Tweeting
Why would anyone reasonably expect this model to apply to the responses to tweets? Sure, there might be some relationship between those responses and time of day or duration since some relevant event--but why would those responses exhibit any kind of stationary structure or even any kind of temporal correlation whatsoever? This post is so similar to several hundred nearly identical previous posts of yours that one is led to infer you are not giving any consideration to the question, but only reproducing your standard post for advertising purposes.
44m
comment References and Best practices for setting seeds in pseudo-Random Number Generation
That Stata manual page makes important implicit assumptions about why one is using a seed. The main reason I use seeds (in my postings here on CV) is to create reproducible examples. In order to demonstrate that I haven't fiddled with the seed until the example was to my liking(!), I (almost) always use the same seed. This so flagrantly contradicts the Stata advice because I have a different purpose than they must have in mind (which is unstated). The moral here is that best practices depend upon the purpose.
1h
comment 5 defectives rule of thumb
I think there's no question of appropriateness. The suboptimality issue is a matter of quantifying how erroneous your results might become as a result of using Normal-theory calculations for Poisson distributions. Take a look at Poisson distributions with intensities less than $5$ and decide whether they look sufficiently Normal to trust your analysis. They very well might be--but taking this step is necessary to support your answer.
2h
comment Solving a scrambled image puzzle with a genetic algorithm
@EngrStudent Randomization is of little help here, I believe. I don't understand the "endgame" idea. The pieces along the boundary of the puzzle will have one (or more) sides to which no edges are connected, but that does not seem to lead to any special problems. It is known in advance that $n$ rectangles making a rectangular puzzle will include $n^2 - 4n + 4$ edges so you simply stop the algorithm once those are found. One way to improve the algorithm would be to encode the piece sides at the outset in a data structure (a hash table, e.g.) allowing similar sides to be found in $O(1)$ time.
2h
comment 5 defectives rule of thumb
But if you are applying procedures inappropriate for the setting, why should anyone be convinced of what they imply? That hardly justifies calling the result "easily derivable" or "intuitive"! You are not rescued by large samples (and the implicit appeal to normality via the CLT), precisely because the events are so rare.
2h
comment 5 defectives rule of thumb
The confidence intervals you propose are inappropriate for rare events and small samples. A better confidence interval will automatically exceed zero. A limit for "large $n$" is not applicable here, because $n_1$ must be kept constant, whence $p$ is diminishing. This is the setting for a Poisson distribution analysis, not a Normal distribution analysis.
2h
reviewed Reviewed Minimum variance for sum of three random variables
2h
comment Minimum variance for sum of three random variables
Are you asking to solve the problem allowing the unknown covariances to be any values consistent with mathematical restrictions on covariances? Note that these are more restrictive than what you have inferred from the Cauchy-Schwarz Inequality.
3h
comment Help finding critical on a hypothesis contrast
I added the likelihood-ratio tag as a hint.
3h
revised Help finding critical on a hypothesis contrast
edited tags; edited tags
3h
revised Maximum Liklehood estimator of Poisson
edited tags
3h
comment Maximum Liklehood estimator of Poisson
Hint: Based on the table (which you haven't yet used, so it ought to be good for something!), for what value of $s$ for these data would the likelihood be maximized? (The whole thrust of (iv) seems to be to test your understanding of the relationships between maxima and the various values and derivatives of a function.)
3h
comment Looking for and dealing with collinearity in a GLM
What does the contingency table of host vs parasite look like? (From the single "control" count for parasites and single "1" count for hosts, it's already clear it will have some zero cells.)
3h
comment t test for slope with binary variable
@lowtech The binary variable here is independent: it merely distinguishes the two sets of samples.
3h
comment Detecting a trend to increase, in a time series, in real time
Welcome! I added the quality-control tag to give you some pointers to a collection of highly relevant techniques.
3h
revised Detecting a trend to increase, in a time series, in real time
edited tags
3h
comment Poisson Process
Exploit the homogeneity of the process. With each vehicle one of two things can happen: either a time gap of at least $a$ occurs, in which case the length of time to cross equals the current wait plus $a$, or else the gap--call it $X$--is smaller than $a$. In the latter case it's the same as if the whole process has restarted. Thus, the expected waiting time is increased in this case by the expectation of $X$ conditional on $X\lt a$ (which you need to compute in terms of $\lambda$). Thus you will obtain a recursive formula for $\mathbb{E}(T)$ (which is easily solved).