3,194 reputation
824
bio website dataorigami.net
location Ottawa
age
visits member for 2 years, 3 months
seen 14 hours ago

ex-quant doing data @shopify. We're hiring data analysts, scientists and statisticians! Screencaster at www.dataorigami.net


Sep
10
comment What is the relationship between $Y$ and $X$ in this plot?
@chl you rock for donating to a bounty to whuber =)
Sep
8
comment PYMC Confusion: are observed nodes fixed or stochastic?
Ah, that's right! That's little-known PyMC thing. Thanks for bringing that back to my attention. Personally, I've moved away from Bayesian survival analysis for three reasons: i) computational difficulties - this post goes into them, and it can get worse. ii) I rarely need a point-estimate distribution when I perform survival analysis - I'm mostly interested in kaplan-meier curves. iii) I like the non-parametric nature of kaplan-meier estimates, and the semi-parametric form of Aalens. Bayesian SA requires a few too many strong assumptions.
Sep
7
answered PYMC Confusion: are observed nodes fixed or stochastic?
Sep
7
reviewed Approve suggested edit on What is the equation for an ARIMA (2,1,0)?
Sep
7
reviewed Approve suggested edit on p-values of Mann-Whitney U test identical for raw and log-transformed data
Sep
4
comment Inferring prior distribution
Good points @guy, for brevity and simplicity I chose the Beta distribution, though I also believe the Beta is flexible enough to capture most interesting distributions. I'm curious about what you mean by "because the prior will swamp the data eventually", have time to elaborate?
Sep
4
revised Inferring prior distribution
edited tags
Sep
4
comment Inferring prior distribution
@Uwat, when you have a even somewhat non-trivial model, the only way to perform Bayesian inference is with MCMC. I would recommend Bayesian Methods for Hackers (I'm the author btw) for an intro to Bayesian methods (chapter 1 & 2) and MCMC (chapter 3). For "reconstruct", it's best to not use an individual reconstruction. There are a few choices you could make: 1) use the average values of alpha, beta. 2) Use the MAP of alpha, beta (better). 3) Use the average over the distributions (also better)
Sep
3
comment E-mail answered probability after n days of waiting for a reply - based on a sample of e-mails and replies
I think survival analysis is the way to go here: it deals with time to events (in your case: reply). Specifically the Kaplan-Meier estimate is what you want. This gives you a new view of your data set, and inference is more natural in this setting.
Sep
3
revised Inferring prior distribution
added 108 characters in body
Sep
3
comment Inferring prior distribution
Here's the ipython %hist: gist.github.com/CamDavidsonPilon/1fed2295083e660b776a
Sep
3
answered Inferring prior distribution
Sep
3
comment Inferring prior distribution
This sounds like a hierarchical model. If I wanted to recreate the dataset, here's what I'd do: Let $D$ be $Beta(\alpha, \beta)$ (reasonable since we are dealing with probabilities). We don't know $\alpha, \beta$, so we assign priors to them, say exponential for both with some $\lambda$ hyperparameter. Then we draw the $p_i$ for each $i$, and sample $X_i$ from the binomials. Let me write something up...
Aug
27
awarded  Taxonomist
Aug
4
comment Name for the Bayesian posterior probability that a regression coefficient is larger than zero
@Zen yes I was too hasty - you are correct.
Aug
3
comment Name for the Bayesian posterior probability that a regression coefficient is larger than zero
Bayesian one-sided p-value is the best choice. Alternatively, you can give human-readable context (I'm assuming this is for a report or article?): the posterior probability the coefficient is greater than 0 is... - no need for potentially misleading names
Jul
25
reviewed Approve suggested edit on Estimation of regression with autocorrelated errors
Jul
13
comment Plain english explanation of the Rayleigh distribution?
I think your second example, with the hazard rate, is terrific and something I did not know
Jul
8
answered How to prepare interactions of categorical variables in scikit-learn?
Jul
2
awarded  Curious