3,289 reputation
925
bio website dataorigami.net
location Ottawa
age
visits member for 2 years, 6 months
seen 10 hours ago

ex-quant doing data @shopify. We're hiring data analysts, scientists and statisticians! Screencaster at www.dataorigami.net


Dec
8
comment Simulation of a process consist of Brownian motion and Poisson process
Why do you think P1 - P2 will behave like that? Your sample looks deterministic, but P1 - P2 will be random.
Nov
23
awarded  Good Question
Sep
27
revised Survival Analysis tools in Python
added 113 characters in body
Sep
24
awarded  Autobiographer
Sep
19
comment Is my Bayesian analysis correct?
+1 to that paper, I highly recommend it
Sep
19
comment Is my Bayesian analysis correct?
This line: sample_mean = sum(sample1+sample2). Did you want to divide by sample size?
Sep
10
comment What is the relationship between $Y$ and $X$ in this plot?
@chl you rock for donating to a bounty to whuber =)
Sep
8
comment PYMC Confusion: are observed nodes fixed or stochastic?
Ah, that's right! That's little-known PyMC thing. Thanks for bringing that back to my attention. Personally, I've moved away from Bayesian survival analysis for three reasons: i) computational difficulties - this post goes into them, and it can get worse. ii) I rarely need a point-estimate distribution when I perform survival analysis - I'm mostly interested in kaplan-meier curves. iii) I like the non-parametric nature of kaplan-meier estimates, and the semi-parametric form of Aalens. Bayesian SA requires a few too many strong assumptions.
Sep
7
answered PYMC Confusion: are observed nodes fixed or stochastic?
Sep
7
reviewed Approve What is the equation for an ARIMA (2,1,0)?
Sep
7
reviewed Approve p-values of Mann-Whitney U test identical for raw and log-transformed data
Sep
4
comment Inferring prior distribution
Good points @guy, for brevity and simplicity I chose the Beta distribution, though I also believe the Beta is flexible enough to capture most interesting distributions. I'm curious about what you mean by "because the prior will swamp the data eventually", have time to elaborate?
Sep
4
revised Inferring prior distribution
edited tags
Sep
4
comment Inferring prior distribution
@Uwat, when you have a even somewhat non-trivial model, the only way to perform Bayesian inference is with MCMC. I would recommend Bayesian Methods for Hackers (I'm the author btw) for an intro to Bayesian methods (chapter 1 & 2) and MCMC (chapter 3). For "reconstruct", it's best to not use an individual reconstruction. There are a few choices you could make: 1) use the average values of alpha, beta. 2) Use the MAP of alpha, beta (better). 3) Use the average over the distributions (also better)
Sep
3
comment E-mail answered probability after n days of waiting for a reply - based on a sample of e-mails and replies
I think survival analysis is the way to go here: it deals with time to events (in your case: reply). Specifically the Kaplan-Meier estimate is what you want. This gives you a new view of your data set, and inference is more natural in this setting.
Sep
3
revised Inferring prior distribution
added 108 characters in body
Sep
3
comment Inferring prior distribution
Here's the ipython %hist: gist.github.com/CamDavidsonPilon/1fed2295083e660b776a
Sep
3
answered Inferring prior distribution
Sep
3
comment Inferring prior distribution
This sounds like a hierarchical model. If I wanted to recreate the dataset, here's what I'd do: Let $D$ be $Beta(\alpha, \beta)$ (reasonable since we are dealing with probabilities). We don't know $\alpha, \beta$, so we assign priors to them, say exponential for both with some $\lambda$ hyperparameter. Then we draw the $p_i$ for each $i$, and sample $X_i$ from the binomials. Let me write something up...
Aug
27
awarded  Taxonomist