7,631 reputation
1936
bio website thinkinator.com
location Washington DC, United States
age
visits member for 4 years, 1 month
seen 8 hours ago

Long-time R user, new Stata user, occasional Stan user, and currently learning Scala. Data Scientist at ERI, an incredible company where we do everything from fraud detection and visualization, to training and and text mining.


Dec
6
awarded  Notable Question
Dec
3
answered What is a good Gini decrease cutoff for feature inclusion based upon random forests?
Dec
2
comment Questions revolving GMM & EM
Supervised v Unsupervised has to do with whether the algorithm uses training data that is labeled or not, respectively. You're correct that specifying or learning K has a supervised or unsupervised flavor to it, but that application of the terms would be non-standard and misleading.
Dec
2
comment clustering vs fitting with a distribution
Have you looked at stats.stackexchange.com/questions/69424/…
Dec
1
comment Taking only a single data set from a multiple imputation?
EdM: Very good thought, I'll look into it. Not sure how easy it will be to tap into the output of downstream code, combine things, and re-inject for even further-downstream. I'd still be interested in statistical answers as well.
Dec
1
asked Taking only a single data set from a multiple imputation?
Nov
28
comment Creating a “certainty score” from the votes in random forests?
From R's randomForest package help: $type$: one of "response", "prob", or "votes", indicating the type of output: predicted values, matrix of class probabilities, or matrix of vote counts.
Nov
26
comment Marginalising over standard deviation of normal to get the posterior on mean
Sounds like you'd need to create your own step method in pymc, or perhaps you could use a Beta distribution instead of Normal. I'm not an expert on this, though.
Nov
22
comment How can I analyze my incoming email?
Word clouds are colorful, entertaining, and a whole host of things, but they're not useful for any kind of analysis. Perhaps if you were writing poetry or a song based on the words, but not for analysis.
Nov
21
comment How can I analyze my incoming email?
@Superbest: 1) Wolfram's exploratory approach is useful, even if specific measurements are not applicable in your case. 2) Even if the timing of incoming emails is not important for announcements from a particular source, that doesn't mean it's not important in other cases. 3) Word clouds are a waste of time.
Nov
21
comment Is it unusual for MEAN to outperform ARIMA?
+1 this is very important. It's easy to think you have an impressive forecast until you compare it to the naive random walk. Really deflates you. And I believe that it was the best weather forecasting algorithm until the 1990's... Or something like that.
Nov
17
comment Graduate Degree Choices for Data Science?
@StasK: I like Stata a lot -- I bought it last year -- and did not mean to criticize it or its users. It's more popular with economists than statisticians, I'd guess. The point I was trying to make is that the stereotype that excludes "Statistician" from "Data Science" is a perception that Data Science is all about big data (Hadoop, huge SQL, etc) that's often free-form-ish (text mining, etc) and requires teams of Data Scientists, while statisticians work independently on experiments, etc. It's a stereotype, of course.
Nov
15
comment Graduate Degree Choices for Data Science?
+1 for both of your points. A lot of my co-workers went through the NC State program. I'd add that a computer science M.S. in a program that's machine-learning-heavy would also be a good option -- assuming the OP is reasonably skilled at programming, which they'll need anyhow to be a "Data Scientist". I think the bias against statisticians is the stereotype that they are more theoretical or that they use Stata, etc, to work on smaller-scale problems. In terms of economics, I have to say that my favorite stats bloggers are economists.
Nov
14
comment Is it valid to use quantile regression with only categorical predictors?
@Scortchi: ah, my poor reading comprehension! So then the question boils down to "is it valid to use regression with only categorical predictors?"
Nov
13
comment Is it valid to use quantile regression with only categorical predictors?
How would you define 'quantile' in a categorical context?
Nov
4
comment Software for easy-yet-robust data exploration
+1 for Stata. You can do all of the basics through point-n-click, but it also spits out the command that point-n-click generated so you can learn/modify it. It's also spreadsheet-like in how you interface with its data, though you don't put formulas in cells of course.
Nov
4
comment How to calculate p value from distribution of test statistic
@Phil2014: In R, for example, if your test statistics are in the vector X, you would do something like quantile (X, c(0.025, 0.975)) (two-sided) or quantile (X, 0.95) (one-sided).
Nov
1
awarded  Yearling
Oct
29
comment Logistic Regression - Model Does not Fit
Please clarify: "the likelihood of the question getting answered" refers to having one or more answers posted, or to having an answer accepted?
Oct
29
comment Logistic Regression - Model Does not Fit
I'd disagree with your hypothesis that the number of comments is correlated with having an answer. People tend to comment on a question instead of answering, either because the question is unclear and needs work, or because they think that the answer they have is too trivial to put as an actual answer. See my next comment...