meta user
network profile
|
May 13 |
comment |
Why do we use k-means instead of other algorithms? More expensive as in slow and memory-intensive. |
|
Apr 30 |
comment |
Expected value and variance of log(a) I think the question was about the "inverse" of the log-normal, i.e. where a normal rv A leads to log-normal X = exp(A), the questioner was asking about the distribution of X = log(A), which is undefined (due to sometimes requiring the log of a negative number). There may be some results for a truncated normal, but they're likely to be messy. |
|
Aug 15 |
comment |
Is Perkins et al.'s “skill score” an application of Bayes' theorem? To talk about conditional probabilities you need "pairing" between the two variables. There needs to be a correspondence where X=x when Y=y. If you don't care about that relationship, you don't care about conditional probability. One area where you probably do care about it though, is between different modelled variables. You might want to talk about the distribution of temperatures given a particular ENSO index, for example. Similarly, you might look at the distribution of T conditional on a model parameter, then use Bayes' theorem to get a distribution for that parameter, given observations. |
|
Aug 10 |
comment |
Is Perkins et al.'s “skill score” an application of Bayes' theorem? The conditional distribution describes what one variable looks like at a particular value the other variable. In this case it's the distribution of observed values over days when the model predicts a temperature T (p(o|m=T)) or the distribution of model values when we observe a temperature T (p(m|o=T)). More loosely, conditional statistics are about the properties of paired values: how each model value relates to its corresponding observation. What you suggest is "valid", but it's still a measure of how similar the distributions of those measurements are, not the measurements themselves. |
|
Jul 30 |
comment |
SVD of a matrix with missing values You infer missing values the same way you'd predict future values, by using the relevant components of the SVD decomposition. Predicting future ratings is exactly the same problem as inferring missing values. If you've got a good way to infer missing values, just use that to predict future ratings. If you don't, then that's what SVD is for. |
|
May 20 |
comment |
Decomposing the normal distribution I was somewhat hoping the answer would be yes, but thanks! I don't have easy access to a copy of Feller - is it possible to sketch a proof of the theorem? It seems quite counterintuitive. |
|
Apr 9 |
comment |
random forest oob error increase as more tree build Out of bag, i.e. measured on the subset of the data not used to fit a particular tree. |
|
Apr 5 |
comment |
Standard errors in weighted least squares The kind of weighting scheme which is desirable always depends on what you're trying to do. I think we need to know more about what kind of questions you're trying to answer before we can give any advice. |
|
Apr 4 |
comment |
Cross-correlation of massive arrays In R on a similarly specced machine: a <- matrix(rnorm(1e7 * 16), 1e7, 16); system.time(cor(a)) takes about 32 seconds. I think there's something funny going on here. |
|
Apr 4 |
comment |
Probability of a randomly generated string being already present in a data set While this is definitely a very similar question to the suggested duplicate, it's not exactly the same. Given the level of the question, I don't think it's reasonable to assume that the questioner could infer the answer to this question from the (excellent) answer to the other. |
|
Apr 4 |
comment |
Why would R return NA as a lm() coefficient? $p > n$ is just a special case of colinearity - if there are fewer observations than predictors, colinearity is a given. You're right about interaction terms though, although I'm pretty sure that's not what's happening here. |
|
Mar 31 |
comment |
Is an implementation of a density function for a logit-normal distribution available in R? You can always calculate the logitnormal PDF analytically. The formula is on the Wikipedia page. |
|
Mar 29 |
comment |
Grouping strings by common themes? I've used the R topicmodels package quite a bit. I'd certainly recommend it over rolling your own code - there's some documentation with a worked example at cran.r-project.org/web/packages/topicmodels/vignettes/…. The specific formatting of each document doesn't really matter, as everything's going to be reduced to a "bag of words" representation anyway. Just throw all the associated text into one string. |
|
Mar 29 |
comment |
Bracket Theory and Probability It's not clear exactly what you mean by "accuracy" here. Do you mean the probability that the player who wins is the player with the objectively highest skill? Do you care about relative rankings of second, third place, etc? |
|
Mar 23 |
comment |
How to select discrete cosine transform coefficients as a feature vector? The arrangement of coefficients shouldn't matter to a neural network. Just drop the coefficients you don't want, and use what's left in whatever order is most convenient. |
|
Mar 23 |
comment |
Base classifiers for boosting I see what you're saying, but I still feel that "good enough" is a bit of an unsatisfying answer. I do take your point about the lack of fiddling with hyper parameters though. |
|
Mar 23 |
comment |
Base classifiers for boosting I think speed may be something of a red herring here - most of the time I'd rather have a more accurate classifier than something which is slightly faster to train. After all, CPU time is fairly cheap nowadays. |
|
Mar 22 |
comment |
Classification with one dominant predictor Thanks! I agree that boosting is likely to be a good way to approach this, and I'll look into the things you've suggested. I'm still interested to know if there's a good way to tackle this by transforming the problem. |
|
Mar 21 |
comment |
Looking for correct distribution for modelling foosball scoring The Elo rating system is named after Arpad Elo, and therefore isn't capitalized all the way through. |
|
Mar 20 |
comment |
How to use weights for imbalanced data in R's randomForest? This question confuses me - are you looking for a general way to do stratified sampling for regression forests (hard, because there's no obvious classes to weight), or is it for this problem in particular? This problem looks very much like a straight-up binary classification problem, in which case all you need to do is convert your buy/noBuy column to a factor. |
