meta user
network profile
| bio | website | moderntoolmaking.blogspot.com |
|---|---|---|
| location | San Francisco, CA | |
| age | 26 | |
| visits | member for | 2 years, 4 months |
| seen | 1 hour ago | |
| stats | profile views | 645 |
|
7h |
comment |
Search for interactions using carets rfe function @dafeda Great! Also, you may need to convert back from a matrix to a data.frame. I'm not sure if you can use a matrix with rfe. |
|
8h |
answered | Search for interactions using carets rfe function |
|
1d |
awarded | Popular Question |
|
1d |
answered | Drawing inferences from word counts |
|
1d |
comment |
Use Random Forest model to make predictions from sensor data @dafeda Hah, thanks! |
|
1d |
answered | Use Random Forest model to make predictions from sensor data |
|
1d |
comment |
Scikit-learn reports memory error when fitting Gaussian process model Furthermore, the next line (53) is going to cause you even more problems, when it tries to allocate a 1.25e9 x 150 element array. By my calculations (which could be incorrect) this array will require almost 700GB of RAM! |
|
1d |
comment |
Scikit-learn reports memory error when fitting Gaussian process model It looks like python is trying to create 50,000 x 25,000 = 1.25e9 row by 2 column array. This works out to an array with 2.5e9 elements. Assuming integers in python take 32 bits of RAM, thats roughly 9.3 GB of RAM. It possible that you've already used 7GB of RAM by the time you get to this point in the code. I don't think there's much you can do, except reduce the training set size, get more RAM, or try a different model. |
|
1d |
comment |
Scikit-learn reports memory error when fitting Gaussian process model How much RAM do you have? Perhaps you need more. Are all your features numeric? I'm not familiar with scikit-learn, but I know that in R if you accidentally include a factor variable with many levels in your model it can cause you to run out of RAM. |
|
May 19 |
comment |
Variable selection / Dataset reduction for large datasets (in R) Maybe do it a few times with a 20k subset, and look how frequently each feature gets selected? |
|
May 17 |
comment |
Variable selection / Dataset reduction for large datasets (in R) #4. Seems like a silly idea if you have too many variables... This turns each continuous variable into a several categorical ones, which adds a lot more variables to your dataset. Try a simple logistic regression using glm and see how long it takes to fit. Also try taking a random sample of your dataset (maybe 10k rows) to test your modeling workflow. |
|
May 16 |
comment |
Why isn't the Dantzig selector popular in applied statistics? I suspect because it is fairly new, and, as far as I can tell from 5 minutes of googling, no standard implementation exists that applied statisticians could use on their own data. It seems like a cool method. If the authors would write a package for R and python to implement their method, I'd love to try it out. |
|
May 15 |
comment |
Exogeneous regressors in auto.arima and using them in forecast function in R @user2370334 Good catch. The argument is xreg. Type ?forecast.Arima for more info. |
|
May 15 |
answered | Exogeneous regressors in auto.arima and using them in forecast function in R |
|
May 13 |
revised |
Which glm algorithm to use when predictors are numerical as well as categorical? deleted 38 characters in body |
|
May 13 |
answered | Which glm algorithm to use when predictors are numerical as well as categorical? |
|
May 9 |
comment |
R libraries for deep learning Exactly. Theano is very alluring! |
|
May 8 |
comment |
R libraries for deep learning I've seen this code too-- thanks for linking to it. It's good to see RBMs starting to show up in R, but I feel like R is still years behind python in terms of deep learning. I'd love to see some full featured libraries for modern neural nets! |
|
May 8 |
accepted | R libraries for deep learning |
|
May 2 |
revised |
Appropriate statistical test to test if probabilities are accurate added 1 characters in body |
