3
votes
1answer
92 views

Should my test set be balanced or imbalanced?

I have an imbalanced dataset (90% class 0 10% class 1), should I first split it in training and test set, then balance my training set (my test set would still be imbalanced), or could I randomly ...
-1
votes
1answer
75 views

validating a regression model

I want to build a regression model to predict daily income from customers. I have 2 problems: 1. Choosing data for the training set - do I use daily income from 1 month ago, 6 month ago etc. 2. How do ...
2
votes
3answers
720 views

Back propagation neural network data input advice

I am currently doing a project which involves pothole detection and neural networks. So far, I have an Android phone that reads Accelerometer readings and writes the X,Y,Z Axis aswell as the Amplitude ...
23
votes
2answers
2k views

How to draw valid conclusions from “big data”?

"Big data" is everywhere in the media. Everybody says that "big data" is the big thing for 2012, e.g. KDNuggets poll on hot topics for 2012. However, I have deep concerns here. With big data, ...
4
votes
1answer
996 views

Best practices for measuring and avoiding overfitting?

I am developing automated trading systems for the stock market. The big challenge has been overfitting. Can your recommend some resources describing methods for measuring and avoiding overfitting? I ...
20
votes
4answers
483 views

As a reviewer, can I justify requesting data and code be made available even if the journal does not?

As science must be reproducible, by definition, there is increasing recognition that data and code are an essential component of the reproduciblity, as discussed by the Yale Roundtable for data and ...
3
votes
1answer
200 views

What resources/methods exist for testing/validation or evaluation of Statistical Methods

I'm interested in the process of testing or validating a particular implementation of a statistical method, and what datasets and/or published analysis exist that could be used to do this in practice. ...