Tagged Questions
3
votes
1answer
30 views
Should my test set be balanced or imbalanced?
I have an imbalanced dataset (90% class 0 10% class 1), should I first split it in training and test set, then balance my training set (my test set would still be imbalanced), or could I randomly ...
-1
votes
1answer
62 views
validating a regression model
I want to build a regression model to predict daily income from customers. I have 2 problems:
1. Choosing data for the training set - do I use daily income from 1 month ago, 6 month ago etc.
2. How do ...
2
votes
3answers
546 views
Back propagation neural network data input advice
I am currently doing a project which involves pothole detection and neural networks. So far, I have an Android phone that reads Accelerometer readings and writes the X,Y,Z Axis aswell as the Amplitude ...
16
votes
2answers
2k views
How to draw valid conclusions from “big data”?
"Big data" is everywhere in the media. Everybody says that "big data" is the big thing for 2012, e.g. KDNuggets poll on hot topics for 2012. However, I have deep concerns here. With big data, ...
4
votes
1answer
671 views
Best practices for measuring and avoiding overfitting?
I am developing automated trading systems for the stock market. The big challenge has been overfitting. Can your recommend some resources describing methods for measuring and avoiding overfitting?
I ...
19
votes
4answers
418 views
As a reviewer, can I justify requesting data and code be made available even if the journal does not?
As science must be reproducible, by definition, there is increasing recognition that data and code are an essential component of the reproduciblity, as discussed by the Yale Roundtable for data and ...
2
votes
1answer
174 views
What resources/methods exist for testing/validation or evaluation of Statistical Methods
I'm interested in the process of testing or validating a particular implementation of a statistical method, and what datasets and/or published analysis exist that could be used to do this in practice.
...
