Tagged Questions

1
vote
2answers
71 views

Split dataset randomly

I have a database with 500 records. I want to split these records to 75% and 25% *randomly*in order to use the different datasets for training and testing to machine learning algorithms. Does anyone ...
-1
votes
1answer
68 views

Easy way to test usefulness of data for stock market analysis? [closed]

I have a lot of data (gigs) that may be useful in predicting equity prices. I can import these as a series of features (columns) in a table where the companies are rows. I have time series information ...
1
vote
0answers
61 views

Determining optimal height for regression tree

I have a data set of approximately 400,000 records (for those of you who know, the data set is the one provided by yahoo for their yahoo learning to rank challenge). From this data set I learn a ...
6
votes
1answer
1k views

R vs Python for Data Analysis [duplicate]

Possible Duplicate: Python as a statistics workbench I am just starting out with data analysis and machine learning. From the books that I am reading/have read Python and R seem to be the ...
0
votes
1answer
165 views

Distant supervision: supervised, semi-supervised, or both?

"Distant supervision" is a learning scheme in which a classifier is learned given a weakly labeled training set (training data is labeled automatically based on heuristics / rules). I think that both ...
2
votes
2answers
150 views

Comparing 2 classifiers with unlimited training data

I would like to compare 2 text classifiers C1 and C2, which can be trained with "unlimited" noisy training datasets, meaning that you can use as much data as you want for training, such data being ...
4
votes
1answer
84 views

Learning from relational data

Settings Many algorithms operate on a single relation or table, while many real-world databases store information in multiple tables (Domingos, 2003). Question What types of algorithms learn well ...
3
votes
0answers
272 views

High-dimensional Regression Datasets [closed]

Am looking for pointers to publicly(online) available high-dimensional regression datasets for evaluating my research work. By high-dimensional, am looking for regression datsets with the number of ...
0
votes
0answers
101 views

Possible projects for Data Analysis course? [closed]

I am taking a Data Analysis course and I need to do a project. I'd like to learn some possible hot topics that I can use for that project. Computer vision and image recognition are particularly ...
0
votes
0answers
104 views

Where can I find airline ticket prices datasets? [closed]

I am looking for statistics on international flight ticket prices that contains the departure city and the arrival city, the date of purchase and the departure date. The reason for this is that I want ...
2
votes
2answers
258 views

Baum-Welch training example

I'm the author of a new Baum Welch trainer using MapReduce for the Apache Mahout project (https://issues.apache.org/jira/browse/MAHOUT-627) I'm looking for an example with a reasonably small data set ...
8
votes
3answers
188 views

What is the most efficient way of training data using least memory?

This is my training data: 200,000 Examples x 10,000 Features. So my training data matrix is - 200,000 x 10,000. I managed to save this in a flat file without having memory issues by saving every ...
2
votes
3answers
122 views

What impact does increasing the training data have on the overall system accuracy?

Can someone summarize for me with possible examples, at what situations increasing the training data improves the overall system? When do we detect that adding more training data could possibly ...
5
votes
1answer
332 views

Can I have too much data?

I am training a large set of neural networks for a quite simple task. 10 of the networks have the same configuration, but have different amount of data. The 10 networks each have one hidden layer, ...
4
votes
2answers
156 views

How to classify country names given possible alternate spellings or abbreviations?

Let's say I have a list of users who have specified the country they reside in by typing in something. I want to find the total number of users who came from the US, the UK, and everywhere else. But, ...
5
votes
4answers
460 views

Benchmark dataset for decision tree algorithm

I'm implementing a decision tree algorithm, and I'd like to get a feel for how it performs relative to other implementations. Can anyone recommend popular datasets for training and testing decision ...
11
votes
7answers
970 views

What do statisticians do that can't be automated?

Will software eventually make statisticians obsolete? What is done that can't be programmed into a computer?
3
votes
1answer
105 views

Is it valid to judge the superiority of a classification technique using only artificial data?

... where the significance of the results has been checked with statistical tests. If no, then why? side note: The data is generated based on specified prior probabilities.
0
votes
1answer
216 views

Is there a corpus specifically for categories like sports, entertainment, or health?

I am experimenting with Classification algorithms in ML and am looking for some corpus to train my model to distinguish among the different text categories like sports, weather, technology, football, ...
0
votes
2answers
210 views

Factors that affect variation in the data?

What is the practical way to identify the factors that create variation in a data of a dataset? What category does this question fall into? Are there a set of algorithms that can be used for this ...
4
votes
1answer
637 views

Best practices for measuring and avoiding overfitting?

I am developing automated trading systems for the stock market. The big challenge has been overfitting. Can your recommend some resources describing methods for measuring and avoiding overfitting? I ...
2
votes
2answers
256 views

Learning multiple output

Can you suggest me an algorithm and probably a real code for multiple output learning, where input of the model is vector of around 10 000 values and output is, for each input vector, an output vector ...
2
votes
1answer
176 views

Interpretation of “one” feature change in a supervised classifier

i'm making experiments using app. 5000 labeled dataset.i'm trying different supervised ML algorithm to evaluate the results.The vector size is 13 with the labels (totally 12 features+1 label) and i ...
13
votes
5answers
2k views

Free data set for very high dimensional classification

What are the freely available data set for classification with more than 1000 features (or sample points if it contains curves)? There is already a community wiki about free data sets: ...