0
votes
1answer
20 views

how to compute minimum required vc dimension for a classifer to classify a specific data

Suppose we're given an N dimensional data to classify. To cope with this task we may choose a classifier that suits our desires more. However obviously not every classifier is capable of classifying ...
0
votes
0answers
17 views

What is data augmented by the additive inverse?

I am reading Biclustering of expression data (Cheng and Church, 2000) The paper is about the Cheng and Church biclustering algorithm and its main metric, the mean squared residue (MSR). It is said ...
1
vote
0answers
11 views

What sort of feedback button should I implement to measure the difficulty of a task

---background info--- My company has built a Search Engine Optimisation product that can suggest monthly prioritised SEO activities (task suggestions), that will have the most benefit to the end user. ...
1
vote
0answers
20 views

What do I call a set of datasets

For training a Machine learning model, I have 3 datasets: Training Validation Testing Normally I obtain there by dividing up the full dataset into pieces. I've created a function in my code to ...
0
votes
0answers
25 views

How to reduce the dimension of a test data and make it uncorrelated?

I am working on classification of 16000 cell images. Each of them consists of 706 features related to intensity, morphology, colocalisation and texture of the cell. The train set consists of 11 ...
1
vote
1answer
47 views

Difference in meaning of these terms: Dataset vs Corpus

I have a question pertaining to definition of terms that I couldn't find answer for. What's the difference between Dataset and Corpus? I've seen them being used almost interchangeably. My ...
0
votes
1answer
72 views

data normalisation problems

I am pretty new in machine learning and hence facing a lot of confusion in data normalisation concepts. Someone pls clarify the following doubts : 1) While normalising a data matrix of m-samples x ...
0
votes
2answers
120 views

How to use the datasets to practice machine learning algorithms?

I'm a beginner in ML, and I want to practice with some algorithms I learned from the book, after searching stats.SE, I found some places where I can get some datasets. But I have this stupid question, ...
1
vote
0answers
85 views

Organizing data to feed random forests

I'm willing to apply machine learning with R (I will start with random forests then maybe have a look at NNs) on some data, but I don't know where to start, ...
1
vote
1answer
91 views

Machine-learning input data distribution

I'm trying to build a binary 1/0 ML classification algorithm, and was thinking about how to set up the input dataset. If the event I want to predict (the 1's) occur relatively less frequently in the ...
1
vote
1answer
75 views

Entropy in decision tree classification

In the decision tree based classification technique. What is the difference among the different approaches like entropy, gini index? When to use entropy and when to use gini index?.
2
votes
1answer
46 views

How to classify data having sub-instance features?

I am trying to use machine learning on some peculiar (at least for me) data. Usually, when I do machine learning I am use to have the data in this format: ...
-2
votes
1answer
67 views

What are different measures that say something about a given dataset? [closed]

Given a data set (having n instances and m features), what are the different measures that give some insight about some properties of the data set? In other words, if there are two such data sets, ...
0
votes
2answers
85 views

Can I replenish my dataset with unlabeled data?

I have a database of time series data generated by giroscope sensor. A small part of the data is already labeled and used to fit the model. My strong assumption is that most of the samples left in ...
1
vote
0answers
39 views

Discretizing all attributes vs. discretizing only the class label

I have a numeric dataset and I would like to apply classification algorithms on it. So I am using Weka's Discretize filter to convert the numeric values to discrete. However I wonder what can be the ...
1
vote
2answers
152 views

Split dataset randomly

I have a database with 500 records. I want to split these records to 75% and 25% *randomly*in order to use the different datasets for training and testing to machine learning algorithms. Does anyone ...
-1
votes
1answer
89 views

Easy way to test usefulness of data for stock market analysis? [closed]

I have a lot of data (gigs) that may be useful in predicting equity prices. I can import these as a series of features (columns) in a table where the companies are rows. I have time series information ...
1
vote
0answers
74 views

Determining optimal height for regression tree

I have a data set of approximately 400,000 records (for those of you who know, the data set is the one provided by yahoo for their yahoo learning to rank challenge). From this data set I learn a ...
8
votes
1answer
3k views

R vs Python for Data Analysis [duplicate]

Possible Duplicate: Python as a statistics workbench I am just starting out with data analysis and machine learning. From the books that I am reading/have read Python and R seem to be the ...
3
votes
1answer
746 views

Distant supervision: supervised, semi-supervised, or both?

"Distant supervision" is a learning scheme in which a classifier is learned given a weakly labeled training set (training data is labeled automatically based on heuristics / rules). I think that both ...
2
votes
2answers
227 views

Comparing 2 classifiers with unlimited training data

I would like to compare 2 text classifiers C1 and C2, which can be trained with "unlimited" noisy training datasets, meaning that you can use as much data as you want for training, such data being ...
7
votes
2answers
109 views

Learning from relational data

Settings Many algorithms operate on a single relation or table, while many real-world databases store information in multiple tables (Domingos, 2003). Question What types of algorithms learn well ...
4
votes
0answers
419 views

High-dimensional Regression Datasets [closed]

Am looking for pointers to publicly(online) available high-dimensional regression datasets for evaluating my research work. By high-dimensional, am looking for regression datsets with the number of ...
2
votes
2answers
388 views

Baum-Welch training example

I'm the author of a new Baum Welch trainer using MapReduce for the Apache Mahout project (https://issues.apache.org/jira/browse/MAHOUT-627) I'm looking for an example with a reasonably small data set ...
9
votes
3answers
252 views

What is the most efficient way of training data using least memory?

This is my training data: 200,000 Examples x 10,000 Features. So my training data matrix is - 200,000 x 10,000. I managed to save this in a flat file without having memory issues by saving every ...
2
votes
3answers
192 views

What impact does increasing the training data have on the overall system accuracy?

Can someone summarize for me with possible examples, at what situations increasing the training data improves the overall system? When do we detect that adding more training data could possibly ...
5
votes
1answer
404 views

Can I have too much data?

I am training a large set of neural networks for a quite simple task. 10 of the networks have the same configuration, but have different amount of data. The 10 networks each have one hidden layer, ...
4
votes
2answers
230 views

How to classify country names given possible alternate spellings or abbreviations?

Let's say I have a list of users who have specified the country they reside in by typing in something. I want to find the total number of users who came from the US, the UK, and everywhere else. But, ...
6
votes
4answers
963 views

Benchmark dataset for decision tree algorithm

I'm implementing a decision tree algorithm, and I'd like to get a feel for how it performs relative to other implementations. Can anyone recommend popular datasets for training and testing decision ...
12
votes
7answers
1k views

What do statisticians do that can't be automated?

Will software eventually make statisticians obsolete? What is done that can't be programmed into a computer?
3
votes
1answer
112 views

Is it valid to judge the superiority of a classification technique using only artificial data?

... where the significance of the results has been checked with statistical tests. If no, then why? side note: The data is generated based on specified prior probabilities.
0
votes
1answer
344 views

Is there a corpus specifically for categories like sports, entertainment, or health?

I am experimenting with Classification algorithms in ML and am looking for some corpus to train my model to distinguish among the different text categories like sports, weather, technology, football, ...
0
votes
2answers
245 views

Factors that affect variation in the data?

What is the practical way to identify the factors that create variation in a data of a dataset? What category does this question fall into? Are there a set of algorithms that can be used for this ...
4
votes
1answer
993 views

Best practices for measuring and avoiding overfitting?

I am developing automated trading systems for the stock market. The big challenge has been overfitting. Can your recommend some resources describing methods for measuring and avoiding overfitting? I ...
2
votes
3answers
388 views

Learning multiple output

Can you suggest me an algorithm and probably a real code for multiple output learning, where input of the model is vector of around 10 000 values and output is, for each input vector, an output vector ...
2
votes
1answer
188 views

Interpretation of “one” feature change in a supervised classifier

i'm making experiments using app. 5000 labeled dataset.i'm trying different supervised ML algorithm to evaluate the results.The vector size is 13 with the labels (totally 12 features+1 label) and i ...
16
votes
5answers
4k views

Free data set for very high dimensional classification

What are the freely available data set for classification with more than 1000 features (or sample points if it contains curves)? There is already a community wiki about free data sets: ...