training (or estimation) of statistical models or algorithms.

learn more… | top users | synonyms

0
votes
0answers
4 views

R Caret train / rfe optimize for positive predictive value instead of Accuracy or Kappa

In train or rfe I can only set Accuracy or Kappa. Is there a way to edit the functions to define a scoring function? I am using Kappa at the moment but I need to optimize for positive predictive Value ...
0
votes
1answer
17 views

Large number of positive labels in classifier when actual population has few

I have been tasked to help with a sort of classifier. In the make up of the problem the set we want to identify as "Positive" is know to be very very small. However the training set I have been given ...
0
votes
0answers
8 views

R caret: leave subject out cross validation with data subset for training? [migrated]

I want to perform leave subject out cross validation with R caret (cf. this example) but only use a subset of the data in training for creating CV models. Still, the left out CV partition should be ...
0
votes
1answer
19 views

Rolling forecasts: training versus forecast accuracy evaluation

Questions: Are rolling forecast examples (like the ones below) only useful for evaluating a model's accuracy, or can a rolling forecast be used to train a model? Are models trained using a rolling ...
0
votes
0answers
12 views

Combining results from tests after re-shuffling data

I am fine-tuning a neural network for a binary multilabel classification problem. Basically I am trying to predict 64 binary labels for each input. However, my dataset is somewhat small for the task ...
0
votes
2answers
17 views

What do we learn from a test set?

Suppose I split my data into two parts -- a training set (having 80% of my data) and a testing (20%) set. I train a model on my training set, and test it on the test set. What do we learn from ...
0
votes
0answers
10 views

Training vs. Testing Score [duplicate]

In a recent analysis I'm doing, I found that after splitting my data into 70% training data and 30% testing data, my training score is slightly lower than my testing score. To my knowledge, ...
0
votes
0answers
9 views

Compare the quality of two train sets

I wonder what is the right way to evaluate the quality of the dataset. I have two train sets and one fixed test set. I want to show that one train set is better than the other. The simplest way is ...
0
votes
1answer
37 views

Choosing number of samples to train a model

(On behalf of a colleague) I have performed some modelling based on a naïve Bayes classifiers model (weighted genomic risk score) and obtained reasonable ROCAUC results (used ROCR, pROC, and SDMtools ...
0
votes
0answers
18 views

Using two different methods for training and inferring

I have a model which is intractable to take derivatives wrt parameters and estimate them based on maximum likelihood. Even with an deterministic approximation like variational inference this is ...
0
votes
1answer
87 views

How to train convolutional neural networks with multi-channel images?

I have $m$ labeled images, each with 224x224 pixels and 5 different image channels. What is the best way to train a CNN architecture using this data when $m$ is small (less than 2000)? Is it possible ...
0
votes
1answer
27 views

Backpropagate multiple hidden layers

I have created a feed forward neural network using the sigmoid activation function and backpropagation. I was wondering if I would be able to use backpropagation the same way for two hidden layers as ...
2
votes
0answers
35 views

Targets of 0.1/0.9 instead of 0/1 in neural networks and other classification algorithms

Rumelhart, Hinton and Williams (PDF) wrote in 1986 in the context of training a neural network (page 12): One other feature of this activation function should be noted. The system can not ...
0
votes
0answers
66 views

Need to resize features in XGBoost

I am scaling all the numeric features in my train data in values between 0 and 1 but is it truly necessary? Does the algorithm performance is improved doing so or it can deal with different ranges of ...
0
votes
0answers
25 views

Bootstrap sampling to evaluate a model

In order to properly estimate model prediction performance I use bootstrapping to estimate the confidence interval of the performance measure. I perform a repeated random sampling to generate $B$ ...
1
vote
0answers
152 views

What is the minimum sample size required to train a Deep Learning model - CNN?

It is true that the sample size depends on the nature of the problem and the architecture implemented. But, on average, what is the typical sample size utilized for training a deep learning framework? ...
0
votes
0answers
31 views

The meaning of “training accuracy”?

If I split my data set into testing, training (further separated into subtraining and validation data set in cross-validation). In the context of machine learning and esp. in those ROC comparing the ...
0
votes
2answers
75 views

How to create a variable that is present in test data set but not in train?

Im try to do a classification but i have a variable production budget which is present in test dataset and not in train. so how do i proceed. could i impute that variable somehow. i dont want to drop ...
1
vote
1answer
283 views

How do I use deep learning (Convolution Neural Network) with small training data-set?

I have an image data set of around 180 images in 60 classes (3 images per class). I am able to build a classifier using feature matching. However, I want to try Convolution Neural Networks and see if ...
0
votes
0answers
23 views

How do I consider the whole training set in the process of Hidden Markov Model training?

I'm trying to train a Hidden Markov Model following theory from the book "Pattern Classification" by Duda, Hart, and Sotrk. For the HMM learning they discuss Forward-Backward algorithm there, ...
1
vote
1answer
35 views

Can I use cross validation on a subset of the training set to select hyperparameters?

I am using R, and I had a dataset with 400000 rows and 800 columns, training a random forest model with only 100 trees on this dataset will take me about 1 and half hour on my laptop. So I went on and ...
3
votes
3answers
129 views

Why does the training error usually underestimate the test error?

I understand that most algorithms are optimized to minimize the training error but why is the test error usually larger then the training error? Is there a statistical reason why?
0
votes
1answer
40 views

Real world model training in R: how to get instant feedback?

I want to train a model. I can just randomly choose method (e.g. random forest), put whole dataset, wait a few hours, check accuracy, plot every possible curve (like accuracy vs train size) and see ...
4
votes
4answers
356 views

Does increase in training set size help in increasing the accuracy perpetually or is there a saturation point?

I am using a boosted trees classifier which is giving better accuracy then all other linear classifier I tried. I have almost an unlimited training data at my disposal , I wanted to know if there is a ...
4
votes
1answer
131 views

ML / train-test-validate: What is allowed when?

As someone getting started in machine learning, I am trying to get my head around the rules / good practices to follow when building, testing and validating supervised ML models in order not to ...
0
votes
1answer
51 views

Logistic Regression with empty cells

I have a data set from which I need to train a model and use it for prediction. Let's say I want to predict what people say about food items produced by a cake shop. Let's assume people have stated ...
0
votes
1answer
45 views

Feeding categorical data to classifier

Suppose I have the dataset in the following format: ...
0
votes
2answers
72 views

Trying to understand how I should split data between (train and test) Vs. predict?

I'm working on a model to predict churn. I understand the concept of training and testing, or at least I thought I did. Let's say it's the first of the month and our database has 10,000 subscribers, ...
0
votes
1answer
63 views

GBM Performance on different sampling techniques

I am working on a healthcare data set for breast cancer patients. This data set is class imbalances and the distribution of positive and negative classes is 80%/20%. In order to deal with the class ...
1
vote
3answers
116 views

splitting a training data set based on classifier accuracy

I have training data with ~600K instances. If I split the training data into four segments and build four separate classifiers, I get much higher accuracy for each model than if I train a single model ...
3
votes
1answer
81 views

Minimum length of training data?

I want to obtain a prediction model using support vector regression on a time series data set. In literature, I have read that the break up ratio for training/testing/validation of data should be ...
0
votes
2answers
197 views

How to formulate data for neural network with “class” inputs and a numerical output

I'm just starting to play with neural networks (via PyBrain). I've got some questions about problem formulation. I've taken a bunch of rugby data (very topical), ...
0
votes
1answer
588 views

How to train a Gaussian mixture hidden markov model

I want to build a hmm with continuous observations modeled as Gaussian mixtures. The way I understand the training process is that it should be made in 2 steps. 1) Train the GMM parameters first ...
1
vote
1answer
109 views

Inverse progression for training & validation data during training with H2O

I'm training on a dataset with 3600 columns. 100948 training rows & 25238 validation rows. These are the R commands I'm using: ...
1
vote
0answers
56 views

How to train radial basis function for function approximation?

There is an Autoregressive model of order 1 (AR(1)) that is excited by a non-linear signal as the input: $$x_t = \rho x_{t-1} + u_t \tag{1}$$ The time series $u_t$ is generated from a Mackey-Glass ...
1
vote
1answer
129 views

Is this training dataset enough for training and testing classification model?

My training dataset contains just 2 classes with 40 features. In case 1, class 1 has 35 samples and class 2 has 700 samples. In case 2, class 1 has 65 samples and class 2 has the same value as ...
0
votes
0answers
53 views

Training set selection

I have the following question for a project I'm working on. I am trying to find the best strategy to select the best training set in a dataset. I have a dataset with a few billions rows. I am trying ...
0
votes
1answer
135 views

Creating a test set with imbalanced data

I am working on a binary random forest using R. mu data set consists of 300 cases classes 1 and 2100 cases class 0. I am planning to evaluate my model using the model prediction and the AUC and for ...
1
vote
0answers
79 views

too many ties in knn? how to solve this problem

I use the knn model to train my data and then eliminate accuracy via cross-validation, but when I use the following code, I get the error: ...
0
votes
0answers
36 views

How to detect contradictory examples in training data

I want to implement an online learning classification algorithm. Therefore I want to detect contradictory examples in my training data once the user wants to add a new example to the training data. ...
0
votes
0answers
150 views

TfidfVectorizer: should it be used on train only or train+test

When training a model it is possible to train the Tfidf on the corpus of only the training set or also on the test set. It seems not to make sense to include the test corpus when training the model, ...
0
votes
0answers
278 views

Random Forest online/incremental learning in R

Is there a Random Forest implementation available in R, that supports online learning? My alternative approach was to use the popular randomForest package and combine Random Forests (the existing one ...
1
vote
0answers
154 views

Test MAPE < Train MAPE using auto.arima()

I am trying to build a forecasting model for the passenger vehicles registrations in a given country, and I wanted to use $auto.arima$ function from the $forecast$ package to estimate a simple ARIMA ...
1
vote
0answers
24 views

Should the number of normal samples always be more than that of anomalous samples in training set for anomaly detection?

I am trying to train an anomaly detection algorithm (one-class svm) on a data set with a few hundred positive samples and several thousands negative examples. Is it mandatory that I train the model ...
0
votes
1answer
264 views

Extracting Standard Errors Caret Model

I have tuned a glm net model with caret using the train function. I am trying to extract the coefficients and standard errors of those coefficients for the best tuned model. Following this CV post I ...
0
votes
0answers
222 views

In machine learning, may I train correctly a neural network with input real data and output validation Boolean data?

I have a matrix made of ~ 100 rows and 12 columns. Each entry contains a real value. The first 6 columns refer to a particular concept (firstClass), the following 6 to another one (secondClass), and ...
1
vote
0answers
23 views

Learning over Multinomial data

I have a training data with 68 features... Each of which is a different multinomial distribution. Eg. Feature 1 can take 1 of 4 values while feature 2 can take one of 10 values. Which classifier or ...
5
votes
1answer
384 views

Card Games and Neural Network Inputs

Looking for some pointers on how best to structure a neural net that deals with a card game. In Gin (rummy), you have a 10-card hand and you're trying to make melds and sets out of your cards. A meld ...
0
votes
1answer
49 views

Cross-Validation in binary classification using only 10 positive samples (SVM)

I have a binary classification problem for which only $10$ positive samples are available for training. Negatives are in general in abundance, but I choose to use solely $70$ ($7$ negatives per one ...
0
votes
2answers
254 views

How to divide dataset into training and test set in Recommender Systems?

I am working on two simple recommender systems - Collaborative (item-item a user-user) and Content Based. I would like to evaluate prediction accuracy of these systems. I am used to divide dataset ...