training (or estimation) of statistical models or algorithms.

learn more… | top users | synonyms

0
votes
0answers
33 views

How to prove the reliability of a predictive model to executives?

I trained data from 500 devices to predict their performance. Then I applied my trained model to a test data set for another 500 devices and show pretty good prediction results. Now my executives want ...
0
votes
1answer
26 views

How does cross-validation in train (caret) precisely work?

I have read quite a number of posts on the caret package and I am specifically interested in the train function. However, I am not completely sure if I have understood correctly how the train function ...
1
vote
1answer
17 views

How to choose the test set size when the training set size is given?

I have data on 64 subjects collected in a medical setting. With the help of ROC curve analysis and bootstrapping, I have identificed two predictors for illness(present or not present) in the group. ...
0
votes
0answers
4 views

LOOCV in R returns error [migrated]

In the past days I began getting familiar with R (I come from MATLAB and Python). I wanted to try out the caret package (pretty awesome) and I keep getting the following error message when I try to ...
2
votes
2answers
63 views

Good examples/books/resources to learn about applied machine learning (not just ML itself)

I've taken an ML course previously, but now that I am working with ML related projects at my job, I am struggling quite a bit to actually apply it. I'm sure the stuff I'm doing has been researched/...
0
votes
2answers
20 views

How to quantitatively determine when to stop training ANN

I've implemented an artificial recurrent neural network and want to start training it on a variety of tasks. I've extensive searching online and haven't found a satisfactory answer of how the ...
1
vote
2answers
33 views

SOM based on a not euclidean distance

Suppose one has trained a SOM on a certain number of data. Without explaining all the procedure, one can say that the SOM algorithm produces a certain number of prototypes and the new elements coming ...
0
votes
1answer
86 views

Cross-validation of multiple subjects with multiple instances

I have a training set of 50 subjects with about 550-600 measurements each. One measurement consists of 24 features and one class label (1 or 0). So my data looks like this (simplified): ...
0
votes
1answer
101 views

SVM model training set vs test set

I am trying to train an SVM model using Forest Fire data. I split up my data into a test and training set. I am fairly new to this type of analysis but I'm not sure what role the test data plays or ...
0
votes
0answers
12 views

Removing malicious training examples from trained linear regression model

Let's say that I trained a linear regression model with $N$ training examples $X$ and targets $y$. After training, I realize that some small subset of training examples $X_e$ were measured erroneously,...
0
votes
1answer
36 views

Why L1 Regularization does not work with Calculus Training methods?

I quite understand What is L1 and L2 regularization, but the authors of articles keep saying that: To summarize, L1 regularization sometimes has a nice side effect of pruning out unneeded features ...
0
votes
0answers
16 views

R Caret train / rfe optimize for positive predictive value instead of Accuracy or Kappa

In train or rfe I can only set Accuracy or Kappa. Is there a way to edit the functions to define a scoring function? I am using Kappa at the moment but I need to optimize for positive predictive Value ...
0
votes
1answer
22 views

Large number of positive labels in classifier when actual population has few

I have been tasked to help with a sort of classifier. In the make up of the problem the set we want to identify as "Positive" is know to be very very small. However the training set I have been given ...
0
votes
1answer
40 views

Rolling forecasts: training versus forecast accuracy evaluation

Questions: Are rolling forecast examples (like the ones below) only useful for evaluating a model's accuracy, or can a rolling forecast be used to train a model? Are models trained using a rolling ...
0
votes
0answers
16 views

Combining results from tests after re-shuffling data

I am fine-tuning a neural network for a binary multilabel classification problem. Basically I am trying to predict 64 binary labels for each input. However, my dataset is somewhat small for the task ...
0
votes
2answers
23 views

What do we learn from a test set?

Suppose I split my data into two parts -- a training set (having 80% of my data) and a testing (20%) set. I train a model on my training set, and test it on the test set. What do we learn from ...
0
votes
0answers
10 views

Training vs. Testing Score [duplicate]

In a recent analysis I'm doing, I found that after splitting my data into 70% training data and 30% testing data, my training score is slightly lower than my testing score. To my knowledge, shouldn'...
0
votes
0answers
12 views

Compare the quality of two train sets

I wonder what is the right way to evaluate the quality of the dataset. I have two train sets and one fixed test set. I want to show that one train set is better than the other. The simplest way is ...
0
votes
1answer
41 views

Choosing number of samples to train a model

(On behalf of a colleague) I have performed some modelling based on a naïve Bayes classifiers model (weighted genomic risk score) and obtained reasonable ROCAUC results (used ROCR, pROC, and SDMtools ...
0
votes
0answers
19 views

Using two different methods for training and inferring

I have a model which is intractable to take derivatives wrt parameters and estimate them based on maximum likelihood. Even with an deterministic approximation like variational inference this is ...
2
votes
1answer
207 views

How to train convolutional neural networks with multi-channel images?

I have $m$ labeled images, each with 224x224 pixels and 5 different image channels. What is the best way to train a CNN architecture using this data when $m$ is small (less than 2000)? Is it possible ...
0
votes
1answer
29 views

Backpropagate multiple hidden layers

I have created a feed forward neural network using the sigmoid activation function and backpropagation. I was wondering if I would be able to use backpropagation the same way for two hidden layers as ...
2
votes
0answers
38 views

Targets of 0.1/0.9 instead of 0/1 in neural networks and other classification algorithms

Rumelhart, Hinton and Williams (PDF) wrote in 1986 in the context of training a neural network (page 12): One other feature of this activation function should be noted. The system can not ...
0
votes
0answers
107 views

Need to resize features in XGBoost

I am scaling all the numeric features in my train data in values between 0 and 1 but is it truly necessary? Does the algorithm performance is improved doing so or it can deal with different ranges of ...
0
votes
0answers
29 views

Bootstrap sampling to evaluate a model

In order to properly estimate model prediction performance I use bootstrapping to estimate the confidence interval of the performance measure. I perform a repeated random sampling to generate $B$ ...
1
vote
0answers
217 views

What is the minimum sample size required to train a Deep Learning model - CNN?

It is true that the sample size depends on the nature of the problem and the architecture implemented. But, on average, what is the typical sample size utilized for training a deep learning framework? ...
0
votes
0answers
34 views

The meaning of “training accuracy”?

If I split my data set into testing, training (further separated into subtraining and validation data set in cross-validation). In the context of machine learning and esp. in those ROC comparing the ...
0
votes
2answers
97 views

How to create a variable that is present in test data set but not in train?

Im try to do a classification but i have a variable production budget which is present in test dataset and not in train. so how do i proceed. could i impute that variable somehow. i dont want to drop ...
2
votes
1answer
395 views

How do I use deep learning (Convolution Neural Network) with small training data-set?

I have an image data set of around 180 images in 60 classes (3 images per class). I am able to build a classifier using feature matching. However, I want to try Convolution Neural Networks and see if ...
0
votes
0answers
23 views

How do I consider the whole training set in the process of Hidden Markov Model training?

I'm trying to train a Hidden Markov Model following theory from the book "Pattern Classification" by Duda, Hart, and Sotrk. For the HMM learning they discuss Forward-Backward algorithm there, ...
1
vote
1answer
40 views

Can I use cross validation on a subset of the training set to select hyperparameters?

I am using R, and I had a dataset with 400000 rows and 800 columns, training a random forest model with only 100 trees on this dataset will take me about 1 and half hour on my laptop. So I went on and ...
3
votes
3answers
151 views

Why does the training error usually underestimate the test error?

I understand that most algorithms are optimized to minimize the training error but why is the test error usually larger then the training error? Is there a statistical reason why?
0
votes
1answer
43 views

Real world model training in R: how to get instant feedback?

I want to train a model. I can just randomly choose method (e.g. random forest), put whole dataset, wait a few hours, check accuracy, plot every possible curve (like accuracy vs train size) and see ...
4
votes
4answers
378 views

Does increase in training set size help in increasing the accuracy perpetually or is there a saturation point?

I am using a boosted trees classifier which is giving better accuracy then all other linear classifier I tried. I have almost an unlimited training data at my disposal , I wanted to know if there is a ...
4
votes
1answer
148 views

ML / train-test-validate: What is allowed when?

As someone getting started in machine learning, I am trying to get my head around the rules / good practices to follow when building, testing and validating supervised ML models in order not to ...
0
votes
1answer
55 views

Logistic Regression with empty cells

I have a data set from which I need to train a model and use it for prediction. Let's say I want to predict what people say about food items produced by a cake shop. Let's assume people have stated ...
0
votes
1answer
48 views

Feeding categorical data to classifier

Suppose I have the dataset in the following format: ...
0
votes
2answers
82 views

Trying to understand how I should split data between (train and test) Vs. predict?

I'm working on a model to predict churn. I understand the concept of training and testing, or at least I thought I did. Let's say it's the first of the month and our database has 10,000 subscribers, ...
0
votes
1answer
65 views

GBM Performance on different sampling techniques

I am working on a healthcare data set for breast cancer patients. This data set is class imbalances and the distribution of positive and negative classes is 80%/20%. In order to deal with the class ...
1
vote
3answers
127 views

splitting a training data set based on classifier accuracy

I have training data with ~600K instances. If I split the training data into four segments and build four separate classifiers, I get much higher accuracy for each model than if I train a single model ...
3
votes
1answer
83 views

Minimum length of training data?

I want to obtain a prediction model using support vector regression on a time series data set. In literature, I have read that the break up ratio for training/testing/validation of data should be ...
0
votes
2answers
234 views

How to formulate data for neural network with “class” inputs and a numerical output

I'm just starting to play with neural networks (via PyBrain). I've got some questions about problem formulation. I've taken a bunch of rugby data (very topical), ...
0
votes
1answer
915 views

How to train a Gaussian mixture hidden markov model

I want to build a hmm with continuous observations modeled as Gaussian mixtures. The way I understand the training process is that it should be made in 2 steps. 1) Train the GMM parameters first ...
1
vote
1answer
118 views

Inverse progression for training & validation data during training with H2O

I'm training on a dataset with 3600 columns. 100948 training rows & 25238 validation rows. These are the R commands I'm using: ...
1
vote
0answers
58 views

How to train radial basis function for function approximation?

There is an Autoregressive model of order 1 (AR(1)) that is excited by a non-linear signal as the input: $$x_t = \rho x_{t-1} + u_t \tag{1}$$ The time series $u_t$ is generated from a Mackey-Glass ...
1
vote
1answer
161 views

Is this training dataset enough for training and testing classification model?

My training dataset contains just 2 classes with 40 features. In case 1, class 1 has 35 samples and class 2 has 700 samples. In case 2, class 1 has 65 samples and class 2 has the same value as ...
0
votes
0answers
56 views

Training set selection

I have the following question for a project I'm working on. I am trying to find the best strategy to select the best training set in a dataset. I have a dataset with a few billions rows. I am trying ...
0
votes
1answer
164 views

Creating a test set with imbalanced data

I am working on a binary random forest using R. mu data set consists of 300 cases classes 1 and 2100 cases class 0. I am planning to evaluate my model using the model prediction and the AUC and for ...
1
vote
0answers
89 views

too many ties in knn? how to solve this problem

I use the knn model to train my data and then eliminate accuracy via cross-validation, but when I use the following code, I get the error: ...
0
votes
0answers
167 views

TfidfVectorizer: should it be used on train only or train+test

When training a model it is possible to train the Tfidf on the corpus of only the training set or also on the test set. It seems not to make sense to include the test corpus when training the model, ...