training (or estimation) of statistical models or algorithms.

learn more… | top users | synonyms

1
vote
1answer
22 views

Does “caret” avoid data snooping due to preprocessing in model tuning? [closed]

Although the obvious answer to my question is yes, since caret is a professional, well-known tool, I tend to be skeptical when using implemented functionality from ...
1
vote
0answers
12 views

asynchronous vs synchronous training

As I understood that the difference is asynchronous vs synchronous is the following: "When a neural network is viewed as a collection of connected computation devices, the question arises whether the ...
0
votes
0answers
13 views

Is it the right to split a shuffled training set into training and validation sets?

I want to create a validation set for CIFAR10 dataset which can be found here. The training set has a file named train.txt which contains a list of image's path ...
0
votes
0answers
8 views

implementing contractive autoencoder weight decay

I'm trying to implement the idea behind the contractive auto-encoder and penalty the jacobian of output of an MLP wrt its input to get make it less sensitive to noise. there is a scalar coefficient ...
0
votes
0answers
9 views

What are common strategies for undersampling a class?

The goal is to sample the most useful points for building the best classifier. I am familiar with active learning strategy. Are there other common techniques? Intuitively, I am thinking about: 1. ...
4
votes
4answers
134 views

Should I get 100% classification accuracy on training data?

I've been getting inconsistent results with a binary classification problem I'm trying to solve using a linear classifier and a custom feature extraction pipeline, and decided to do a quick check of ...
1
vote
1answer
18 views

How to design a train and test set from a labeled dataset with class imbalance?

The labeled dataset I am using is almost 80% positive examples, 20% negative examples. However, I do not know the distribution of the data fed into the classifier. In this case, does it make sense ...
4
votes
1answer
33 views

Do examples in the training and test sets have to be independent?

I am working on a machine learning problem where there are several data points collected per user. Some of the points are good and some are bad. I want to get a good assessment of the machine ...
0
votes
0answers
22 views

Matlab : How to apply neural network for binary input features

I have inputs that were real valued. I then normalized them to a range [0,1] and binarized using the mean of each feature component (column) using the solution given in the earlier Question asked ...
0
votes
0answers
20 views

Are RSS and R^2 related to training error only?

While reading An Introduction to Statistical Learning, I stumbled across the following (p. 210): [...] the model containing all of the predictors will always have the smallest $RSS$ and the ...
0
votes
0answers
32 views

Optimisation of a noisy function in R

I have an C++ program that takes about 50 input parameters and then simulates a something. It then returns one number. The simulation is CPU intensive. Further, the output is noisy, meaning that even ...
0
votes
0answers
10 views

Is using pairwise comparison instead of single label value a valid way to augment training data

There are many papers using siamese architecture to do pairwise ranking. For example: suppose my training set is $\{X,Y\}$, where $Y$ is the continuous value. I can find a regressor to minimize $\|h(X)...
5
votes
3answers
301 views

Is overfitted model with higher AUC on test sample better than not overfitted one

i am participating in a challange in which I have created a model that performs 70% AUC on train set and 70% AUC on hold-out test set. The other participant has created a model that performs 96% AUC ...
0
votes
0answers
39 views

How to prove the reliability of a predictive model to executives?

I trained data from 500 devices to predict their performance. Then I applied my trained model to a test data set for another 500 devices and show pretty good prediction results. Now my executives want ...
0
votes
1answer
38 views

How does cross-validation in train (caret) precisely work?

I have read quite a number of posts on the caret package and I am specifically interested in the train function. However, I am not completely sure if I have understood correctly how the train function ...
1
vote
1answer
19 views

How to choose the test set size when the training set size is given?

I have data on 64 subjects collected in a medical setting. With the help of ROC curve analysis and bootstrapping, I have identificed two predictors for illness(present or not present) in the group. ...
2
votes
2answers
78 views

Good examples/books/resources to learn about applied machine learning (not just ML itself)

I've taken an ML course previously, but now that I am working with ML related projects at my job, I am struggling quite a bit to actually apply it. I'm sure the stuff I'm doing has been researched/...
0
votes
2answers
26 views

How to quantitatively determine when to stop training ANN

I've implemented an artificial recurrent neural network and want to start training it on a variety of tasks. I've extensive searching online and haven't found a satisfactory answer of how the ...
1
vote
2answers
42 views

SOM based on a not euclidean distance

Suppose one has trained a SOM on a certain number of data. Without explaining all the procedure, one can say that the SOM algorithm produces a certain number of prototypes and the new elements coming ...
0
votes
1answer
93 views

Cross-validation of multiple subjects with multiple instances

I have a training set of 50 subjects with about 550-600 measurements each. One measurement consists of 24 features and one class label (1 or 0). So my data looks like this (simplified): ...
1
vote
1answer
145 views

SVM model training set vs test set

I am trying to train an SVM model using Forest Fire data. I split up my data into a test and training set. I am fairly new to this type of analysis but I'm not sure what role the test data plays or ...
0
votes
0answers
12 views

Removing malicious training examples from trained linear regression model

Let's say that I trained a linear regression model with $N$ training examples $X$ and targets $y$. After training, I realize that some small subset of training examples $X_e$ were measured erroneously,...
0
votes
1answer
41 views

Why L1 Regularization does not work with Calculus Training methods?

I quite understand What is L1 and L2 regularization, but the authors of articles keep saying that: To summarize, L1 regularization sometimes has a nice side effect of pruning out unneeded features ...
0
votes
0answers
27 views

R Caret train / rfe optimize for positive predictive value instead of Accuracy or Kappa

In train or rfe I can only set Accuracy or Kappa. Is there a way to edit the functions to define a scoring function? I am using Kappa at the moment but I need to optimize for positive predictive Value ...
0
votes
1answer
24 views

Large number of positive labels in classifier when actual population has few

I have been tasked to help with a sort of classifier. In the make up of the problem the set we want to identify as "Positive" is know to be very very small. However the training set I have been given ...
0
votes
1answer
55 views

Rolling forecasts: training versus forecast accuracy evaluation

Questions: Are rolling forecast examples (like the ones below) only useful for evaluating a model's accuracy, or can a rolling forecast be used to train a model? Are models trained using a rolling ...
0
votes
0answers
19 views

Combining results from tests after re-shuffling data

I am fine-tuning a neural network for a binary multilabel classification problem. Basically I am trying to predict 64 binary labels for each input. However, my dataset is somewhat small for the task ...
0
votes
2answers
25 views

What do we learn from a test set?

Suppose I split my data into two parts -- a training set (having 80% of my data) and a testing (20%) set. I train a model on my training set, and test it on the test set. What do we learn from ...
0
votes
0answers
10 views

Training vs. Testing Score [duplicate]

In a recent analysis I'm doing, I found that after splitting my data into 70% training data and 30% testing data, my training score is slightly lower than my testing score. To my knowledge, shouldn'...
0
votes
0answers
13 views

Compare the quality of two train sets

I wonder what is the right way to evaluate the quality of the dataset. I have two train sets and one fixed test set. I want to show that one train set is better than the other. The simplest way is ...
0
votes
1answer
42 views

Choosing number of samples to train a model

(On behalf of a colleague) I have performed some modelling based on a naïve Bayes classifiers model (weighted genomic risk score) and obtained reasonable ROCAUC results (used ROCR, pROC, and SDMtools ...
0
votes
0answers
19 views

Using two different methods for training and inferring

I have a model which is intractable to take derivatives wrt parameters and estimate them based on maximum likelihood. Even with an deterministic approximation like variational inference this is ...
2
votes
1answer
287 views

How to train convolutional neural networks with multi-channel images?

I have $m$ labeled images, each with 224x224 pixels and 5 different image channels. What is the best way to train a CNN architecture using this data when $m$ is small (less than 2000)? Is it possible ...
0
votes
1answer
33 views

Backpropagate multiple hidden layers

I have created a feed forward neural network using the sigmoid activation function and backpropagation. I was wondering if I would be able to use backpropagation the same way for two hidden layers as ...
2
votes
0answers
40 views

Targets of 0.1/0.9 instead of 0/1 in neural networks and other classification algorithms

Rumelhart, Hinton and Williams (PDF) wrote in 1986 in the context of training a neural network (page 12): One other feature of this activation function should be noted. The system can not ...
0
votes
0answers
136 views

Need to resize features in XGBoost

I am scaling all the numeric features in my train data in values between 0 and 1 but is it truly necessary? Does the algorithm performance is improved doing so or it can deal with different ranges of ...
0
votes
0answers
38 views

Bootstrap sampling to evaluate a model

In order to properly estimate model prediction performance I use bootstrapping to estimate the confidence interval of the performance measure. I perform a repeated random sampling to generate $B$ ...
1
vote
0answers
276 views

What is the minimum sample size required to train a Deep Learning model - CNN?

It is true that the sample size depends on the nature of the problem and the architecture implemented. But, on average, what is the typical sample size utilized for training a deep learning framework? ...
0
votes
0answers
37 views

The meaning of “training accuracy”?

If I split my data set into testing, training (further separated into subtraining and validation data set in cross-validation). In the context of machine learning and esp. in those ROC comparing the ...
0
votes
2answers
106 views

How to create a variable that is present in test data set but not in train?

Im try to do a classification but i have a variable production budget which is present in test dataset and not in train. so how do i proceed. could i impute that variable somehow. i dont want to drop ...
2
votes
1answer
473 views

How do I use deep learning (Convolution Neural Network) with small training data-set?

I have an image data set of around 180 images in 60 classes (3 images per class). I am able to build a classifier using feature matching. However, I want to try Convolution Neural Networks and see if ...
0
votes
0answers
24 views

How do I consider the whole training set in the process of Hidden Markov Model training?

I'm trying to train a Hidden Markov Model following theory from the book "Pattern Classification" by Duda, Hart, and Sotrk. For the HMM learning they discuss Forward-Backward algorithm there, ...
1
vote
1answer
43 views

Can I use cross validation on a subset of the training set to select hyperparameters?

I am using R, and I had a dataset with 400000 rows and 800 columns, training a random forest model with only 100 trees on this dataset will take me about 1 and half hour on my laptop. So I went on and ...
3
votes
3answers
166 views

Why does the training error usually underestimate the test error?

I understand that most algorithms are optimized to minimize the training error but why is the test error usually larger then the training error? Is there a statistical reason why?
0
votes
1answer
45 views

Real world model training in R: how to get instant feedback?

I want to train a model. I can just randomly choose method (e.g. random forest), put whole dataset, wait a few hours, check accuracy, plot every possible curve (like accuracy vs train size) and see ...
4
votes
4answers
406 views

Does increase in training set size help in increasing the accuracy perpetually or is there a saturation point?

I am using a boosted trees classifier which is giving better accuracy then all other linear classifier I tried. I have almost an unlimited training data at my disposal , I wanted to know if there is a ...
4
votes
1answer
157 views

ML / train-test-validate: What is allowed when?

As someone getting started in machine learning, I am trying to get my head around the rules / good practices to follow when building, testing and validating supervised ML models in order not to ...
0
votes
1answer
59 views

Logistic Regression with empty cells

I have a data set from which I need to train a model and use it for prediction. Let's say I want to predict what people say about food items produced by a cake shop. Let's assume people have stated ...
0
votes
1answer
50 views

Feeding categorical data to classifier

Suppose I have the dataset in the following format: ...
0
votes
2answers
83 views

Trying to understand how I should split data between (train and test) Vs. predict?

I'm working on a model to predict churn. I understand the concept of training and testing, or at least I thought I did. Let's say it's the first of the month and our database has 10,000 subscribers, ...