Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

The tag has no usage guidance.

2
votes
2answers
29 views

Different metrics for training and evaluation/comparison?

When does one train and cross-validate some models on a particularly objective (say, likelihood), and then evaluate or compare the models based on a different metric (say, AUC)? Any advantages and ...
1
vote
0answers
16 views

Training an ANN further once it reaches 100 % accuracy on training set

I have a very simply question: Does it make sense to further train an ANN once it reaches an accuracy of 100 % on the training data? I'm facing a binary classification problem and read this article ...
0
votes
0answers
23 views

Weights after training neural network all become negative

I'm making a neural network. The training output for all pairs is either 0 or 1. I've noticed that if I add only a single training pair with target output '1' and 9 other pairs with '0', my weights ...
0
votes
0answers
27 views

Training Accuracy/Loss and Validation Accuracy/Loss

I have a few questions about the training process of a learning algorithm. My understanding is that each time a mini-batch is fed into the algorithm, I should calculate a loss and accuracy so that at ...
1
vote
0answers
30 views

My Neural Network gives a wrong prediction when I specify more nodes

I recently got interested in neural nets. After reading a bunch I tried to make one in python using numpy. I fed in some sample input and output data. When I train the neural net and and then ask it ...
0
votes
0answers
23 views

Training RBM with (normal) contrastive divergence vs persistent contrastive divergence

I implemented RBM by using PyTorch here and trained it with CalTech 101 28x28 Silhouettes dataset containing binary images. I implemented both (naive) contrastive divergence and persistent ...
1
vote
0answers
12 views

Considering Test Error when Computing Classification Confidence

Given is binary classifier, which, together with the class-label also returns a corresponding probability/confidence, e.g., logistic regression. Even with only very few training examples, such a ...
0
votes
0answers
34 views

What does a bouncing Training Error mean in an LSTM?

I have an Bidirectional LSTM of size 1024, running over a sequences of variable length. My dataset is 100,000 items large, with a 256 batch size. I'm taking the last relevant step, concatenating ...
0
votes
2answers
25 views

Learning Curves: Should the training set size be increased incrementally or Random Selection?

I am trying to write a bespoke learning curve function. I was wondering how is it usually implemented. When the size of the training set is increased - Is it normally increased by adding new samples ...
0
votes
0answers
35 views

Performance Evaluation Metrics used in Training, Validation and Testing

Which specific performance evaluation metrics are used in training, validation and testing and why? I am thinking error metrics (RMSE, MAE, MSE) are used in validation, and testing should use a wide ...
0
votes
1answer
76 views

Can 'independent' test set be included in data exploration without biasing generalisation error estimate?

If I have some data, say 1000 labelled examples (A/B, 500 of each), from which I define an independent test set of 100 labelled examples (A/B, 50 of each), then can I legitimately have a 'cursory ...
7
votes
3answers
1k views

If Mean Squared Error = Variance + Bias^2. Then How can the Mean Squared Error be lower than the Variance

I was reading the Introduction to Statistical Learning. Here it is shown that:- In a later example, the train and test MSE are plotted. I wanted to know if both the bias^2 and variance are positive ...
0
votes
0answers
221 views

Time series data pre-processing for anomaly detection

I am using a good volume of time series data that spans over two months [November and December 2015] containing time-stamp observations. A total of about 6 million samples. I use the portion of clean ...
0
votes
0answers
196 views

Expected prediction error

1.I am confused by question (c) from Caltech's Learning From Data. I got (M-1)/2M for question (a) and 2^N for question (b). Any formula concerning E(out) in the textbook is inequality. I found the ...
0
votes
1answer
24 views

Implications of training a deep network on batches using different sampling methods

During training of a deep network on data with high class-imbalance, I notice the network's predictions tend to mimic the class distribution in the training batches. What is the expected behavior of ...
4
votes
2answers
963 views

Error increase on L2 regularization in an NN

When introducing L2 regularization on my neural network, there is a point during training where the error starts to increase after having reached a value very close to 0. This is due to the fact that ...
4
votes
0answers
744 views

Which loss function to use when training LSTM for time series?

I'm experimenting with LSTM for time series prediction. The example I'm starting with uses mean squared error for training the network. I know that other time series forecasting tools use more "...
2
votes
1answer
2k views

Training loss increases with time

I am training a model (Recurrent Neural Network) to classify 4 types of sequences. As I run my training I see the training loss going down until the point where I correctly classify over 90% of the ...
1
vote
1answer
124 views

Identifying overfitting by comparison between training error and test error

I got a problem in which the classification task is very hard to accomplish, because the features are not very informative. Anyway I'm trying to get some results (even poor) using a neural network (...
0
votes
0answers
39 views

Which model is the best in this case and why?

I am a newbie in machine learning. I have a dataset and I have split them into 2 parts include 80% of data for training whereas 20% for testing. After training and validation I have some results as ...
0
votes
0answers
22 views

Training errors bigger than validation errors

I'm using H2O distributed random forest for a binary classification problem of a dataset with mix categorical and real features. However, my validation set errors are smaller than the training set ...
0
votes
0answers
87 views

Generalization MSE error of normal jointly distribution data

We have features $\boldsymbol{x}$ and label $y$ related by a simple linear regression model: $$y = {\bar{w}}^T\boldsymbol{x} + \varepsilon$$ with some non-random weight vector (true but unknown) $\...
0
votes
1answer
43 views

How does shuffling influence ANN training?

I am wondering how shuffling will influence ANN training. I assume, we a batch training using the entire data for training at each epoch. Shuffling does not change the value of the error $$ E={\frac ...
4
votes
1answer
278 views

Prove that the expected MSE is smaller in training than in test

This is Exercise 2.9 (p. 40) of the classic book "The Elements of Statistical Learning", second edition, by Hastie, Tibshirani and Freedman. In the book, it's mentioned that this exercise was brought ...