Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [overfitting]

Modeling error (especially sampling error) instead of replicable and informative relationships among variables improves model fit statistics, but reduces parsimony, and worsens explanatory and predictive validity.

0
votes
0answers
17 views

Difference of network between testing and training on the same dataset (No training and testing)

I was training and dense net model on emotion recognition on the sewa dataset. Therefore, at the end I have 2 outputs. One for arousal and the other for valence (These dimensions for emotions). So I ...
0
votes
0answers
8 views

Can overfit happen in spite of validation and what to do with it?

Let's consider a standard situation where we need to find a predictive model. We train all the available model using a training data set. We validate all the trained model using a validation data ...
0
votes
1answer
12 views

How does train-validation-test procedure deals with the sampling error of the accuracy measure?

Let's consider a standard model selection procedure: We have N different untrained models (for example linear regression, neural network, decision tree and so on). We use a data set A to train each ...
0
votes
0answers
28 views

High dimensional regression overfitting

Consider the linear regression model \begin{equation} \boldsymbol{y} = \boldsymbol{X}\boldsymbol{\beta} + \boldsymbol{\epsilon} \end{equation} where we assume $\boldsymbol{X}$ is $n$-by-$p$, with $p &...
0
votes
0answers
25 views

Why do we take `(Bias) ^2` in total error in a model? [duplicate]

I was recently studying some book and few blogs and come to note that : Total error = Bias^2 +Variance + irreducible error Also, I know that these are the errors ...
0
votes
1answer
37 views

When does my unsupervised autoencoder start to overfit?

I am working on anomaly detection using an autoencoder neural network with $1$ hidden layer. This is an unsupervised setting, as I do not have previous examples of anomalies. The input data has ...
0
votes
0answers
11 views

Using Data Augmentation Drops the accuracy by 40% datagen.flow [on hold]

I working on a project with a group, we are using different pre-trained models with imagenet weights and we added 3 dense layers where we freeze the model layers , we are using MURA dataset but a ...
1
vote
1answer
30 views

Fitting model on whole dataset, more or less epochs ? (w.r.t validation accuracy)

When tuning my neural networks hyperparameters I use 20% of the data set as validation data. With the holdout set I observe the validation accuracy and validation loss. In my case the model starts ...
1
vote
1answer
26 views

Finding the appropriate polynomial fit in Python

Is there a function or library in Python to automatically compute the best polynomial fit for a set of data points? I am not really interested in the ML use case of generalizing to a set of new data, ...
1
vote
1answer
30 views

With test accuracy being equal, is it better to have lower training accuracy?

Suppose we train two models on a training set, and then test them both on the training set itself, and on a test set. We have some accuracy metric we're using to evaluate them. Both models score ...
1
vote
1answer
20 views

Distinguishing between overfitting and wrong model selection

I built a dozen of different models using caret package for classifying customer purchase habits into 5 categories (catA, catB, catC, catD, none) based on 4 numeric ...
3
votes
1answer
29 views

why too many epochs will cause overfitting?

I am reading the 《deep learning with python》. In chapter 4, about Fighting overfitting, I have two questions. why increasing epochs may cause overfitting? I know ...
0
votes
1answer
26 views

Emotion detection: neural network overfitting on audio files

I am working on an analysis of audio data to understand emotions using the RAVDESS dataset. The input is the Mel-frequency cepstral coefficients (MFCCs) of each audio file, extracted using a Python ...
0
votes
1answer
16 views

Should the validation set have the same ratio in the categories as the whole data?

I'm currently working on a classification problem. The variable Y in 70% of cases is 0 and in 30% of cases is 1. Does my validation set have to have this same proportion? I ask because after using ...
0
votes
0answers
32 views

Cox proportional hazards model, problem with correlated predictors or overfitting?

I have a question concerning a Cox proportional hazard model (in R) on which I would love to get your opinion and feed back! I think that there is a problem, however I would like to be sure and ...
0
votes
1answer
21 views

Why do we use multiple epochs and why does it not lead to over fitting

I have found many answers to the question of what an epoch is, but none to this: Why do we use multiple epochs when training a neural net? How does this not lead to overfitting? From my ...
0
votes
0answers
8 views

Dataset that overfits with increasing test error curve

I want to test some optimization algorithm for DNN to see if it prevents overfitting. I'm looking for a images dataset that overfits with SGD s.t. test 0-1 error curve increases with iterations. If it ...
0
votes
0answers
37 views

How does a covariance matrix over fit if we have too few data points?

I am reading this : - Honey I shrunk the Sample Covariance Matrix The author on page 2 says that : The crux of the method is that those estimated coefficients in the sample covariance matrix that ...
1
vote
0answers
24 views

SVM model overfitting?

I have a multi-class (10 classes) classification problem. I am using one-vs-rest SVM modeling with sklearn.svm.SVC. I want to know whether my model is over-fitting. For train set accuracy is 100% ...
0
votes
1answer
47 views

Is the best model always one with best test score, even though it looks overfit?

I'm making a binary classification model using gradient boosting (lightgbm). I usually use learning curves to check if my model is overfitting. The metric I'm using is sklearn's average precision-...
0
votes
1answer
62 views

Is this a sign of (bad) local minima encountered by this RNN?

On a binary classification task, I can get perfect performance on the training set, but no matter how strongly I regularize the recurrent neural network (dropout 0.99, L2 weight penalties of 0.01) the ...
2
votes
1answer
44 views

Is RJMCMC robust to overfitting?

I have noticed that RJMCMC is often described as robust to overfitting. I am struggeling a bit with the intuition for this. Why doesn't the Reversible jump Markov Chain Monte Carlo (RJMCMC) always ...
1
vote
1answer
74 views

How to test for overfitting in a TAR model in R?

I want to fit a threshold autoregressive model, and I'm using the tar package in R. For ARIMA models, I could check if a model was overfit by looking at the values of standard errors as compared to ...
5
votes
2answers
3k views

Dealing with singular fit in mixed models

Let's say we have a model ...
1
vote
0answers
10 views

Random forest regressor has a negative score [duplicate]

I am using a RandomForestRegressor. When I check the score for the model with the training data, it's Rregressor.score(X_train,y_train) 0.8357837327169805 but when I check the score using the test ...
-1
votes
0answers
18 views

What's the relationship between overfitting and network depth?

Intuitively, given a neural network with a fixed number of parameters, as the network grows deeper, it can learn a richer structure and has a bigger hypothesis space. But deeper networks now often ...
1
vote
1answer
24 views

Relation between number of features, higher order polynomial features and overfitting

Recently I came across an information stating that, if we have too many features, the model is most likely to overfit. I not sure why exactly this is happening. I mean, if I don’t use any higher order ...
1
vote
0answers
34 views

Bias and over-fitting in Maximum Likelihood estimation

In his book, "Pattern recognition and Machine learning", Bishop talks about the influence of the bias and overfitting in the MLE framework. Here is a quote from p.28, just before he has shown that the ...
1
vote
0answers
14 views

Why, when we use the logarithmic function, weigh less to overfit? [closed]

Why, when we use the logarithmic function, weigh less to overfit, or why I create less local logarithmic errors, they wanted an answer with proof.
0
votes
0answers
10 views

Why after multiple epoc the performance decrease on the train data?

I have a network and I notice that after multiple epoc the performance on the train data start decreasing. Why?
2
votes
3answers
135 views

Maximum likelihood estimators and overfitting

In his book, Bishop claims that overfitting is caused by an unfortunate property of the Maximum likelihood estimator. I dont really understand how the MLE relates to overfitting. To me, roughly, ...
2
votes
1answer
141 views

Why L1 regularization can “zero out the weights” and therefore leads to sparse models? [duplicate]

I'm aware there is a very relevant explanation on L1 regularization's effect on feature selection at here: Why L1 norm for sparse models [Ref. 1]. To better understand it I'm reading Google's ...
0
votes
1answer
70 views

Does dropout regularization prevent overfitting due to too many iterations?

For image classification problem, let's say, and given a neural network to train on, if you were to run too many iterations for a single image of a cat would not generalize well into other images of ...
0
votes
0answers
95 views

How to interpret/choose alpha in ridge regression

I have questions on how to apply ridge regression on my data set, which has about 75 samples with 8 features (x's) and usually 3 targets (y's). I tried the following feature engineering methods. ...
1
vote
1answer
27 views

Model overfitting when using two separate datasets for train and test

I have two datasets generated from two FPGA cricuits having almost same design. Both have 17 features as binary values where the last column is the class label <...
4
votes
1answer
64 views

Does Leave One Out cross validation increase the chance of overfitting?

By increasing the size of the training set the model memorize more data. Thus, will using leave one out increase the chance of overfitting?
1
vote
2answers
118 views

How to prevent overfitting in Gaussian Process

I'm training Gaussian Process models on a relatively small data set, which have 8 input features and 75 input data. I tried different kernels and find the following kernel (2 RBF + a white noise)...
0
votes
1answer
98 views

How to prevent overfitting with regression using ranger (randomforest)

I use caret to train the model (on Boston dataset from the mlbench package). Here is the code ...
3
votes
2answers
352 views

Can overfitting be a good thing in some cases?

I know the goal of machine learning is to create generalizable models and therefore overfitting is undesirable. However, I wonder if it could be desirable in some cases. For example, let's say I ...
0
votes
1answer
106 views

how to choose model when training accuracy is lower than validation accuracy while training neural network?

Below is a specific case but a general situation i find myself involved with in my job. This question is intended at getting ideas on how to pick the best model: Dataset: rows: 10,166, features: ...
1
vote
0answers
36 views

Bias-Variance terminology for loss functions in ML vs cross-validation — different things?

I am a bit confused about the use of variance and bias across the machine learning and statistical learning literature. In particular, the bias-variance trade-off arises from the fact that one can ...
0
votes
0answers
25 views

Obtaiting the Training Error from a RandomForestRegressionModel

I was planning to plot the evolution of my RandomForestRegressionModel when tunning the hyperparameters. One thing I would like to evaluate is the overfitting. For ...
0
votes
0answers
21 views

Comparing Model's Performance on the Train and Test Sets

I've developed a model that predicts a future value of a parameter for the next 72 hours (only 11 hours presented on the chart). I've obtained the hyperparameters for my model with use of ...
4
votes
1answer
471 views

Can it be over fitting when validation loss and validation accuracy is both increasing?

Training a simple neural network over a very sparse matrix (Has 2400 features and 18000 train rows) for a binary classification problem. At the end of 1st epoch validation loss started to increase, ...
1
vote
2answers
101 views

Techniques to avoid overfitting

I have heard of several techniques to avoid overfitting: Validation curve: which let us choose the set of parameter with the minimum step between validation score and training score. But it seems ...
0
votes
1answer
107 views

Why does cross entropy loss for validation dataset deteriorate far more than validation accuracy when a CNN is overfitting?

I have noticed that the cross-entropy loss for validation dataset deteriorates after a certain number of epochs when training CNN's or MLP's. This is, of course, the sign that the network is ...
3
votes
5answers
770 views

Is using both training and test sets for hyperparameter tuning overfitting?

You have a training and a test set. You combine them and do something like GridSearch to decide the hyperparameters of the model. Then, you fit a model on the training set using these hyperparameters, ...
8
votes
3answers
892 views

Overfitting on purpose

Would it make sense to overfit a model on purpose? Say I have a use case where I know the data will not vary much respect to the training data. I'm thinking here about traffic prediction, where the ...
0
votes
0answers
20 views

A good performance with n-fold cross validation and overfitting

Is it possible to train a model (for regression or classification) that has overfitted but that has a good performance in a n-fold cross validation?
0
votes
0answers
46 views

Overfitting, cross validation and validation curve

Some time ago i used to choose my hyperparameters only relying on the Test score returned by cross validation. But after reading How does cross-validation overcome the overfitting problem? i doubted ...