Questions tagged [overfitting]

Modeling error (especially sampling error) instead of replicable and informative relationships among variables improves model fit statistics, but reduces parsimony, and worsens explanatory and predictive validity.

Filter by
Sorted by
Tagged with
1
vote
0answers
10 views

How does the number of epochs affecting GANs training?

In CNN training, increasing the number epochs would lead to overfitting. However, to train a GAN, would a too large number of training epochs matter? Indeed, I also do not understand what does it mean ...
0
votes
2answers
36 views

Why don't we use regularization on decision tree split?

I heard people ask which one is better: Linear regression with regularization or Random Forest. My question is why can't you use regularization with Random Forest? My understanding is that different ...
0
votes
1answer
34 views

Preprocessing: Why do we remove constant/invariant predictors?

For a recent research project we used machine-learning. In the preprocessing phase we removed 2 predictors because they contained mostly uninformative 0 values (x1 = 100%, x2 = 99%). Is it the right ...
0
votes
0answers
15 views

Does an aberration imply overfitting?

I was reading-up on over-fitting for my project with a small dataset and it's clear that fluctuations in validation loss and accuracy imply overfitting, but does that include constant oscillation or ...
1
vote
2answers
34 views

Is it possible to test overfitting with randomized data?

I have built machine learning models for a classification problem with four classes. They run at around 70% nested cross validation accuracy. I am looking to do further testing to check of ...
0
votes
0answers
25 views

Way to stop model from overfitting in automated training pipeline?

I'm currently training a gradient boosting model for which I want to create an automated training pipeline containing hyperparameter optimization with hyperopt and also cross-validation. While trying ...
1
vote
1answer
32 views

Number of Covariates in Cox PH Model and Overfitting

I have a small time to event dataset (N=20) where patients are given one of two drugs (drug) at varying doses (...
1
vote
1answer
18 views

Rule of thumb Overfit in a MLP or is it possible with N = 135

I want to say ahead that I highly appreciate any Literature recommendation (Book, blogg, ...) besides ESL and ISL. My Question: Is it possible to train a 3 layer multilayer perceptron (mlp) in a ...
0
votes
2answers
19 views

Standard Deviation (SD) as additional metric for model evaluation?

I wanted to ask whether you think that it can be useful to compare the standard deviation of the predictions (not the standard error!) in addition to other metrics like RMSE to get an idea on the ...
4
votes
2answers
833 views

Can K-fold cross validation cause overfitting?

I am learning $k$-fold cross validation. Since each fold will be used to train the model (in $k$ iterations), won't that cause overfitting?
4
votes
1answer
47 views

Do out-of-sample fitting methods solve the problem of over-fitting?

Suppose we have a regression model, and we want to fit this to training data, and then make predictions on test data. There is a well-known danger that out-of-sample predictions will be poor, due to "...
19
votes
1answer
1k views

Mathematical/Algorithmic definition for overfitting

Is there a mathematical or algorithmic definition of overfitting? Often provided definitions are the classic 2-D plot of points with a line going through every single point and the validation loss ...
0
votes
0answers
34 views

What is the best evaluation of training the sequential modeling?

It's concerned with the probabilistic modeling of the sequential dataset. As far as my understanding, well-known RNN methodologies consist of two steps: firstly, train the model representing $p(y_{i}|...
0
votes
0answers
10 views

Estimating Gaussians with (Gaussians+cross entropy) or (output+MSE)?

model used :- neural nets x is an N-dimensional vector. y is a real number. p(x,y) is the joint probability distribution over x and y. I know p(y|x) is a Gaussian probability distribution for ...
0
votes
0answers
41 views

Is it possible to have overfitting due to high bias?

Overfitting is usually associated with high variance, whereas underfitting is associated with high bias. But one of my professors at uni mentioned that overfitting might be caused by high variance and/...
1
vote
0answers
15 views
1
vote
3answers
32 views

What happen if I choose the hyperparameters of a classifier based on lowest generalization error?

In this question, the OP asked about a situation that he/she combined training and test datasets into an agumented dataset and then tuned the hyperparameters for best accuracy and then use the ...
0
votes
0answers
26 views

Is the model over-fitting the data?

On the y-axis you've got RMSE and on the x-axis you've got the number of epochs. Then in blue, the validation error, in red the training error. What do you think is the optimal number of epochs ...
1
vote
0answers
17 views

Why is my keras resnet50 model overfitting? [duplicate]

I have applied Keras ResNet-50 on a small x-ray image dataset. I tried making layers both trainable and non-trainable, but my model validation accuracy doesn't improve above 50%. I don't understand ...
0
votes
0answers
33 views

Is there life after over-fitting?

At this point in a talk by Nando de Freitas, there is an answer to an audience question, about how theory has got left behind in statistics, but theory is still important, and he gives an example ...
7
votes
1answer
45 views

How to prevent overfitting? [duplicate]

I'm aware of the concept of overfitting in Machine Learning. The main advice for dealing with it, usually is regularization. Is there other practical advice to avoid overfitting?
0
votes
0answers
17 views

Neural network outputs extreme probabilities

I currently have two scenarios that I'm unable to understand: A multi-class neural network classifier: this model's final softmax layer outputs very extreme "probabilities" for each class for the ...
0
votes
0answers
21 views

Harrell's Unreliability Index (U) ( binary logistic regression) - interpretation

One of the key aspects of regression models is to avoid overfitting. In the context of binary logistic regression Frank Harrell (Regression Modelling Strategies) suggested to fit a new binary ...
0
votes
0answers
21 views

Implications of deploying a predictive model overfitting training data but consistent in validation folds (classification)

If a model is build on very dirty data, it is common to not be able to prevent an overfitted result even with rigorous regularization attempts. However, it is also common that some lift-producing ...
0
votes
0answers
36 views

Model Not Performing Well On Validation Data - Customer Attrition Modeling

I am modeling customer churn for the online subscription. I looked back 90 days into customers’ data, using number customer watching behavior etc. I get a pretty strong model based on test data. <...
0
votes
0answers
27 views

Machine Learning Model Evaluation

If a model is overfitted that means decent gap between training curve and testing/validation curve but achieves good precision and recall score,does that still indicate that the model is decent?
0
votes
1answer
22 views

Can a model be complicated enough to overfit every validation fold during a k-fold cross validation process?

During k-fold cross-validation, is it possible that a model is so sophisticated (e.g., with many hyperparameters to be grid searched) that it gives a good score on almost every validation fold? It's ...
1
vote
1answer
21 views

Scoring rules for count models on: training data vs. validation data

In order to evaluate and compare count models (e.g. Poisson regression), we can calculate scoring rules (e.g. Brier Score, Dawid-Sebastiani score, etc.) which are explained here: Error metrics for ...
0
votes
0answers
13 views

Dynamic factor analysis

I have been running a DFA on n=40 time series containing percent cover estimates for a single species of algae. The time series represent 40 locations that span the geographical range of the species, ...
0
votes
0answers
34 views

How to do variable selection for Gradient boosting models like Xgboost and LightGBM

I am building a classification model with about ~110 variables and that gave me an AUC of about 71.96 on validation. I added about 10 more features and my AUC value decreased to 71.56 (which led to ...
1
vote
2answers
87 views

What is the difference between overfitting and “not learning”

I am trying to build a Random Forests (RF) model using around 2000 observations and a number of features (can be 50 or can 1000, I still do not know which features are to be used). One way to ...
1
vote
1answer
44 views

Can we use a mixture of normal distributions while optimising likelihood?

Let's assume that we generate some values by a mixture of two Gaussians. Now we want to find the parameters of the two Gaussians by likelihood maximisation. One good expect that the optimisation will ...
0
votes
0answers
39 views

Controlling over-fitting in local cross-validation LightGBM

I am training a lightgbm model on a binary problem (~20% of events) with below parameters: ...
7
votes
2answers
1k views

How does one most easily overfit?

This is a weird question, I know. I'm just a noob and trying to learn about different classifier options and how they work. So I'm asking the question: Given a dataset of n1-dimensions and n2-...
1
vote
0answers
47 views

Cross-Validation on a multiple linear regression model, negative values?

I'm trying to demonstrate that, using a linear model with too many predictors, that the correlation can be artificially inflated, and that k-fold cross validation can expose overfitting. To do this, ...
5
votes
1answer
149 views

Can small SGD batch size lead to faster overfitting?

I have feedforward neural net, trained on cca 34k samples and tested on 8k samples. There is 139 features in dataset. The ANN does classification between two labels, 0 and 1, so I am using sigmoid ...
0
votes
1answer
23 views

Mathematical Motivation of Splitting Into Training and Testing Sets

In Learning from Data course taught by Caltech Professor Yaser Abu-Mostafa the following notation is used to describe the in sample and out of sample errors. $E_{in}[h]=\dfrac{1}{N}\sum_{n=1}^Ne(h(...
2
votes
1answer
39 views

Cross validation and over-fitting

I've read many posts on this site that claim something along the lines of "I used cross-validation to prevent over-fitting". Which leads me to my question, does cross-validation actually prevent over-...
0
votes
0answers
34 views

cross-validation: feature selection and hyperparameter tuning. Is nesting necessary?

I am a little bit confused by the use of feature selection inside a K-fold CV together with hyperparameter tuning. So I have my dataset. I split in training & test as usual, and work on training ...
0
votes
1answer
44 views

Signs of Overfitting in Precision/Recall Curve

plz look at the following figures. As you cann see the precision is always 100% no matter which threshold (x-axis in logarithmic scale) you set! Also the second figure shows that we have a perfect ...
0
votes
0answers
22 views

Training neural network with a single instance at a time

For a semantic segmentation problem attempted using neural networks, does it make sense to try achieving overfitting with single training example and then (depending on generalization error) add more ...
1
vote
0answers
14 views

como eu posso ajustar parameteros um sistema com duas equações com a função nls()? [closed]

I Ned to fit this system of equations: I'm using nls.lm(), but want to use the nls() function # rm(list= ls()) df=read.table( text =" 0 0.010000000000000 0 0....
0
votes
0answers
180 views

Very High Training Accuracy and very low Testing Accuracy CNN

I'm using 3 layer CNN with 8, 16, and 32 filters, each of size 5 X 5. I'm getting an training accuracy of 99.97%. Testing accuracy of 41.11%. Total classes: 605 Train Set: Each class has 7 samples ...
1
vote
1answer
34 views

Proof that estimator in overfitted model is still unbiased

Assume that the true population model is given by $y=x'\beta+\epsilon$, where $x$ and $\beta$ are k-dimensional vectors, and suppose that when performing a linear regression, we accidentally overfit ...
0
votes
1answer
125 views

99% on the first epoch: overfitting [closed]

I am working with time-series data and I am trying to classify the Fault happening in the system. The problem is no matter what I try so far, I get 99.79 validation accuracy on the very first epoch. ...
0
votes
0answers
16 views

May negative dataset cause CNN model over/under-fitting?

To put you into context, let me explain a bit what I'm trying to achieve. I'm using YOLOv3 (doesn't really matter now) convolutional neural network to detect traffic signs in full images. I'm training ...
3
votes
0answers
162 views

Overfitting in Random Forest Classifier?

I would like some help from you in a classification model that I am developing. In summary, the problem is: – Classification problem with binary outcome (0/1) – The classifier is a Random Forest ...
1
vote
1answer
33 views

Neural Networks - Difference between 1 and 2 layers?

I'm currently working on a regression problem, using neural networks to constrain parameters for a complex physical scenario. I am searching the hyperparameter space for the best model and have thus ...
0
votes
1answer
30 views

validation vs test vs training accuracy, which one to compare for claiming model overfitting?

I have read on the several answers here and on the internet that cross-validation helps to indicate that if the model will generalize well or not and about overfitting. But I am confused that which ...
0
votes
1answer
34 views

Bad performance on training set and increasing validation loss: Overfitting or underfitting?

From my understanding of over- and underfitting, the two behaviors are not completely mutually exclusive. Overfitting can pretty much always be achieved if the model has enough capacitance (is able to ...