Questions tagged [overfitting]

Modeling error (especially sampling error) instead of replicable and informative relationships among variables improves model fit statistics, but reduces parsimony, and worsens explanatory and predictive validity.

Filter by
Sorted by
Tagged with
0
votes
0answers
6 views

catboost does not overfit - how is that possible?

I'm fitting and evaluating a CatBoostRegressor and a XGBRegressor to the same regression problem. I tried matching their ...
0
votes
0answers
28 views

How to determine whether my training is good enough? [closed]

I'm new to deep learning. Could you tell me whether my work is overfitting or underfitting? Because usually the loss can be trained to be less than 1, is this an underfitting? But after I reduce some ...
0
votes
0answers
15 views

Is it normal to have a high AUC in train test

I have Random Forest classification model which is already tunned using k-folds cross validation, when I train the model in the train set, the auc gives me 0.97, in test set is 0.75. Is it normal to ...
0
votes
0answers
19 views

Is it bad if the train score is much higher thanthe test score?

I'm doing a binary classification problem and using accuracy as my metric (I know that's not always advisable but, for now, it makes sense here). I trained a random forest classifier with grid search ...
1
vote
1answer
36 views

Do these plots show that the model is overfitted?

Recently I had a discussion with one of my colleagues regarding the concept of overfitting. I have a model that shows the following training behavior. The first plot shows how the value of loss ...
7
votes
1answer
227 views

How to simplify a singular random structure when reported correlations are not near +1/-1

I have read in several answers to questions on this site that the best way to choose the random structure for a mixed effects model is by using theoretical knowledge. On the other hand I have also ...
3
votes
2answers
40 views

Can overfitting happen if I have number of data points that way more than number of features?

In most cases, more parameters in the model, more data is needed. My question is: can overfitting happen if I have number of data points that way more than number of features (for example, the ...
0
votes
0answers
6 views

What's an acceptable difference between cross validation score and test score?

I'm seriously getting stressed out trying to getting round overfitting on the titanic data set and I wonder why my CV score is never close to the testing score. Whats an acceptable difference between ...
1
vote
0answers
14 views

Decrease hyparam 'C' in SVM classifier

In a hypothetical case where I have a small dataset and I break it into train/test. Then I tune the hyperparams doing k-fold on the train set and choose the 'C' hyperparameter that maximizes my AUC on ...
0
votes
0answers
46 views

How does randomisedCV work?

I am making a binary classifier with unbalanced classes (of ratio 1:10). I tried KNN, RFs, and XGB classifier. I am getting the most good precision-recall tradeoff and F1 score from XGB classifer. So ...
1
vote
0answers
40 views

Explain the source of overfitting in choosing an additional parameter

I was going through this reddit post. It describes the 'timelapse and prediction of Wuhan coronavirus infections in Mainland China'. The guy chose to use $y=a+b*(exp(c*x)-1)$ to describe the ...
1
vote
1answer
26 views

Why Does My Model Perform Better Without Dimensionality Reduction (features > samples)

I created a classifier (a linear SVM in scikit-learn) to classify tweets about the fat acceptance movement (yeah that's a thing) as supporting the movement, opposing the movement, or having an unclear ...
0
votes
0answers
33 views

Understanding Overfitting in neural network

i have built a fully connected neural network in order to predict a physical quantity. The dataset that I use is composed by 18 features and 1 label. The samples are 8504. I ran a lot of time the code ...
0
votes
1answer
27 views

Can a neural network be trained repeatedly on subsets of data to fight overfitting?

I've used random forests for years but I'm less experienced building neural networks. Overfitting is obviously a concern for both decision trees and neural networks. Statistically, random forests are ...
1
vote
0answers
19 views

Difference between likelihood and marginal likelihood and why the second is used inBayesian model selection to avoid overfitting

In Bayesian Model selection we integrate the likelihood and choose the model that maximize this quantity also called marginal likelihood , in order to protect us from overfitting. What is the ...
0
votes
1answer
27 views

Stable model or overfitting?

I have a dataset of 150 patients (2:1 ratio of classes) and 78 features. I performed backwards elimination using logistic regression feature importance to end up with 13 features (SVC classifier). I ...
1
vote
1answer
30 views

Over-fitting SARIMA model

I am currently running an iterative process(for loop) to determine best ARIMA model for monthly sales data according to smallest AIC and MAPE. Box-Jenkins methodology clearly states to choose the ...
0
votes
0answers
12 views

Is there a rule of thumb when comparing model accuracy of training and testing set?

When we compare our model accuracy on our training and testing data, a large difference is good indicator that our model might be overfitting. But how large must this difference be? Is there any rule ...
0
votes
0answers
17 views

Interpreting Validation Curves to Fix Overfitting for Random Forests

i'm curently struggeling to interpret validation curves for my model. See the image. My training data set size is around 1200. I'm using 10-fold cv and the Random Forest Algorithm. Before i took a ...
1
vote
1answer
36 views

Interpretation of learning curves - large gap between train and validation loss

I am trying to train a neural network to predict the quality (good or bad) of produced parts based on the parameters of the production (31 parameters). The network is trained with 121620 samples and ...
0
votes
1answer
67 views

When should we avoid overfitting? [closed]

In the case of model fitting, if we are sure both training and test data have no noise, still must avoid overfitting? (training data may be insufficient)
3
votes
0answers
45 views

Over “specification” of a statistics model?

For a simple example, I am fitting data with a likelihood function generated by a normal distribution. The first model is the normal distribution with two-parameters. The second competing model is the ...
1
vote
0answers
20 views

Does the statistical power for testing regression coefficients depend on the number of model parameters?

Given a generalized linear model (GLM), does the number of regression coefficients affect the power to detect a significant effect size for an individual regression coefficient i.e. as more variables ...
5
votes
2answers
108 views

Does too many variables in a regression model affect inference?

Regression models can be used for inference on the coefficients to describe predictor relationships or for prediction about an outcome. I'm aware of the bias-variance tradeoff and know that including ...
3
votes
1answer
39 views

how to avoid overfitting in XGBoost model

I try to classify data from a dataset of 35K data point and 12 features Firstly i have divided the data into train and test data for cross validation After cross validation i have built a XGBoost ...
2
votes
0answers
18 views

What are the available method that can alleviate the overfitting problem in traditional OLS problem, but still can get a linear fitting?

Recently, I have read the paper https://static1.squarespace.com/static/56def54a45bf21f27e160072/t/5a0d0673419202ef1b2259f2/1510803060244/The_Sampling_Error_in_Estimates_of_Mean-...
2
votes
1answer
34 views

how to avoid overfittig with xgboost and how to increase accuracy

I am doing a binary classification problem, I got to train 85% accuracy, but test accuracy is 72%, I tried different parameters, Cross valid, But overfitting doesn't change, please help me on how to ...
0
votes
0answers
26 views

Doubt in understanding graphs

In this paper, on page $16$, can you please explain the plots? What kind of diagrams are these? What do we have on $x$ and $y$ coordinates in the first plot? What does the lines with very high slopes ...
1
vote
1answer
29 views

Doubt on choice of $L_1$ regularization parameter for lasso

In this paper, on page $14$, I have some doubts in "How to choose $\lambda$" part. The author says Let $\beta^{(−i)}_{lasso}$ be the LASSO solution obtained using $(X^{(−i)},y^{(−i)})$. How can we ...
3
votes
3answers
203 views

May using more features decrease the accuracy of a classifier?

I'm testing a project. I have train and test data. There are 182 features and 1000 samples for train and 3500 for test. If I select certain columns of data and apply naive Bayes classifier to them, I ...
0
votes
0answers
20 views

Cook's distance and AUC/Accuracy

I am working with balanced data set and trying to build a logistic regression model for prediction. I have 322 observations, 6 continuous independent variables, 152(170) positive(negative) values. ...
0
votes
1answer
25 views

Is this kind of stacked ensemble method prone to over-fitting?

I am working on a stacked ensemble method. I trained three classifiers as my first-layer models and one Logistic Regression as my second layer model. I then stacked both the first-layer models and ...
0
votes
0answers
15 views

Is it possible to evaluate too many models?

Even with nested cross-validation in which model selection is occurring on the inner loop, is it possible that the best model identified by the inner loop for testing on the outer loop is an overfit ...
0
votes
0answers
17 views

object detection loss

I have trained an ssd detector in my own dataset and the values of train loss and val loss are shown in the picture. However in all the epochs the value of val loss is lower than that of train loss ? ...
1
vote
0answers
14 views

How should I think about model selection when I will be aggregating predictions?

I'm estimating a model and then using it to predict values in another dataset. After predicting the values, I will aggregate the values to find averages within income quantiles. I know the ...
0
votes
0answers
10 views

High initial test validation score for Neural Network

From only the first epoch of training my NN with SGD (I use Xavier initialisation for weights), the accuracy shoots off to 92%, and then flattens out. The same thing happens with loss (but lower, of ...
0
votes
0answers
17 views

Is it possible to have overfitting within the first epoch of training?

Usually after training a few epochs we have overfitting and stop the training. But, is there any circumstances or is it possible that overfitting happens within the first epoch of training? Maybe ...
1
vote
1answer
53 views

Can logistic regression output a non linear curve?

I have a doubt regarding logistic regression.I know that it separates the data into 2 parts.Is it possible that it leads to a curve as shown in the example of underfitting and overfitting What i know ...
0
votes
0answers
11 views

What is the relationship between Validation loss and Training loss when considering Overfitting? [duplicate]

Here I have results from my training stage I have been told that this would not be considered as overfitting, however, it seems the line follows the dots well and the validation loss is higher than ...
0
votes
0answers
15 views

What does it indicate when Train and Test plot run parallel?

In the Model Accuracy plot of my NN model, the Train and Test shows a steep curve upto 50 epochs after which those run parallel upto 1500 epochs. Is it correct that bifurcation at a point signifies ...
0
votes
3answers
43 views

How to increase data samples to prevent over fitting?

As you know increase a data samples on training is a way to preventing of over fitting. I'm working on an UCI data set with 198 samples and 34 feature this is my data set's dimension and I wanted to ...
0
votes
0answers
20 views

Polynomial regression : how to find the best polynomial degree ? Chi2 or equivalent already built-in in Python numpy?

I am studying the stability of numerical derivatives as a function of the step I take to compute these derivatives. With a derivative with 15 points (obtained by the finite difference method), I get ...
2
votes
1answer
104 views

Overfitting in extreme gradient boosting

My situation is: 36,197 observations/ 125 outcomes in training data 26 predictors A relatively successful prediction model has been built in a similar dataset using just logistic regression; I ...
0
votes
3answers
84 views

Is this a case of network overfitting?

I am writing a network that classifies different species of butterfly, I have 9 epochs total. I have reached a wall as my major is in Physics, I am wondering if anyone can spot any distinct issues ...
0
votes
0answers
18 views

Detecting correct changepoint using cpt.reg and envept

I have used cpt.reg from envcpt and changepoint beta, and I am now getting these results, where I need to detect the blue arrow but it is not. I could change the penalties to try detect it, however ...
1
vote
1answer
58 views

Why is training of extremely deep fully-connected NNs difficult?

Practitioners know that if we increase the number of full-connected layers in Neural Network (NN), then at some points the NN performance starts to degrade. The natural reason is that we have ...
0
votes
0answers
25 views

Are my regularization results telling me that my model isn't overfitted?

In fitting a neural network, initial validation and learning curves seemed to indicate my model was overfitted. After trying a few options to decrease overfitting that had no impact, I'm questioning ...
0
votes
0answers
33 views

Scikit-learn: avoid overfitting in Gaussian Process regression

I am training a Gaussian Process to learn the mapping between a set of coordinates x,y,z and some time series. In a nutshell, my question is about how to prevent my ...
15
votes
5answers
2k views

Can increasing the amount of training data make overfitting worse?

Suppose I train a neural network on dataset A and evaluate on dataset B (that has a different feature distribution than dataset A). If I increase the amount of data in dataset A by a factor of 10, is ...

1 2 3 4 5 13