Questions tagged [tuning]
The tuning tag has no usage guidance.
32
questions
0
votes
0
answers
15
views
what results are still usable from non-nested cross validation when tuning hyperparameters (and reconciling that with Optuna)
I'm trying to wrap my head around nested cross validation for the purposes of hyperparameter tuning.
Part i)
If I was to run hyperparameter tuning with cross validation (without nesting), say with ...
- 101
0
votes
0
answers
14
views
Why validation loss can drop a lot at a higher learning rate?
When we are choosing the optimal learning rate for a neural network, I thought the normal validation_loss & learning rate trend looks like this:
Today when I'm running this pytorch forecasting ...
- 289
0
votes
0
answers
13
views
Hyperparameters chosen by CV on train dataset don't perform well on validation/test dataset
I've used the following objective function to assess best hyper-parameters using Hyperopt():
...
- 123
0
votes
1
answer
60
views
How do we perform hyperparameter tuning on parameter of data augmentation?
I was wondering how do we perform hyperparameter tuning on parameters of data augmentation. Suppose I have to select the best pair of (alpha, alpha) values of beta distribution that works best on data ...
- 13
2
votes
0
answers
34
views
Using hyperopt for finding n_estimators for ensemble model?
I am trying to tune my model (An ensemble model with xgboost, lightbgm and catboost) with hyperopt, but I don't know how to find the optimal n_estimators value. When I use the ...
- 121
2
votes
2
answers
223
views
Range of Values for Hyperparameter Fine-Tuning in Random Forest Classification
I have implemented a random forest classifier. At the moment, I am thinking about how to tune the hyperparameters of the random forest. Of course, I am doing a gridsearch type of algorithm while ...
- 535
0
votes
0
answers
22
views
Why do the training and validation accuracy drop after tuning?
I thought my MLP (multi-layer perceptron)'s accuracy will increase after tuning. However, the accuracy dropped.
Then someone told me that I should add Dropout layers with 50% dropping.
I did that. ...
- 1,518
1
vote
0
answers
20
views
How many epochs should I iterate for tuning an ANN?
I have been using Keras-Tuner for tuning my ANN before going into training. The tuner seems to be iterating forever even though I set a limit of 1000 epochs.
After that, I have decided to terminate ...
- 1,518
1
vote
0
answers
71
views
Tuning parameters for multiple regression models [closed]
I am trying to compare multiple regression algorithms to estimate biomass (dependant variable) : KNeighborsRegressor, GaussianProcessRegressor, LinearRegression, BayesianRidge, Ridge, SGDRegressor, ...
- 55
2
votes
1
answer
41
views
Tuning hyperparameters with simulated data, do I need to use cross-validation or can I just give it simulated data sets from different seeds?
I am doing a method comparison of some machine learning models across certain scenarios. I simulated data where associations are known. To me, this seems like a simple way to have as much data as I ...
0
votes
1
answer
18
views
Size of the training set versus complexity of the model during hyper-parameter tuning
In general, the higher is the complexity of a model (number of parameters) the larger should be the training size to try to avoid overfitting. With neural networks, one can have one or more hidden ...
- 1
1
vote
0
answers
21
views
xgboost hyperparameters: interactions that make the model overfit on training set
I am dealing with a classification problem on an unbalanced dataset (positive class is just above 1% of the sample).
I did hyperparameter tuning using a train-validation split, and then finally ...
- 111
0
votes
0
answers
23
views
Any advice on parameter tuning on large dataset for KNN and SVM using R
I am pursuing KNN and SVM models on a somewhat large dataset (80k training observations, 360k test observations, 23 features). I randomly picked some values of k between 1 and sqrt(n) and only testing ...
- 1
0
votes
0
answers
31
views
How to analyze output of PPO2 (clipped version)?
I am implementing the PPO2 algorithm for Atari Pong. The PPO is min(, clip(,,) * A) version, NOT the (beta * KL) constraint version. How to tune hyper-parameters by loss function output?
Say, for each ...
0
votes
0
answers
21
views
Choosing the Best Tuning Parameter Considering the Cutoff Point in Classification
Suppose that we want to predict a binary response "y" using "LASSO".
We may first want to select the tuning parameter lambda (shrinkage parameter) using cross-validation.
Here, the ...
- 501
0
votes
1
answer
53
views
Streamlining / Optimizing conditional Rules in a rule engine using ML techniques
The problem is defined as follows :
We are dealing with a Rule engine used to classify a credit risk as 'Good', 'Medium' or 'bad'. Say the rule engine which has say 10 rules. These rules are ...
- 275
2
votes
0
answers
56
views
How many hyperparameters should be optimized in machine learning?
I am using neural network algorithms for a relatively large dataset with 1700 obs and 40 features. I performed optimizing by nested cross validation. I also wanted to compare 5 algorithm with each ...
- 197
0
votes
0
answers
240
views
Train and test score - overfitting?
I have hourly time series data with a range of two years. I want to test my model when predicting my target variable (continuous) for a specific week. I'm doing the following:
Splitting my data into ...
- 57
2
votes
1
answer
24
views
Need to do hyperparamter tuning for new features?
Suppose I have a set of features(say 100 features) and spent a lot of time doing hyperparameter tuning to get a good model. Now I have a few new features(say less than 5 new features) added into the ...
- 643
2
votes
0
answers
48
views
High dimensional hyperparameter tune
Many already known optimization techniques rely on past data (Bayesian optimization for instance) and perform really well for a bunch of hyperparameters. Is there, however, a good tuner/tuning method ...
- 21
0
votes
1
answer
50
views
Hyperparameter tuning vs weight tweaking in Cross-Validation: should I consider 2 different validation sets?
Let's say I have 1000 Samples and want to build an ANN. Then I split my dataset into train set (800) and test set (200). After that, I do the following
Cross-validate my train set with different ...
- 3
0
votes
1
answer
53
views
Cross-validation for hyperparameter tuning
I've read as many topics regarding hyperparameter tuning as I could, and I developed the following algorithm for hyperparameter tuning & final model building
Split the data in train set (80%) &...
- 3
1
vote
0
answers
65
views
How can one use Grid Search without overfitting the model?
I checked several questions, like Overfitting during model selection - AutoML vs Grid search and Hyperparameter tuning using grid search/randomised search, but I don't think any of them answer my ...
- 23
6
votes
1
answer
99
views
Is there a hard distinction between hyperparameter vs parameter in machine learning?
I was watching Andrew Ng's lecture on the difference between parameter vs hyperparameter, https://www.youtube.com/watch?v=VTE2KlfoO3Q&ab_channel=Deeplearning.ai, and a question came to me.
Is ...
- 674
1
vote
1
answer
1k
views
Proper way to incorporated CalibratedClassifierCV in cross-validation in Scikit
I'm creating some classifiers for a binary classification problem. I want to find out three things:
Which algorithm I should use.
Which set of hyperparameters I
should use.
If I should calibrate the ...
- 11
2
votes
2
answers
229
views
Tuning hyperparameters never affects weights?
I am trying to better understand “tuning the hyperparameters”. I understand how to use GridSearchCV, I found the below explanation useful:
“As we do not know whether those parameters affect each other,...
- 659
0
votes
0
answers
23
views
Is it possible to refine the parameters of a model by optimizing on something other than performance?
Suppose I have a model with some parameters which I want to tune so that the model performs better on some task. If I understand correctly, many supervised learning methods will train the model and ...
- 83
2
votes
1
answer
338
views
Perceptual Loss Layers Selection
I understand that in order to improve your generative model performance it is quite useful to compare your output and the target in the feature space, as stated in the paper Perceptual Losses for Real-...
- 654
1
vote
0
answers
63
views
Is Hyperparameter Optimization A Discrete Or Continuous Problem?
I'm currently learning AI/machine-learning with Python and Scikit-learn. Not having a strong background in math, I'm confused on a certain point.
Say I want to tune the parameters of a machine ...
0
votes
1
answer
341
views
how to do the hyper parameter tunning for one class svm in r programming?
x is input (single column)
tuned <- tune.svm(x=x, y =NULL, data=x,
type= 'one-classification',
tunecontrol = tune.control(sampling = "fix"))
For this I am ...
- 1
3
votes
1
answer
56
views
Model tuning in the presence of incorrect training labels
I have a situation where I have a large amount of labeled data (~40 million records) with a binary outcome variable that has about 50% positive and 50% negative cases. The issue is that I know that ...
- 1,408
4
votes
2
answers
1k
views
Cross-validation for (hyper)parameter tuning to be performed in validation set or training set?
I am learning about the use of cross-validation with grid-search to choose the best hyperparameter for SVM. The problem I came across is the references and examples of its application do not follow a ...
- 1,017