Questions tagged [tuning]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
15 views

what results are still usable from non-nested cross validation when tuning hyperparameters (and reconciling that with Optuna)

I'm trying to wrap my head around nested cross validation for the purposes of hyperparameter tuning. Part i) If I was to run hyperparameter tuning with cross validation (without nesting), say with ...
user avatar
  • 101
0 votes
0 answers
14 views

Why validation loss can drop a lot at a higher learning rate?

When we are choosing the optimal learning rate for a neural network, I thought the normal validation_loss & learning rate trend looks like this: Today when I'm running this pytorch forecasting ...
user avatar
  • 289
0 votes
0 answers
13 views

Hyperparameters chosen by CV on train dataset don't perform well on validation/test dataset

I've used the following objective function to assess best hyper-parameters using Hyperopt(): ...
user avatar
0 votes
1 answer
60 views

How do we perform hyperparameter tuning on parameter of data augmentation?

I was wondering how do we perform hyperparameter tuning on parameters of data augmentation. Suppose I have to select the best pair of (alpha, alpha) values of beta distribution that works best on data ...
user avatar
  • 13
2 votes
0 answers
34 views

Using hyperopt for finding n_estimators for ensemble model?

I am trying to tune my model (An ensemble model with xgboost, lightbgm and catboost) with hyperopt, but I don't know how to find the optimal n_estimators value. When I use the ...
user avatar
2 votes
2 answers
223 views

Range of Values for Hyperparameter Fine-Tuning in Random Forest Classification

I have implemented a random forest classifier. At the moment, I am thinking about how to tune the hyperparameters of the random forest. Of course, I am doing a gridsearch type of algorithm while ...
user avatar
  • 535
0 votes
0 answers
22 views

Why do the training and validation accuracy drop after tuning?

I thought my MLP (multi-layer perceptron)'s accuracy will increase after tuning. However, the accuracy dropped. Then someone told me that I should add Dropout layers with 50% dropping. I did that. ...
user avatar
  • 1,518
1 vote
0 answers
20 views

How many epochs should I iterate for tuning an ANN?

I have been using Keras-Tuner for tuning my ANN before going into training. The tuner seems to be iterating forever even though I set a limit of 1000 epochs. After that, I have decided to terminate ...
user avatar
  • 1,518
1 vote
0 answers
71 views

Tuning parameters for multiple regression models [closed]

I am trying to compare multiple regression algorithms to estimate biomass (dependant variable) : KNeighborsRegressor, GaussianProcessRegressor, LinearRegression, BayesianRidge, Ridge, SGDRegressor, ...
user avatar
2 votes
1 answer
41 views

Tuning hyperparameters with simulated data, do I need to use cross-validation or can I just give it simulated data sets from different seeds?

I am doing a method comparison of some machine learning models across certain scenarios. I simulated data where associations are known. To me, this seems like a simple way to have as much data as I ...
user avatar
0 votes
1 answer
18 views

Size of the training set versus complexity of the model during hyper-parameter tuning

In general, the higher is the complexity of a model (number of parameters) the larger should be the training size to try to avoid overfitting. With neural networks, one can have one or more hidden ...
user avatar
1 vote
0 answers
21 views

xgboost hyperparameters: interactions that make the model overfit on training set

I am dealing with a classification problem on an unbalanced dataset (positive class is just above 1% of the sample). I did hyperparameter tuning using a train-validation split, and then finally ...
user avatar
  • 111
0 votes
0 answers
23 views

Any advice on parameter tuning on large dataset for KNN and SVM using R

I am pursuing KNN and SVM models on a somewhat large dataset (80k training observations, 360k test observations, 23 features). I randomly picked some values of k between 1 and sqrt(n) and only testing ...
user avatar
  • 1
0 votes
0 answers
31 views

How to analyze output of PPO2 (clipped version)?

I am implementing the PPO2 algorithm for Atari Pong. The PPO is min(, clip(,,) * A) version, NOT the (beta * KL) constraint version. How to tune hyper-parameters by loss function output? Say, for each ...
user avatar
0 votes
0 answers
21 views

Choosing the Best Tuning Parameter Considering the Cutoff Point in Classification

Suppose that we want to predict a binary response "y" using "LASSO". We may first want to select the tuning parameter lambda (shrinkage parameter) using cross-validation. Here, the ...
user avatar
  • 501
0 votes
1 answer
53 views

Streamlining / Optimizing conditional Rules in a rule engine using ML techniques

The problem is defined as follows : We are dealing with a Rule engine used to classify a credit risk as 'Good', 'Medium' or 'bad'. Say the rule engine which has say 10 rules. These rules are ...
user avatar
  • 275
2 votes
0 answers
56 views

How many hyperparameters should be optimized in machine learning?

I am using neural network algorithms for a relatively large dataset with 1700 obs and 40 features. I performed optimizing by nested cross validation. I also wanted to compare 5 algorithm with each ...
user avatar
  • 197
0 votes
0 answers
240 views

Train and test score - overfitting?

I have hourly time series data with a range of two years. I want to test my model when predicting my target variable (continuous) for a specific week. I'm doing the following: Splitting my data into ...
user avatar
2 votes
1 answer
24 views

Need to do hyperparamter tuning for new features?

Suppose I have a set of features(say 100 features) and spent a lot of time doing hyperparameter tuning to get a good model. Now I have a few new features(say less than 5 new features) added into the ...
user avatar
  • 643
2 votes
0 answers
48 views

High dimensional hyperparameter tune

Many already known optimization techniques rely on past data (Bayesian optimization for instance) and perform really well for a bunch of hyperparameters. Is there, however, a good tuner/tuning method ...
user avatar
  • 21
0 votes
1 answer
50 views

Hyperparameter tuning vs weight tweaking in Cross-Validation: should I consider 2 different validation sets?

Let's say I have 1000 Samples and want to build an ANN. Then I split my dataset into train set (800) and test set (200). After that, I do the following Cross-validate my train set with different ...
user avatar
0 votes
1 answer
53 views

Cross-validation for hyperparameter tuning

I've read as many topics regarding hyperparameter tuning as I could, and I developed the following algorithm for hyperparameter tuning & final model building Split the data in train set (80%) &...
user avatar
1 vote
0 answers
65 views

How can one use Grid Search without overfitting the model?

I checked several questions, like Overfitting during model selection - AutoML vs Grid search and Hyperparameter tuning using grid search/randomised search, but I don't think any of them answer my ...
user avatar
  • 23
6 votes
1 answer
99 views

Is there a hard distinction between hyperparameter vs parameter in machine learning?

I was watching Andrew Ng's lecture on the difference between parameter vs hyperparameter, https://www.youtube.com/watch?v=VTE2KlfoO3Q&ab_channel=Deeplearning.ai, and a question came to me. Is ...
user avatar
  • 674
1 vote
1 answer
1k views

Proper way to incorporated CalibratedClassifierCV in cross-validation in Scikit

I'm creating some classifiers for a binary classification problem. I want to find out three things: Which algorithm I should use. Which set of hyperparameters I should use. If I should calibrate the ...
user avatar
  • 11
2 votes
2 answers
229 views

Tuning hyperparameters never affects weights?

I am trying to better understand “tuning the hyperparameters”. I understand how to use GridSearchCV, I found the below explanation useful: “As we do not know whether those parameters affect each other,...
user avatar
0 votes
0 answers
23 views

Is it possible to refine the parameters of a model by optimizing on something other than performance?

Suppose I have a model with some parameters which I want to tune so that the model performs better on some task. If I understand correctly, many supervised learning methods will train the model and ...
user avatar
2 votes
1 answer
338 views

Perceptual Loss Layers Selection

I understand that in order to improve your generative model performance it is quite useful to compare your output and the target in the feature space, as stated in the paper Perceptual Losses for Real-...
user avatar
  • 654
1 vote
0 answers
63 views

Is Hyperparameter Optimization A Discrete Or Continuous Problem?

I'm currently learning AI/machine-learning with Python and Scikit-learn. Not having a strong background in math, I'm confused on a certain point. Say I want to tune the parameters of a machine ...
user avatar
0 votes
1 answer
341 views

how to do the hyper parameter tunning for one class svm in r programming?

x is input (single column) tuned <- tune.svm(x=x, y =NULL, data=x, type= 'one-classification', tunecontrol = tune.control(sampling = "fix")) For this I am ...
user avatar
3 votes
1 answer
56 views

Model tuning in the presence of incorrect training labels

I have a situation where I have a large amount of labeled data (~40 million records) with a binary outcome variable that has about 50% positive and 50% negative cases. The issue is that I know that ...
user avatar
  • 1,408
4 votes
2 answers
1k views

Cross-validation for (hyper)parameter tuning to be performed in validation set or training set?

I am learning about the use of cross-validation with grid-search to choose the best hyperparameter for SVM. The problem I came across is the references and examples of its application do not follow a ...
user avatar
  • 1,017