Questions tagged [hyperparameter]

A parameter that is not strictly for the statistical model (or data generating process), but a parameter for the statistical method. It could be a parameter for: a family of prior distributions, smoothing, a penalty in regularization methods, or an optimization algorithm.

Filter by
Sorted by
Tagged with
0 votes
0 answers
5 views

With keras tuner, is there a canonical / best practice way to pass non-hyperparameters to the model building function?

My code handles meta data and execution steps for an ensemble of deep learning pipelines. Model discovery and tuning is done with keras tuner. A range of logical parameters and hyperparameters are set ...
user avatar
0 votes
0 answers
6 views

Hyper parameters to tune in Bayesian network?

In tree based models and neural networks, we can optimised the models by tuning the hyper parameters(such as: learning rate, number of neutrons.. etc). Is there a hyper parameters to tune in Bayesian ...
user avatar
  • 1
2 votes
1 answer
72 views

Should folds in k-fold CV actually be representative?

I have read somewhere that the k of the k-fold CV should be picked in such a manner as to have representative validation sets (folds). This seems to me to be contradictory since the leave-one-out CV ...
user avatar
0 votes
1 answer
23 views

evaluating scoring metrics during hyperparameter tuning

I'm struggling with a couple of concepts related to hyperparameter tuning. I'm developing a model (gradient boosted tree) in python using sklearn. Currently, I'm in the process of using sklearn's ...
user avatar
0 votes
0 answers
17 views

what results are still usable from non-nested cross validation when tuning hyperparameters (and reconciling that with Optuna)

I'm trying to wrap my head around nested cross validation for the purposes of hyperparameter tuning. Part i) If I was to run hyperparameter tuning with cross validation (without nesting), say with ...
user avatar
  • 101
0 votes
0 answers
13 views

Hyperparameters chosen by CV on train dataset don't perform well on validation/test dataset

I've used the following objective function to assess best hyper-parameters using Hyperopt(): ...
user avatar
0 votes
0 answers
15 views

Hyperparameter tuning on training data vs validation data

If we divide the data into training data, validation data, and testing data, I remember the lesson from Andrew Ng saying we use the validation data for hyperparameter tuning purpose. (you can see this ...
user avatar
0 votes
0 answers
13 views

Does it make sense to worry about stability of parameters?

I'm working on a problem where I'm using grid search on logistic regression and I'm checking two parameters, penalty and C. I ...
user avatar
1 vote
1 answer
50 views

Can the term 'hyperparameter' apply to non-ML modelling?

Commonly when modelling biological systems, some parameters may be from elsewhere or previous modelling fits, and are not being investigated in the current model. These seem to be equivalent to the ML ...
user avatar
  • 15
0 votes
0 answers
13 views

Developing an optimized model after nested cross validation

I have been assigned to learn about k-fold cross-validation for my class. As an extension, I wanted to learn more about nested cross-validation. I understand that nested cross-validation involves ...
user avatar
1 vote
1 answer
102 views

How to choose a model's hyperparameters in terms of the variance?

I was solving this question about tuning hyperparameters and I don't understand how to choose the number of hyperparameters by using the training error (TE) and the validation error (VE). Define the ...
user avatar
  • 111
1 vote
0 answers
26 views

How to get the best num_boost_round on the full training data?

I have a huge training data of size 5.5 GBs with over 55m rows. Because iterating over the whole dataset again and again was too slow, I used a 1% sample of this whole data to select the best ...
user avatar
1 vote
0 answers
146 views

Obtaining approximate posterior probabilities with Bayesian cross-validation

(Apologies to anyone that may have been following this question: I have decided to rewrite it to make it more succinct. As a result, comments below now appear out of context.) Given a set of models $\{...
user avatar
  • 1,125
2 votes
0 answers
74 views

Hyperparameters tuning on GANs

I have seen this post talking about how to tune hyperparameters on GANs. I'm actually wondering, more generally, how does one go about tuning hyperparameters on GANs. Obviously you cannot (I mean you ...
user avatar
2 votes
1 answer
36 views

Is it logical to combine cross-validation estimator like RidgeCV with cross_val_score in sklearn?

I was going through solutions for a regression problem competition on Kaggle here. Many solutions for the problem are combining cross-validation estimators like RidgeCV, LassoCV with cross_val_score ...
user avatar
1 vote
0 answers
19 views

Machine learning regularization parameter lambda proof [closed]

Consider the regularized empirical risk minimization problem L(x) + λ * r(x), where L(x) is the empirical risk, r(x) is the regularizer, and lambda is the regularization hyper-parameter. I have 2 ...
user avatar
  • 11
2 votes
2 answers
62 views

Using k-fold cross-validation of random forest: how many samples are used to create a tree?

I'm trying to tune the hyperparameters of my RandomForestRegressor created in python with sklearn with bootstrap = True using <...
user avatar
  • 21
2 votes
1 answer
78 views

Why does XGBoost with cross-validation perform worse on test holdout than unvalidated model?

I have an XGBoost model that I fit on some X data directly out of the box: ...
user avatar
2 votes
2 answers
145 views

How to choose $\gamma$ parameter in Focal Loss?

I would like to know if exists a rule of thumb to set the $\gamma$ parameter in Focal Loss when we have very imbalanced classes. The focal loss first appeared in Focal Loss for Dense Object Detection ...
user avatar
  • 221
1 vote
0 answers
29 views

Bayesian optimization by groups

I am currently comparing different models for a text classification task using tweets. Some of them are decision trees (random forests) and, since I augment my data periodically, I need to do ...
user avatar
  • 11
1 vote
1 answer
39 views

Package for hyperparameter optimization with categorical values

Context I'm trying to solve a black-box optimization problem, and I can "reformulate" parts of the problem is different ways that may lead to lower or higher costs, and which can interact ...
user avatar
0 votes
0 answers
18 views

Discussion of hyperparamter to optimize

For my thesis I consider to add a small discussion about hyperparameter optimization (HPO). I have five tree based models I would like to tune. Namely RandomForest, GBM, XGBoost, LightGBM and CatBoost....
user avatar
  • 218
1 vote
1 answer
27 views

What is the difference between Bayesian Optimizaiton for hyperparameters and using validation set when training?

While studying hyperparameter tuning in Machine Learning, I have come to read Bayesian Optimization for Hyperparameter Tuning and Using validation set when training the model but it is kind of ...
user avatar
  • 2,583
13 votes
4 answers
1k views

How simple should a Baseline model be?

I was making baseline model and was wondering how much time I should spend on it. I've found a lot of article about the purpose of a baseline model, for instance; A baseline takes only 10% of the ...
user avatar
1 vote
1 answer
62 views

Hyperparameter tuning required for feature selection using wrapper methods?

I am working on binary classification with class proportion of 77:23 (977 records) Currently, I am exploring the feature selection approaches and came across methods like below a) Featurewiz b) ...
user avatar
  • 1,780
1 vote
0 answers
11 views

Are there any hyperparameter search approaches which fine-tune rather than train from scratch?

Whenever I hear about any hyperparameter search approach for training neural networks, the models are always trained from scratch. I'm wondering if there is any validity in starting from a converged ...
user avatar
  • 73
0 votes
0 answers
18 views

Hyperparameter optimization for a reinforcement learning agent under a special case

Let's say an RL agent is deployed(statically; no exploration or learning) on a real dynamic system. From time to time, due to the non-stationarity of the system, the agent is being trained on a data-...
user avatar
  • 153
2 votes
2 answers
109 views

How to deal with random parameters in MLOps

I have a XGB model ready to go to production, in validation I discovered that the random seed makes reasonable difference in the performance of the model, which is pretty good, but for some seeds it's ...
user avatar
0 votes
0 answers
61 views

Nested cross-validation with feature selection and hyperparameter tuning

I have been searching for a while for an answer, I have found a lot of good content! However, I am still doubting the way I am performing my training procedure with cross-validation. I am training a ...
user avatar
  • 1
0 votes
0 answers
32 views

5 fold cross validation for hyper parameter tuning on same set as feature selection

I performed hyperparameter tuning and feature selection on a dataset and I am wondering if it is correct. I have two datasets, a training set and a test set. The training set is split into a train2 ...
user avatar
  • 1
0 votes
2 answers
45 views

XGBoost heavily overfitting when containing the minimum/maximum of a variable?

I've been building an XGBoost Regressor model with some good success. Currently, the training accuracy is 68% and testing 66% - indicating some, but not too much, overfitting. However, I've noticed ...
user avatar
1 vote
2 answers
94 views

When is it okay to accept overfit model for production?

I am working on a binary classification problem using random forests (75:25 class proportion).m label 0 is minority class. I am following the below approach a) execute RF with default ...
user avatar
  • 1,780
2 votes
1 answer
127 views

Choosing "Target Entropy" for Soft-Actor-Critic (SAC) algorithm

I am quite familiar with Soft-Actor-Critic (SAC) and its many applications in continuous control RL environments. However, when implementing this algorithm in a practical setting, one thing that still ...
user avatar
  • 163
0 votes
1 answer
27 views

Should I use again Cross-Validation (retrain) on the amalgamation of the training and test set, fur tuning the regularization parameters?

As far as I know, it is a good idea to retrain the model on all the data available (train, validation, and test) after finding the best Hyperparameters values by Cross-Validation. However, some ...
user avatar
  • 67
0 votes
0 answers
67 views

XGBoost: is tuning max depth effectively the same as tuning min_child_weight, or vice versa?

XGBoost allows the user to tune both parameters, max depth and min_child_weight. Since both hyperparameters control the depth of trees, it seems that you would only really need to tune one. However, ...
user avatar
0 votes
1 answer
61 views

How do we perform hyperparameter tuning on parameter of data augmentation?

I was wondering how do we perform hyperparameter tuning on parameters of data augmentation. Suppose I have to select the best pair of (alpha, alpha) values of beta distribution that works best on data ...
user avatar
  • 13
1 vote
1 answer
27 views

features selection and hyperparameter tuning

I am testing optimal subsets of features and I choose the SVM classifier. In the process, the training set is used for feature selection to train models with different subsets, validate the subset on ...
user avatar
  • 11
0 votes
0 answers
57 views

Training, Tuning, Cross-Validating, and Testing Ranger (Random Forest) Quantile Regression Model?

May someone share how to train, tune (hyperparameters), cross-validate, and test a ranger quantile regression model, along with error evaluation? With the iris or Boston housing dataset? The reason I ...
user avatar
  • 113
2 votes
0 answers
34 views

Using hyperopt for finding n_estimators for ensemble model?

I am trying to tune my model (An ensemble model with xgboost, lightbgm and catboost) with hyperopt, but I don't know how to find the optimal n_estimators value. When I use the ...
user avatar
0 votes
0 answers
50 views

What is a good method for applying grid search on ensemble models?

I built an experiment where i am studying the performance of ensemble models for a classification task. Basically, i'm comparing Random Forest with Adaboost. However, Adaboost is built with a mix of ...
user avatar
  • 133
0 votes
0 answers
21 views

Convergence issue tuning parameters for logistic regression to maximise recall

I'm currently working on a fraud detection project and I am trying to do a gridsearch with a logistic regression to find good values for parameters "C" and "class_weight". The ...
user avatar
2 votes
2 answers
234 views

Range of Values for Hyperparameter Fine-Tuning in Random Forest Classification

I have implemented a random forest classifier. At the moment, I am thinking about how to tune the hyperparameters of the random forest. Of course, I am doing a gridsearch type of algorithm while ...
user avatar
  • 535
0 votes
0 answers
22 views

Why do the training and validation accuracy drop after tuning?

I thought my MLP (multi-layer perceptron)'s accuracy will increase after tuning. However, the accuracy dropped. Then someone told me that I should add Dropout layers with 50% dropping. I did that. ...
user avatar
  • 1,518
0 votes
0 answers
10 views

What should I optimize when applying Hidden Markov Model to classification problem?

My goal is to classify device as 1 of 5 possible types based on timeseries of its power consumption. I am using the following procedure: Initialize 5 instances of Hidden Markov Model with Gaussian ...
user avatar
1 vote
2 answers
42 views

Using the mean of predict_proba outputs as an indicator of potential classifier accuracy for semi supervised learning

Compare these code examples: ...
user avatar
  • 61
0 votes
0 answers
13 views

Domain weights as hyperparameters

Suppose I have data from two different sources, $(X_1, y_1)$ and $(X_2, y_2)$, where $X_1$ and $X_2$ have the same variables but different distributions, and similarly for $y_1$ and $y_2$. I want to ...
user avatar
  • 796
1 vote
2 answers
43 views

Can I compare different hyperparameters combination performance on testing dataset

I am runing a GridSearch with different range of hyperparameter values in order to find the best ones based on performance metrics (F1, AUC, etc.). I however have an imbalanced dataset, so I need to ...
user avatar
  • 31
2 votes
0 answers
34 views

Choosing best hyperparameters for multiple regression when number of features is higher than number of samples

I am a chemist mostly, and I do not have much experience in statistical learning. However, I am currently starting work on a problem that requires multiple regression. I have a set of molecules, for ...
user avatar
  • 153
0 votes
0 answers
16 views

Is it legitimate to use the performance on testing dataset to find best hyperparameters with GridSearch

I am runing a GridSearch with multiple machine learning algorithm (classification tree (CT), random foest (RF, LogisticRegression (RL) and Neural Network (NN)) in order to find the best hyperparameter ...
user avatar
  • 31
1 vote
0 answers
28 views

What is the difference between architecture selection and hyperparameter tuning in the context of neural networks

I've already seen the response to this questions here, but it doesn't really contain, what I feel, is an answer to my specific query. It seems to unclear in a lot of the literature (that I've seen) ...
user avatar

1
2 3 4 5
12