Questions tagged [hyperparameter]

A parameter that is not strictly for the statistical model (or data generating process), but a parameter for the statistical method. It could be a parameter for: a family of prior distributions, smoothing, a penalty in regularization methods, or an optimization algorithm.

Filter by
Sorted by
Tagged with
0 votes
0 answers
15 views

What is a perfect distribution to consider for the step increase/ decrease for the reversible jump MCMC

I am trying to understand the hyper parameters in the paper [1] for the model order selection with reversible jump MCMC (RJ-MCMC). There is a hyper parameter $\Lambda$ (The parameter of the Poisson ...
0 votes
0 answers
26 views

gaussian process regression hyperparameter optimizatian Manual tuning vs Automatic parameter tuning

When the Gaussian process uses the ard kernel, if the feature is very obvious, but the prior knowledge knows that this feature is not very important, when using the marginal likelihood function to ...
1 vote
1 answer
22 views

Hyper parameter tunning of stochastic models

I struggle to find a clean answer for this question: how to tune hyperparameters while the model does not produce similar results for different runs? Before starting the tuning, I play around with the ...
  • 11
0 votes
1 answer
21 views

Do you tune hyperparameters for neural networks one at a time?

I wondered whether tuning hyperparameters, such as the learning rate or the amount of layers and neurons, is done seperately or alltogether. E.G. you first tune the amount of neurons and when getting ...
  • 43
1 vote
0 answers
11 views

Temporal cross-validation in forecasting: model selection, hyperparameter tuning and comparison to independent forecast

I'm mainly working with time-series models and want to make sure to build the correct model selection process. Let's consider a forecasting problem and I have two model candiates, Model A and Model B. ...
1 vote
2 answers
35 views

Can SVM overfit even with cross-validation?

I am using SVM regressor models to fit some chemical data related to spectroscopy (I cannot say exactly what data because it is an ongoing research in my group). To combat overfitting, I have used 5-...
  • 163
0 votes
0 answers
12 views

Setting the range of hyper parameter(gamma, cost) of supprot vector regression

I want to run svr with 7960 data. Prior to run svr, I searched the way picking up the optimal value of hyper parameter and found that there is an approach named grid search. But it also had a limit ...
  • 1
0 votes
0 answers
21 views

What is the conclusion from cross validation scores?

I am training a model, and I'm using an xgboost model. I used the following piece of code to find the cross-validation score score=cross_val_score(final_classifier, X, y, cv=5, scoring="f1") ...
  • 203
0 votes
0 answers
21 views

Hessian of log marginal likelihood is rank deficient

I am implementing the Bayesian MAP estimation algorithm presented in [1, Sec. 4.4]. More specifically, I am trying to estimate the hyper-parameters of the main MAP estimator by optimizing the Bayes ...
2 votes
2 answers
107 views

Best way to obtain probabilities and model explanations with imbalanced data

I am currently working on machine learning problem with the following characteristics:  - Data have binary outcomes and are severely imbalanced (positive class is ~0.5% of my sample of ~500,000 data ...
2 votes
2 answers
309 views

KNN: Should we randomly pick "folds" in RandomizedSearchCV?

TL;DR In KNN, K is the hyperparameter so we randomly pick it while performing RandomizedSearchCV. Should we also randomly pick the split [Cross-validation + Train] after k-folding? I am considering ...
2 votes
1 answer
41 views

What does it mean having 1 as best k parameter in K-NN?

I'm working with a large dataset (761 rows and about 57k-60k features) and after doing a feature selection to select the best 10 features I'm using different ML algorithms to classify some cases. In ...
  • 23
0 votes
0 answers
49 views

Validation performance is really problematic; should I give up on increasing the validation performance of my deep learning model? [duplicate]

I am working on a multiclassification problem using time series data. Three datasets are utilized in this study. My deep neural network performs satisfactorily across two different data sets. However, ...
  • 141
1 vote
0 answers
9 views

How to tune hyperparameters on 100 data sets generated by MICE

I am coding in Python and my data consists of 100 imputed data sets created by MICE in R. I am running Scikit-learn ML (Supervised Regression) algorithms to improve the prediction of Warfarin. My aim ...
10 votes
2 answers
1k views

What is wrong with treating everything as a hyperparameter?

I've seen a number of questions like this asking whether certain parameters can be treated as a hyperparameter. Why can't we just treat everything as a hyperparameter? I understand that this is an ...
0 votes
0 answers
9 views

Optimal number of steps per epoch for maximum accuracy on neural networks

This question has a very good answer discussing optimal mini-batch size for training neural networks, that points out that the final accuracy of a model usually decreases when using very large batches ...
1 vote
0 answers
104 views

How to do deal with varying performance across a wide range of parameters?

We are working on a recommendation system. Our data comes from varying sources that make it natural to train a model for each source. Now we come to the crux. For each of the data sources we get ...
  • 93
5 votes
2 answers
229 views

Does gridsearch on random forest/extra trees make sense?

I have seen many posts online about tuning the random forest hyperparameters with a gridsearch. however, since the random forest creates trees with some randomness, does this have sense? Wouldn't the ...
  • 109
0 votes
0 answers
55 views

How much improvement in performance should one expect from hyperparameter tuning?

Is there a general conclusion on what one should expect from hyperparameter tuning? For instance, is it always the case that hyperparameter can only increase the performance from OK to good (say 0.75 ...
  • 1
0 votes
0 answers
100 views

which dataset to send as eval set in xgboost, catboost, and etc, when using optuna

In some boost models there are option to send eval set while fitting the model. for example: ...
  • 77
1 vote
0 answers
14 views

how to evaluate Gaussian Process regressor

I am trying to do a blackbox hyper-parameter optimization with GPR. To achieve I use GPR with Matern core with sklearn. Now I hope to know whether it fitted well, or whether it is over-fitted or under-...
  • 145
2 votes
1 answer
33 views

Is it correct to replace the oldest data when using Gaussian Process in blackbox hyper-parameter optimization?

I try to apply GPR in a blackbox HPO question. My input will have 6 dimensions like X=[x1,...x6]. The implementation is quite straightforward with sklearn with a ...
  • 145
0 votes
0 answers
29 views

How to restore Optuna's finished study from logs?

I wish to restore optuna.study.Study class object from uncomplete (5000 last lines, which is enough for my goals) logs. They look like that: ...
1 vote
1 answer
43 views

How does randomized search cv algorithm work?

I am building a DNN, and I used Randomizedsearchcv from Scikit learn to optimise the hyper-parameters. Hence, I have one question about this: As I understood, the basic of random search is to try out ...
  • 13
1 vote
0 answers
29 views

Why isn't RandomSearchCV returning the optimum parameters for the XGBoost Model, and how can I avoid Overfitting?

I have a dataset for energy consumer customers and binary target variables with which I want to predict the churn for the customers. Counts of target values Not Churn 0: 14153 Churn 1: 1520 I have ...
  • 31
3 votes
1 answer
331 views

What exactly is min_samples_leaf in scikit-learn's RandomForestClassifier?

I'm confused about a particular part of the documentation. I want to know what min_samples_leaf refers to when it's input as a float. min_samples_leaf : int or ...
  • 115
2 votes
1 answer
109 views

How to choose a parameter grid for a model's hyperparameter tuning?

How do people decide on the ranges for hyperparameters to tune? For example, I am tuning an xgboost model, I've been following a guide on kaggle to set the ranges of each hyperparameter to then do a ...
  • 23
1 vote
1 answer
157 views

How to run Bayesian optimization experiments in parallel?

Suppose I have the following hyperparameters for tuning: learning_rate: [0.00001, 0.1] epochs: [200,300,400,....,1000] batch_size: [16,32,64,128] If I want to run experiments using 4 parallel jobs, ...
  • 867
1 vote
0 answers
23 views

How much reduction of hyperparameter experiments can I get using Bayesian optimization vs Grid search?

I have 5 hyperparameters for tuning and the number of combinations of all possible values is 9,360. This means if I want to find the optimal parameter setting using Grid Search, I need to do 9,360 ...
  • 867
1 vote
0 answers
96 views

If a Random Forest scores high accuracy on test set, but high false negative on real-world application, is this an overfit?

I'm very new to Machine Learning and I'm making a Random Forest Classifier to classify if a file is a malware or benign. This is done through analyzing the PE file's headers. The dataset used contains ...
1 vote
0 answers
144 views

Neural network for imbalanced data

I have an imbalanced data (n = 600, about 97% majority and 3% minority) with 20 features and a binary outcome. The data has been split into a training set and a test set (80%/20%). I used H2o autoML ...
1 vote
0 answers
33 views

Hyperparameters tuning for all seeds in MLP [closed]

I am trying to train a multilayer perceptron model and I need help deciding how to tune the hyperparameters. Any help/guide on how to choose the values of the hyperparameters? The outcome is binary ...
0 votes
0 answers
31 views

Can I perform hyperparameter tuning inside K-fold CV?

I am performing leave one subject out cross-validation, using one subject as the independent final test set to get the performance of my model. Can I perform hyperparameter tuning inside each of my K-...
1 vote
0 answers
26 views

Hyper parameters to tune in Bayesian network?

In tree based models and neural networks, we can optimised the models by tuning the hyper parameters(such as: learning rate, number of neutrons.. etc). Is there a hyper parameters to tune in Bayesian ...
  • 21
2 votes
1 answer
114 views

Should folds in k-fold CV actually be representative?

I have read somewhere that the k of the k-fold CV should be picked in such a manner as to have representative validation sets (folds). This seems to me to be contradictory since the leave-one-out CV ...
0 votes
1 answer
81 views

evaluating scoring metrics during hyperparameter tuning

I'm struggling with a couple of concepts related to hyperparameter tuning. I'm developing a model (gradient boosted tree) in python using sklearn. Currently, I'm in the process of using sklearn's ...
0 votes
0 answers
60 views

what results are still usable from non-nested cross validation when tuning hyperparameters (and reconciling that with Optuna)

I'm trying to wrap my head around nested cross validation for the purposes of hyperparameter tuning. Part i) If I was to run hyperparameter tuning with cross validation (without nesting), say with ...
  • 111
0 votes
0 answers
18 views

Hyperparameters chosen by CV on train dataset don't perform well on validation/test dataset

I've used the following objective function to assess best hyper-parameters using Hyperopt(): ...
0 votes
0 answers
37 views

Hyperparameter tuning on training data vs validation data

If we divide the data into training data, validation data, and testing data, I remember the lesson from Andrew Ng saying we use the validation data for hyperparameter tuning purpose. (you can see this ...
  • 1,027
0 votes
0 answers
33 views

Does it make sense to worry about stability of parameters?

I'm working on a problem where I'm using grid search on logistic regression and I'm checking two parameters, penalty and C. I ...
1 vote
1 answer
55 views

Can the term 'hyperparameter' apply to non-ML modelling?

Commonly when modelling biological systems, some parameters may be from elsewhere or previous modelling fits, and are not being investigated in the current model. These seem to be equivalent to the ML ...
  • 15
0 votes
0 answers
18 views

Developing an optimized model after nested cross validation

I have been assigned to learn about k-fold cross-validation for my class. As an extension, I wanted to learn more about nested cross-validation. I understand that nested cross-validation involves ...
1 vote
1 answer
196 views

How to choose a model's hyperparameters in terms of the variance?

I was solving this question about tuning hyperparameters and I don't understand how to choose the number of hyperparameters by using the training error (TE) and the validation error (VE). Define the ...
  • 111
1 vote
0 answers
162 views

How to get the best num_boost_round on the full training data?

I have a huge training data of size 5.5 GBs with over 55m rows. Because iterating over the whole dataset again and again was too slow, I used a 1% sample of this whole data to select the best ...
2 votes
0 answers
92 views

Hyperparameters tuning on GANs

I have seen this post talking about how to tune hyperparameters on GANs. I'm actually wondering, more generally, how does one go about tuning hyperparameters on GANs. Obviously you cannot (I mean you ...
2 votes
1 answer
88 views

Is it logical to combine cross-validation estimator like RidgeCV with cross_val_score in sklearn?

I was going through solutions for a regression problem competition on Kaggle here. Many solutions for the problem are combining cross-validation estimators like RidgeCV, LassoCV with cross_val_score ...
1 vote
0 answers
36 views

Machine learning regularization parameter lambda proof [closed]

Consider the regularized empirical risk minimization problem L(x) + λ * r(x), where L(x) is the empirical risk, r(x) is the regularizer, and lambda is the regularization hyper-parameter. I have 2 ...
  • 11
2 votes
2 answers
1k views

Using k-fold cross-validation of random forest: how many samples are used to create a tree?

I'm trying to tune the hyperparameters of my RandomForestRegressor created in python with sklearn with bootstrap = True using <...
  • 21
2 votes
1 answer
411 views

Why does XGBoost with cross-validation perform worse on test holdout than unvalidated model?

I have an XGBoost model that I fit on some X data directly out of the box: ...
  • 115
4 votes
2 answers
1k views

How to choose $\gamma$ parameter in Focal Loss?

I would like to know if exists a rule of thumb to set the $\gamma$ parameter in Focal Loss when we have very imbalanced classes. The focal loss first appeared in Focal Loss for Dense Object Detection ...
  • 251

1
2 3 4 5
13