Questions tagged [hyperparameter]

A parameter that is not strictly for the statistical model (or data generating process), but a parameter for the statistical method. It could be a parameter for: a family of prior distributions, smoothing, a penalty in regularization methods, or an optimization algorithm.

Filter by
Sorted by
Tagged with
0 votes
0 answers
16 views

How to evaluate performance and optimize hyperparameters for clustering algorithms on a dataset with continuous labels?

I'm working on a clustering problem where my dataset's labels are continuous numerical values, not discrete categories. I'm using t-SNE and UMAP to reduce the dimensionality of my dataset's features ...
Jason Shi's user avatar
1 vote
0 answers
34 views

Hyperparameter tuning for small datasets

I have about 10 small imbalanced datasets (some of them only have about 150 samples). I want to try a bunch of balancing techniques on some models. For that, I'm using the repeated stratified cross-...
beautifularmy's user avatar
0 votes
0 answers
14 views

How to develop shared bottom tower serving different tasks

I have two model classes both pyramid architecture. Let's say first task is predicting user will buy something with architecture [feature_embedding_128, dense_1048, dense_512, dense_128, dense_1] ...
aghd's user avatar
  • 314
0 votes
0 answers
11 views

Hyperparameter optimization for CNN

I have a database of defect images on materials, like holes, cuts, and so on. There is not so much information inside the images, I am aware of it. I am using a CNN, in particular a ResNet50. I know ...
Jonny_92's user avatar
3 votes
1 answer
43 views

Binary decision boundary requiring 2 hidden layers in neural network with limited neurons

I just started learning about neural networks and was wondering what a neural network with 2 hidden layers is able to express over a neural network with just 1 hidden layer (where number of neurons ...
Regina Dea's user avatar
0 votes
0 answers
13 views

Determine if the best output of a large number of model fits is due to overtraining

I have a machine learning model (e.g. Gradient Boost Regressor). I use hyperparameter tunning (e.g. random search) in a Cross Validation loop (e.g. 5 folds) in order to recover the optimal parameters ...
karoto's user avatar
  • 1
0 votes
0 answers
43 views

SMOTE and Sequential Feature Selection Order

Good morning, I am doing the following procedure: Split a Train a Test Dataset ...
Andres Portocarrero's user avatar
0 votes
0 answers
66 views

Advice on Gaussian Process Classifier optimisation best practises? [duplicate]

Hyperparameter Range Determination: My main challenge is in setting effective ranges for hyperparameters such as length_scale, noise_level, and sigma_0. Currently, for length_scale, I've used the ...
Achilleas Pavlou's user avatar
2 votes
0 answers
33 views

Should threshold dependent metrics be used as optimization objectives during hyperparameter tuning for classification models?

What metric should we optimize for during hyperparameter tuning? From what I gather from Frank Harrell's article and other related questions here (Reduce Classification Probability Threshold), the ...
Tianxun Zhou's user avatar
3 votes
0 answers
65 views

Literature on GP with kernels having no closed form

I have to use GP regression on a complex time series, and the kernel function is not known in closed form. I have found a numerical approximation with the Gauss-Laugerre quadrature. It takes the ...
CfourPiO's user avatar
  • 275
0 votes
0 answers
11 views

Why is hyper-parameter tuning decreasing training set performance

Setup: I am using hyperopt for xgboost hyper-parameter tuning. In each trial, the corresponding configuration is evaluated with time-series cross-validation (CV). The CV validation performance the ...
Enk9456's user avatar
  • 33
0 votes
0 answers
26 views

best practices on optimizing feature transforms for a model

I have a regression model that Transforms some time series features using a different halflife for each feature Uses the transformed features along with some other features to create a prediction ...
Arran Duff's user avatar
0 votes
0 answers
12 views

A surrogate function for validation error in order to perform hyper-parameter optimization?

Greed search CV or few other approaches may be computationally expensive in hyper-parameter tuning. Is it possible to come up with a surrogate model or a purposed model for a validation error in order ...
Lakshman's user avatar
0 votes
0 answers
24 views

Model fitting with Chinese Restaurant Process

I am trying cluster a trajectory, consisting of (state, action) sequences, by assigning them to the most likely model that generated them using Chinese Restaurant Process. Basically my goal is to ...
jumov's user avatar
  • 51
0 votes
1 answer
47 views

Is r2 train= 0.977 and r2 test = 0.941 overfitting for a machine laerning regresssion model?

I'm tuning the hyperparameters for different Machine Learning models with Optuna package. For ExtraTreesRegressor I got r2 train = 0.938, r2 test = 0.922 and RMSE test = 0.907. In this case the ...
Etemon's user avatar
  • 121
4 votes
0 answers
146 views

Bayesian Optimization: number of iterations as function of search space dimensionality?

I am performing Bayesian Optimization to select a hyperparameter configuration for my supervised learning model. I understand that with each additional hyperparameter that I choose to optimize, the ...
David's user avatar
  • 1,256
0 votes
0 answers
29 views

Why does my random forest regression model have a 'noisy' fit to training data and a terrible fit to testing data though there is no extrapolation?

I have multiple datasets from an experimental setup and I'm trying to predict one of the system variables using information from the others. I concatenated 80% of the datasets into training data and ...
KN202's user avatar
  • 1
0 votes
0 answers
49 views

Getting difficulties which hyperparameters I have to select (5-Fold CV with optuna to find optimal GB regression model's hyperparameters)

I'm currently searching best hyperparameters for a regression model of Gradient Boost, using 5-Fold Cross-Validation and OPTUNA. However, I'm uncertain about which model's hyperparameters to choose, ...
Guy's user avatar
  • 1
0 votes
0 answers
75 views

Is Hyperparameter Tuning for Maximized Recall a Bad Thing?

I have a somewhat theoretical question: I work in an area that requires a number of anomaly detection solutions. When we approach these problems, we cross-validate and for each fold, we oversample ...
Branden Keck's user avatar
0 votes
0 answers
38 views

Generalized cross-validation (GCV) with nonzero prior mean

I came across this concept of GCV optimization (new to me) for tuning hyperparameters in a model, as an alternative to maximizing the maginal likelihood (MML) of the output, which is what I am used to ...
smallStackBigFlow's user avatar
0 votes
0 answers
33 views

Tuning hyperparameters after multiple runs

I wrote a classifier that uses LGBMClassifier. This is the code: ...
Zag Gol's user avatar
  • 101
1 vote
1 answer
118 views

Elbow method for tuning DBSCAN when the minimum number of points per cluster is one

The elbow method calls for setting the number of nearest neighbors (let's call it $k$) to the minimum number of points for a cluster (let's call it $m$), but what do you do when $m\leftarrow1$? Is the ...
Chris Coffee's user avatar
0 votes
0 answers
13 views

Where does cross-validation fit into a model selection workflow with inference?

Say we have some model, $f(x) = \hat{Y}$, such as linear regression, that estimates an output dependent value for some set of input data. We want to do inference on the coefficients of the model to ...
Estimate the estimators's user avatar
0 votes
1 answer
153 views

Which hyperparameters should I choose to tune for different ML models? [closed]

I'm applying hyperparameter tuning with techniques such as, randomized search and grid search in python. I could find important hyperparameters and tuned them for Random Forest Regressor algorithm, ...
Etemon's user avatar
  • 121
0 votes
0 answers
59 views

Tuning Random Forest results in max_features parameter taking a value of 1. Why?

I did a bayesian optimization tuning for parameters of random forest. With 200 iterations, it seems like 70% of the times, very low values (read 1 or 2) of max_features seems to produce better (...
MSKO's user avatar
  • 61
0 votes
0 answers
14 views

Feature Selection before Hyper-parameter Tuning and Error Estimation

I have a huge pool of features for a classification task. For error estimation of my models, I am using Nested Cross Validation (Nested CV), where I have an inner loop of CV for hyper-parameter tuning ...
Mehdi Ejtehadi's user avatar
0 votes
0 answers
50 views

Low CV-RMSE and negative $R^2$ (comparative)

I am trying to predict a numeric variable using XGBoost with optuna for hyperparameter optimization. I defined two objective functions for optuna, one optimized for very small datasets (5 to 17 ...
RaduIoan's user avatar
5 votes
1 answer
133 views

Doing empirical Bayes with improper prior - marginals that do not exist?

I am considering a Bayesian linear model for which the prior is not proper. The model is as usual $y = X \theta + w$ where $w \sim N(0, \sigma^2)$, and $\theta, \sigma^2$ are unknown. The distribution ...
smallStackBigFlow's user avatar
0 votes
0 answers
25 views

Kernel Density Estimation, Bandwidth Tuning, Independence, and Comparison

like many of us here, I turn to kernel density estimation when I need a nonparametric estimate of a numerical feature's distribution, and in an attempt to assume as little as possible, I usually use ...
user3163829's user avatar
2 votes
0 answers
153 views

How to choose the potential hyperparameters for GridSearchCV on RandomForestClassifier? Will default always be the best?

I'm fairly new to machine learning, and I know similar questions have been asked but I can't find an answer that satisfies my curiosity. I'm working on a Random Forest Classifier model in python, and ...
Sasha Halpern's user avatar
2 votes
0 answers
98 views

Why would a model combining two pre-trained models not even achieve the performance of the best sub-model?

I have two different CNNs trained on the same dataset. One performs a bit better than the other but I believe each can provide different and useful information. I use ...
raquelhortab's user avatar
1 vote
0 answers
33 views

Splitting strategy for performing hyperparameter tuning, algorithm comparison and model validation in one experiment

Let's say that for a supervised machine learning experiment I am using a fixed learning algorithm (e.g. Random Forest), and I want to achieve the following: Choose optimal hyper-parameters for the ...
saveturn's user avatar
2 votes
0 answers
82 views

A hierarchical model with conjugate hyperprior

I have a modeling problem that I am trying to formulate in a Bayesian manner to do inference. Basically, I have a prior where the variance is unknown, and we want to treat it as uncertain (though with ...
smallStackBigFlow's user avatar
1 vote
1 answer
125 views

Optimizing a threshold value on a dependent metric using a classifier trained to optimize a threshold-independent metric

Is it a reasonable approach to train a probabilities classifier by optimizing a threshold-independent metric such as AUC, and then using the trained classifier to calibrate the decision threshold ...
Amit S's user avatar
  • 57
0 votes
1 answer
1k views

How do scale_pos_weight and sample_weight interact in XGBoost?

Question: How do scale_pos_weight and sample_weight interact in XGBoost when used at the same time during training? Are they multiplied, added or something else? Example when added: Example when ...
Glue's user avatar
  • 483
1 vote
0 answers
32 views

Statistical assessment of block size for bootstrapped distribution fitting

I have a set of intensities from unordered independent events (with no date or timestamps), many of which constitute extremes, and I want to generate an extreme value distribution. The only ...
jeremy's user avatar
  • 111
0 votes
0 answers
51 views

Normal-Normal mixture model for variance hyperparameter

Consider the product of two Normal distributions (prior-hyperprior): $$ p(\sigma^2) \propto \mathcal{N}(\theta; 0, s^2 \tau^2 \sigma^2)\mathcal{N}(\sigma^2; 0, c^2 \gamma^2) $$ this is a scale mixture ...
Felipe's user avatar
  • 43
1 vote
0 answers
112 views

Time series prediction problem formatted correctly for LSTM neural networks?

I am new to machine learning, I am trying to find a way to predict voltage waveforms into the future. I have seen examples that successfully predict sinusoids or continuous voltage data based on ...
Maximiliami's user avatar
0 votes
1 answer
51 views

How should I understand the noise variance parameter(s) in multifidelity modeling using Emukit

I am learning the multi-fidelity modeling and have a question about Emukit's mixed_noise parameters, or more general, how should we determine the noise when the ...
Ann's user avatar
  • 43
4 votes
1 answer
390 views

Understanding Sequential Model-Based Optimization for machine learning

One of the hyperparameter tunning approaches that I came across recently is Sequential Model-Based Optimization (SMBO), which is a very smart approach that uses previous iterations in order to find ...
Programming Noob's user avatar
0 votes
0 answers
215 views

Walk forward cross-validation with Optuna and deepar in pytorch forecasting

I want to perform 3 splits walk forward cross validation with expanding training set for the deepar model from the pytorch forecasting framework. When I do walk forward validation, I also want to do ...
Jose_Peeterson's user avatar
1 vote
0 answers
40 views

Hyperparameter Optimization with ASHA algorithm in Sherpa library: strange number of trials

I have developped a DL model to classify images and I am trying to optimize the hyperparameters. Reading the literature, I have found the ASHA algorithm and its implementation in SHERPA, like reported ...
Jonny_92's user avatar
2 votes
0 answers
49 views

What are effective methods to maximize an unknown noisy function?

I have a function that takes a few hundred parameters and which returns a score I want to optimize for - It's a piece of software attempting to play a game against another player. The parameters ...
Borborbor's user avatar
1 vote
1 answer
18 views

Best Way to do Hyperparam Search and Cross-Validation

I'm making experiments to evaluate language models to brazilian portuguese datasets. So, i've made so each dataset is divided in 10 parts, I want to use cross-validation to determine the model's ...
Arthur Franco's user avatar
4 votes
1 answer
1k views

Is there a risk of overfitting when hyperparameter tuning a model

Is there a risk of overfitting when hyperparameter tuning a model using Optuna (or another hyperparameter tuning method ), with evaluation on a validation set and a large number of trials? While a ...
Amit S's user avatar
  • 57
0 votes
1 answer
161 views

Is repeated hyperparameter tuning can lead to overfitting?

I'm performing hyperparameter tuning for a classifier. After I finish, I'm updating the hyperparameter search space and re-tuning the hyperparameters again. I repeat this process a few times. In ...
Amit S's user avatar
  • 57
3 votes
2 answers
2k views

Is it a bad practice to learn hyperparameters from the training data set?

I am working on a project where I am evaluating different machine learning models to be used as scoring functions during in-silico docking. It is a regression problem where the 3D structure data of a ...
fairy_bluebirb's user avatar
1 vote
0 answers
50 views

Determining the number of trees in a random forest model a priori fitting

The answer to "Do we have to tune the number of trees in a random forest?" suggests using as large a number of trees in a forest as possible. Is there a rule-of-thumb for choosing this "...
fool's user avatar
  • 2,440
1 vote
0 answers
45 views

Which hyper parameter to select after nested cross validation?

I am currently doing a nested cross-validation for lasso regression to determine the best lambda value. I am using the mlr3 package with the documentation from https://mlr3book.mlr-org.com/...
nubprog's user avatar
  • 31
4 votes
0 answers
104 views

When does target encoding lead to overfitting

Let us say, we are tasked with setting (average/list) prices that are likely to convert for heterogeneous products (e.g. used cars of all shape and sizes - made up example!). Let us also say that we ...
cs0815's user avatar
  • 2,235

1
2 3 4 5
14