Questions tagged [hyperparameter]

A parameter that is not strictly for the statistical model (or data generating process), but a parameter for the statistical method. It could be a parameter for: a family of prior distributions, smoothing, a penalty in regularization methods, or an optimization algorithm.

Filter by
Sorted by
Tagged with
0
votes
0answers
14 views

Should I worry about in-fit overfitting if out-of-fit accuracy is maximized?

I typically find that random forest always overfits to some degree on the training data. That is, the in-fit R2 is typically substantially higher than the out-of-fit, cross-validated R2. In general, ...
3
votes
1answer
46 views

What is better: Cross validation or a validation set for hyperparameter optimization?

For hyperparameter optimization I see two approaches: Splitting the dataset into train, validation and test, and optimize the hyperparameters based on the results of training on the train dataset and ...
1
vote
1answer
16 views

Correct way (if any!) to apply preprocessing to hold out dataset

After cross validation and grid search the below are the desired pipeline steps and hyper-params for my model. ...
0
votes
0answers
18 views

How do you calculate the probability that a given number of ML model hyper-parameter combinations will include the optimum combination?

Summary I would like to ask the statistics community a question I took from my data-science query. It is about randomized hyper-parameter searching. Although I use SKLearn's ...
0
votes
0answers
10 views

How to tune specific hyperparameters in H2O AutoML

Is it possible to say to the H2O AutoML to tune an additional hyperparameter (eg. the class_balancing param that has to be specified)? I would like that the AutoML ...
-1
votes
0answers
15 views

Is it possible to use multithreading for hyperparameter tuning with keras? [closed]

Since hyperparameter tuning seems to consist in training different models for the same task, I suppose it is a good idea to train them in parallel in order to gain some time. However, my attempt was ...
1
vote
1answer
26 views

Does Gaussian Naive Bayes have paramter to be tuned

I am trying to implement the Gaussian Naive Bayes from a scikit-learn library. I know that the Naive Bayes is based on the Bayes' theorem which is defined in high level as: ...
4
votes
1answer
78 views

Does using Cross-Validation give you the green light to do exhaustive hyper-parameter searches?

By hyper-parameters I mean not only the machine learning algorithm hyper-parameters (learning rate, etc.), but also hyper-parameters like "what's the ideal number of data points to use" or &...
1
vote
1answer
62 views

Is random seed a hyper-parameter to tune in training deep neural network?

For building deep neural networks, there are a lot of random components in each training. On one hand, I feel it is uncanny to "tune" random seed. But in my experience, some random seed just ...
0
votes
0answers
4 views

xgboost hyperparameter optimization and actual performance

I'm splitting my data into train-test and then on the training set I perform a hyper parameter optimization using TPE and then select the best parameters according to my metric of choice. The mean ...
0
votes
0answers
29 views

Can neural networks learn many to one function?

I am thinking of building a secondary Neural Net to train the relation between the hyper-parameters and test set accuracy of my primary Neural Net, so as to maximize it efficiently. And there are 2 ...
0
votes
0answers
22 views

What is the most efficient algorithm for hyper-parameter optimization, specially when comparing Bayesian optimisation and evolutionary algorithms?

I've been reading some papers about how Bayesian optimisation has achieved great results in hyper-parameter tuning when compared with random search or grid search. What about when comparing with ...
0
votes
0answers
23 views

Using normal distribution to do rough hyperparameter tuning

I wanted to ask if this is a valid way of doing hyperparameter tuning. I have 7 parameter for my model. Since I have too many parameters to do a grid search, I was going to try a different method: Do ...
0
votes
0answers
8 views

using “recall” as a metric when doing hyper parameter tuning in python

I have been using Azure ML studio for my prediction tasks. since recall was a more important metric for my project, I could set that in Azure as the metric which should be optimized while training the ...
2
votes
2answers
37 views

Tuning hyperparameters never affects weights?

I am trying to better understand “tuning the hyperparameters”. I understand how to use GridSearchCV, I found the below explanation useful: “As we do not know whether those parameters affect each other,...
1
vote
0answers
36 views

Standard errors of parameters in hyperparameter tuning

I have a model with the parameters $\textbf{θ} = (\textbf{φ}, ψ)$, which I trying to estimate. There is an out-of-the-box solution for ML estimating $\textbf{φ}$ if $ψ$ is fixed. If I treat $ψ$ as ...
0
votes
0answers
7 views

Hyperparameter optimization sampling from log space vs specific random distribution

Why sampling in the log space when we can sample from a specific distribution, for example, why don't we sample the hyperparameters from an exponential distribution? In 5:30 Andrew Ng Lecture he is ...
1
vote
1answer
36 views

How to choose a non-informative or weakly informative hyper priors for my hierarchical bayesian model?

I am learning Bayes on "Applied Bayesian Statistics" by MK Cowles. The chapter about "Bayesian Hierarchical Models" mentioned an example that we estimate a softball player’s ...
0
votes
1answer
8 views

Reducing training dataset for architecture testing

I want to quickly test many CNN architectures on the MNIST dataset, but I don't want to train each on the whole dataset. Is it possible to reduce the training dataset size and still be able to ...
3
votes
2answers
59 views

Why is the inverse chi-squared distribution a natural prior and posterior for an unknown variance of a normal distribution?

Wikipedia says [the inverse-chi-squared distribution] arises in Bayesian inference, where it can be used as the prior and posterior distribution for an unknown variance of the normal distribution. ...
7
votes
1answer
210 views

Cross Validation: Averaging across estimates vs re-estimating on full sample

Suppose you perform cross-validation to obtain an optimal value for some vector of hyperparameters $\lambda$. You ultimately want to predict some new observations $y_\mathrm{query}|X_\mathrm{query}$. ...
0
votes
0answers
15 views

When performing a svm hyper-parameter search for epsilon coef0 and degree which min max and increment values are suggested?

I am using libsvm, but I think this applies to any ML algorithm where these kernels are used. The default implementation of libsvm suggests values for the linear kernel or RBF kernel hyper-parameters ...
0
votes
0answers
23 views

Deriving Hyperparameter updates in Online Interactive Collaborative Filtering

I've been going through "Online Interactive Collaborative Filtering Using Multi-Armed Bandit with Dependent Arms" by Wang et al. and am unable to understand how the update equations for the ...
0
votes
1answer
35 views

SVR optimal hyperparameters are Epsilon = 0, Cost = inf?

I'm running an rbf-kernel SVR with GridSearchCV. I'm optimizing epsilon, cost and gamma. In my hyperparameter gridsearch, the optimal parameters appear "unbounded". Specifically, any epsilon under 1 ...
0
votes
0answers
7 views

Grouped TrainValidationSplit in spark

When performing hyperparameter tuning in Spark, TrainValidationSplit() can be used to evaluate the validation score of the models in order to choose the best hyperparameters. A common situation is to ...
1
vote
1answer
43 views

When to use low discount factor in reinforcement learning?

In reinforcement learning, we're trying to maximize long-term rewards weighted by a discount factor $\gamma$: $ \sum_{t=0}^\infty \gamma^t r_t $. $\gamma$ is in the range $[0,1]$, where $\gamma=1$ ...
0
votes
1answer
29 views

Model selection using nested cross validation

I am working on a school project using remote sensing data, for classification purposes. And I am trying to select the best model (models) for my data. The approach that I adopted is the following: ...
1
vote
1answer
22 views

Grid search extrapolation

Hyperparameter optimization via grid search returns a value of a chosen metric for each set of hyperparameters in the grid. Would it make sense to fit the values of the metric (target variable) using ...
0
votes
0answers
22 views

What are the derivatives of Squared Exponential kernel function w.r.t. characteristic length scale (Gauss Process)

I'm writing a matlab code to implement Gaussian process. In the book: Gaussian Process for machine learning by Carl Edward Rasmussen and Christopher K. I. Williams, the authors define the squared ...
0
votes
0answers
13 views

Best score in SVM

i am new to machine learning and i took the house price dataset from kaggle.com to learn and understand SVM. for regression the best score would be 0.0 and for classification the best score ...
0
votes
1answer
17 views

Are the vectorization settings considered hyperparameters in ML?

Short definition of HP: "In machine learning, a hyperparameter is a parameter whose value is set before the learning process begins. Hyperparameter optimization or tuning is the problem of ...
1
vote
1answer
81 views

Picking lambda for LASSO

Preface: I am aware of this post: Why is lambda "within one standard error from the minimum" is a recommended value for lambda in an elastic net regression? (It is generally recommended to ...
2
votes
1answer
22 views

Is the validation data set used for building the testing model?

Let's assume I split my data into 70% training data, 20% validation data, and 10% testing data. For each hyperparameter I am building a model using the training data and determine the best ...
0
votes
0answers
23 views

Methods to optimize hyperparameters of an ARD kernel for Gaussian Process Regression

I am using Gaussian Process regression to build a model from my feature set, which consists of 40 parameters and ~250 samples in my training set. I've chosen an RBF kernel with ARD (different length ...
2
votes
1answer
20 views

Scale selection for the Hyperparameters

During the scale selection for hyperparameters I have a question generally if we have to select a random number in the interval $[2,5]$ It is fairly straight forward that we select either $2,3,4,5$ ...
2
votes
1answer
20 views

Nested Cross Validation: How to do the whole Shebang (Algorithmic Selection, Model Selection, Parameter Tuning, Preprocessing) [closed]

First post! If you don't want to read the background you can skip to the Problem heading below. Background Hello everyone, I'm a Physics student doing physics education research. My professor wants ...
0
votes
0answers
14 views

Query regarding Hyper parameters tuning in deeplearning

I have a question "Using an appropriate scale to pick hyperparameters", under the section "Hyperparameter tuning". The reason mentioned for choosing a log scale over a linear scale for the hyper ...
1
vote
2answers
44 views

Randomly sampling parameters for model selection

Suppose I'm fitting a complicated (e.g. neural network) model's parameters $\theta$ to some data $D$, and I'm trying to tune hyperparameters (e.g. number of layers, size of layers) $\eta$. Normally I ...
1
vote
2answers
33 views

Benefits of random search over other optimization methods in Neural Network hyperparameter tuning

When reading about methods of hyperparameter tuning of neural networks, I have mainly come across grid search and random search in textbooks and articles online. I was wondering why other optimization ...
0
votes
1answer
26 views

Perceptual Loss Layers Selection

I understand that in order to improve your generative model performance it is quite useful to compare your output and the target in the feature space, as stated in the paper Perceptual Losses for Real-...
1
vote
2answers
140 views

How to choose delta parameter in Huber Loss function?

In Huber loss function, there is a hyperparameter (delta) to switch two error function. Currently, I am setting that value manually. But, I cannot decide which values are the best. So, how to choose ...
1
vote
2answers
33 views

Why and when do we need to tune hyperparameters?

This might come as a basic question. But I need to understand why do we need to tune the hyper parameters in a machine learning model instead of going into a different model altogether. Or to put it ...
0
votes
1answer
41 views

What is the mathematical definition of the 'Elbow Method'?

In K-means algorithm, it is recommender to pick the optimal K, according to the Elbow Method. However all the tutorials explain the elbow method in these 4 steps: Run K-means for a range of K's ...
0
votes
0answers
9 views

Fourier transforms for noise reduction

Given a signal, which is regularly sampled over time and is noisy. The standard method is with a Fourier transform to reduce the noise and minimise the change to the signal. ...
1
vote
1answer
19 views

Identify suitable scoring metric for food prediction

I am using GridSearchCV to find the best parameter that help me tune XGBoost for a food prediction algorithm. I am struggling to identify the best scoring metric that would result in the best profit (...
0
votes
0answers
13 views

Is sampling the training set for hyperparameter optimisation a good speed up solution?

I am using Bayesian hyperparameter optimization for LSTM hyperparameters. This manages to find an optimum set of hyperparameters in much fewer iterations than a grid search. My problem is that I ...
1
vote
2answers
134 views

What is the “tree” structure in Tree Parzen Estimators?

Context In Algorithms for Hyper-Parameter Optimization, the authors propose a "tree-structured" configuration space. Here, a configuration space is a space of hyperparameters. Questions What ...
1
vote
1answer
39 views

How to decouple weight decay strength and model size?

Consider the neural networks' loss function with the cross entropy term and the $L^2$ weight decay term, which are usually written as: $$E = \frac{1}{N_{samples}} \sum_{i=1}^{N_{samples}} \text{...
0
votes
1answer
38 views

how to do the hyper parameter tunning for one class svm in r programming?

x is input (single column) tuned <- tune.svm(x=x, y =NULL, data=x, type= 'one-classification', tunecontrol = tune.control(sampling = "fix")) For this I am ...
0
votes
1answer
115 views

Gamma distribution and hyperparameters

The formula for mean and variance of a gamma distribution is given by a/b and a/b^2 (hyperparameters) respectively.Are they estimates of the posterior gamma distribution? Can prior, likelihood and ...

1
2 3 4 5
10