Linked Questions

0
votes
0answers
16 views

Deriving cost function [duplicate]

I have calculated data from a computer simulation for a wireless network as shown below, here P is an algorithm tuning parameter, 'C' and 'D' are % decrease in collisions and % decrease in delay ...
2
votes
0answers
21 views

High dimensional hyperparameter tune

Many already known optimization techniques rely on past data (Bayesian optimization for instance) and perform really well for a bunch of hyperparameters. Is there, however, a good tuner/tuning method ...
0
votes
0answers
27 views

How do you calculate the probability that a given number of ML model hyper-parameter combinations will include the optimum combination?

Summary I would like to ask the statistics community a question I took from my data-science query. It is about randomized hyper-parameter searching. Although I use SKLearn's ...
1
vote
1answer
53 views

Least squares optimization with expensive model and many parameters

I have a physical model which takes $\sim50$ parameters and gives $\sim 2000$ outputs taking tens of minutes to run. I need to optimise these parameters to give outputs as close as possible to data by ...
0
votes
0answers
35 views

Can neural networks learn many to one function?

I am thinking of building a secondary Neural Net to train the relation between the hyper-parameters and test set accuracy of my primary Neural Net, so as to maximize it efficiently. And there are 2 ...
0
votes
0answers
27 views

What is the most efficient algorithm for hyper-parameter optimization, specially when comparing Bayesian optimisation and evolutionary algorithms?

I've been reading some papers about how Bayesian optimisation has achieved great results in hyper-parameter tuning when compared with random search or grid search. What about when comparing with ...
0
votes
0answers
29 views

Using normal distribution to do rough hyperparameter tuning

I wanted to ask if this is a valid way of doing hyperparameter tuning. I have 7 parameter for my model. Since I have too many parameters to do a grid search, I was going to try a different method: Do ...
0
votes
2answers
97 views

MLE over a large number of parameters

I am working on a model that I am fitting to a medium-sized dataset ($\approx 300,000$ observations). The likelihood I have derived is a mixture model of some Beta distributions which also has the ...
1
vote
1answer
24 views

Grid search extrapolation

Hyperparameter optimization via grid search returns a value of a chosen metric for each set of hyperparameters in the grid. Would it make sense to fit the values of the metric (target variable) using ...
3
votes
1answer
74 views

What is Gaussian in BO: Can I use bayesian optimization for binary classification task?

BO works based on gaussian process, so what is gaussian there? Can I use BO for binary classification task, for example neural network which has output with logistic activation and is minimizing ...
1
vote
2answers
49 views

What kind of optimization am I using?

I would like to know under optimization taxonomy, where do the following method fall? I have a combinatarial optimization problem, that I am trying to solve, Maximize an objective value for the ...
1
vote
1answer
203 views

Are there are alternatives to gradient update rule?

Most optimization techniques (that I'm aware of) for non-linear cost functions that are commonly implemented rely on linearly updating a variable iteratively until a minimum is reached or a condition ...
0
votes
0answers
20 views

Optimizing a single parameter. Best practices?

I've been recently working with an optimization problem, where I have to optimize a single parameter. I cannot test all parameter values, as the process is too expensive. My function(s) look like ...
1
vote
3answers
750 views

Neural Network for input values optimization [duplicate]

I have electric machine, which parameters I measure by 10 sensors. 8 of them measures "input" values and 2 of them result (output). I've got tons of historical data of all of these sensors. I built a ...
1
vote
0answers
85 views

Using a surrogate model for the solution space of an optimization problem

I have an optimization problem: Given a complex $n\times n$ covariance matrix $C$ one must find a complex $n$-vector $v_C^\ast$ which (approximately) minimizes an objective $f_C(v)$ over all space. $...

15 30 50 per page