16 views

### Deriving cost function [duplicate]

I have calculated data from a computer simulation for a wireless network as shown below, here P is an algorithm tuning parameter, 'C' and 'D' are % decrease in collisions and % decrease in delay ...
21 views

### High dimensional hyperparameter tune

Many already known optimization techniques rely on past data (Bayesian optimization for instance) and perform really well for a bunch of hyperparameters. Is there, however, a good tuner/tuning method ...
27 views

### How do you calculate the probability that a given number of ML model hyper-parameter combinations will include the optimum combination?

Summary I would like to ask the statistics community a question I took from my data-science query. It is about randomized hyper-parameter searching. Although I use SKLearn's ...
53 views

### Least squares optimization with expensive model and many parameters

I have a physical model which takes $\sim50$ parameters and gives $\sim 2000$ outputs taking tens of minutes to run. I need to optimise these parameters to give outputs as close as possible to data by ...
35 views

### Can neural networks learn many to one function?

I am thinking of building a secondary Neural Net to train the relation between the hyper-parameters and test set accuracy of my primary Neural Net, so as to maximize it efficiently. And there are 2 ...
27 views

### What is the most efficient algorithm for hyper-parameter optimization, specially when comparing Bayesian optimisation and evolutionary algorithms?

I've been reading some papers about how Bayesian optimisation has achieved great results in hyper-parameter tuning when compared with random search or grid search. What about when comparing with ...
29 views

### Using normal distribution to do rough hyperparameter tuning

I wanted to ask if this is a valid way of doing hyperparameter tuning. I have 7 parameter for my model. Since I have too many parameters to do a grid search, I was going to try a different method: Do ...
97 views

### MLE over a large number of parameters

I am working on a model that I am fitting to a medium-sized dataset ($\approx 300,000$ observations). The likelihood I have derived is a mixture model of some Beta distributions which also has the ...
24 views

### Grid search extrapolation

Hyperparameter optimization via grid search returns a value of a chosen metric for each set of hyperparameters in the grid. Would it make sense to fit the values of the metric (target variable) using ...
74 views

### What is Gaussian in BO: Can I use bayesian optimization for binary classification task?

BO works based on gaussian process, so what is gaussian there? Can I use BO for binary classification task, for example neural network which has output with logistic activation and is minimizing ...
49 views

### What kind of optimization am I using?

I would like to know under optimization taxonomy, where do the following method fall? I have a combinatarial optimization problem, that I am trying to solve, Maximize an objective value for the ...
203 views

### Are there are alternatives to gradient update rule?

Most optimization techniques (that I'm aware of) for non-linear cost functions that are commonly implemented rely on linearly updating a variable iteratively until a minimum is reached or a condition ...
20 views

### Optimizing a single parameter. Best practices?

I've been recently working with an optimization problem, where I have to optimize a single parameter. I cannot test all parameter values, as the process is too expensive. My function(s) look like ...
I have an optimization problem: Given a complex $n\times n$ covariance matrix $C$ one must find a complex $n$-vector $v_C^\ast$ which (approximately) minimizes an objective $f_C(v)$ over all space. \$...