Linked Questions

11
votes
1answer
687 views

Which optimization algorithm to use for problems with many local optima and expensive goal function? [duplicate]

I've got an optimization problem with these properties: The goal function is not cheap to compute. It can be evaluated up to around 10^4 times in the optimization. There's a lot of local optima. High ...
1
vote
3answers
659 views

Neural Network for input values optimization [duplicate]

I have electric machine, which parameters I measure by 10 sensors. 8 of them measures "input" values and 2 of them result (output). I've got tons of historical data of all of these sensors. I built a ...
2
votes
1answer
216 views

Tuning of Hyperparameter [duplicate]

Are there any advanced packages that allows automated tuning of hyperparameters for neural network and traditional machine learning algorithms like XGBoost, random forest (using method like Bayesian, ...
0
votes
2answers
82 views

Optimization algorithm where calculating the function is very costly [duplicate]

I am doing physics simulation and have to minimize a function $f$ of approx. 10 parameters $f=f(x_1, x_2, ... , x_{10} )$. I have already looked into several optimization algorithms (downhill-simplex, ...
1
vote
2answers
50 views

Maximize slow function [duplicate]

So from experience (don't judge me, it was years ago...) I know that when a person with little knowledge about a field try to explain a problem to someone with a lot of experience it can easily lead ...
1
vote
0answers
68 views

Optimization without gradient [duplicate]

I have a non-convex function of several parameters, that takes real values. I am looking for a good algorithm to find local minima. I am reasonably confident I can come up with descent starting points....
1
vote
0answers
63 views

Is there any method for learning regularization parameter? [duplicate]

One of the hardest tasks in using machine learning methods is choosing the appropriate hyper-parameters of the model such as regularization parameter. As far as I know, this task is performed by a ...
79
votes
11answers
9k views

Why should I be Bayesian when my model is wrong?

Edits: I have added a simple example: inference of the mean of the $X_i$. I have also slightly clarified why the credible intervals not matching confidence intervals is bad. I, a fairly devout ...
51
votes
6answers
25k views

Practical hyperparameter optimization: Random vs. grid search

I'm currently going through Bengio's and Bergsta's Random Search for Hyper-Parameter Optimization [1] where the authors claim random search is more efficient than grid search in achieving ...
24
votes
4answers
9k views

When are genetic algorithms a good choice for optimization?

Genetic algorithms are one form of optimization method. Often stochastic gradient descent and its derivatives are the best choice for function optimization, but genetic algorithms are still sometimes ...
21
votes
1answer
12k views

How could stochastic gradient descent save time comparing to standard gradient descent?

Standard Gradient Descent would compute gradient for the entire training dataset. ...
14
votes
3answers
7k views

How to run linear regression in a parallel/distributed way for big data setting?

I am working on a very large linear regression problem, with data size so large that they have to be stored on a cluster of machines. It will be way too big to aggregate all the samples into one ...
11
votes
3answers
538 views

Optimization of stochastic computer models

This is a tough topic for myself to google since having the words optimization and stochastic in a search almost automatically defaults to searches for stochastic optimization. But what I really want ...
1
vote
4answers
14k views

SVM Hyperparameters Tuning

I am using SVM classifier to classify data, My dataset consist of about 1 milion samples, Currently im in the stage of tunning the machine , Try to find the best parameters including a suitable ...
10
votes
1answer
2k views

What are some of the disavantage of bayesian hyper parameter optimization?

I am fairly new to machine learning and statistics but I was wondering why bayesian optimization is not referred more often online when learning machine learning to optimize your algorithm ...

15 30 50 per page