Linked Questions

80
votes
11answers
9k views

Why should I be Bayesian when my model is wrong?

Edits: I have added a simple example: inference of the mean of the $X_i$. I have also slightly clarified why the credible intervals not matching confidence intervals is bad. I, a fairly devout ...
51
votes
6answers
26k views

Practical hyperparameter optimization: Random vs. grid search

I'm currently going through Bengio's and Bergsta's Random Search for Hyper-Parameter Optimization [1] where the authors claim random search is more efficient than grid search in achieving ...
24
votes
4answers
9k views

When are genetic algorithms a good choice for optimization?

Genetic algorithms are one form of optimization method. Often stochastic gradient descent and its derivatives are the best choice for function optimization, but genetic algorithms are still sometimes ...
23
votes
2answers
12k views

How could stochastic gradient descent save time compared to standard gradient descent?

Standard Gradient Descent would compute gradient for the entire training dataset. ...
15
votes
3answers
7k views

How to run linear regression in a parallel/distributed way for big data setting?

I am working on a very large linear regression problem, with data size so large that they have to be stored on a cluster of machines. It will be way too big to aggregate all the samples into one ...
11
votes
3answers
548 views

Optimization of stochastic computer models

This is a tough topic for myself to google since having the words optimization and stochastic in a search almost automatically defaults to searches for stochastic optimization. But what I really want ...
11
votes
1answer
704 views

Which optimization algorithm to use for problems with many local optima and expensive goal function? [duplicate]

I've got an optimization problem with these properties: The goal function is not cheap to compute. It can be evaluated up to around 10^4 times in the optimization. There's a lot of local optima. High ...
11
votes
2answers
5k views

Markov chain Monte Carlo (MCMC) for Maximum Likelihood Estimation (MLE)

I am reading a 1991 conference paper by Geyer which is linked below. In it he seems to elude to a method that can use MCMC for MLE parameter estimation This excites me since, I have coded BFGS ...
10
votes
3answers
347 views

Best way to evaluate PDF estimation methods

I wish to test some of my ideas that I think are better than anything that I have seen. I could be wrong but I'd like to test my ideas and vanquish my doubts by more certain observations. What I have ...
10
votes
1answer
3k views

What are some of the disavantage of bayesian hyper parameter optimization?

I am fairly new to machine learning and statistics but I was wondering why bayesian optimization is not referred more often online when learning machine learning to optimize your algorithm ...
5
votes
1answer
9k views

hyperparameter tuning in neural networks

I was trying to fine tune a neural network model for a multilabel classification problem. I was reading Jason Brownlee 's article for the same. As per the article, there are a number of parameters to ...
5
votes
1answer
2k views

Optimizing a “black box” function: Linear Regression or Bayesian Optimization… what's the difference?

Goal: I have a function $f(x,y)=z$ (two variables for illustration only) which I know almost nothing about--it has a compact domain which I can determine, it is non-negative, and bounded above. My ...
5
votes
2answers
614 views

Regression: find the best degree of polynomial with the best regularization parameter

When trying to predict data using linear regression or classify with logistic regression, with a polynomial, I know how to find the best degree of a polynomial to fits given data when the ...
4
votes
1answer
160 views

What optimization (maximization/minimization) methods exist contours with lots of kinks?

I am wondering what methods exist out there for optimization on problems where your contour has lots of small sharp kinks. For example, suppose we have a contour where there are lots of lots of spiky, ...
3
votes
1answer
749 views

Bayesian Hyperparameter Tuning

I would like to implement Bayesian hyperparameter tuning in Python, but I am struggling to find a complete explanation of how it is exactly working. As anyone links, books or papers I could read about ...

15 30 50 per page