We’re rewarding the question askers & reputations are being recalculated! Read more.
Search type Search syntax
Tags [tag]
Exact "words here"
Author user:1234
user:me (yours)
Score score:3 (3+)
score:0 (none)
Answers answers:3 (3+)
answers:0 (none)
isaccepted:yes
hasaccepted:no
inquestion:1234
Views views:250
Sections title:apples
body:"apples oranges"
URL url:"*.example.com"
Favorites infavorites:mine
infavorites:1234
Status closed:yes
duplicate:no
migrated:no
wiki:no
Types is:question
is:answer
Exclude -[tag]
-apples
For more details on advanced search visit our help page
Results tagged with Search options user 11852

Use this tag for any use of optimization within statistics.

5
votes
The basic idea behind simulated annealing optimization is that it is a random search. Practically you draw samples from a distribution describing your possible solutions and based on certain … ", and result into more rigid structures, ie. more efficient crystallization. Now, in the case of an optimization problem the surface you try to "fill", or more accurately explore, is (in a 2D case) the …
answered Jun 23 '13 by usεr11852 says Reinstate Monic
1
vote
The reason to normalise our variables before their use in a technical analysis is to ensure that their importance appears similar in our context-agnostic procedure. An optimisation procedure does not …
answered Jan 20 by usεr11852 says Reinstate Monic
5
votes
, this is the reason that L-BFGS-B occasionally diverges in the optimization of the $L_2$ regularised regression conducted while the gradient descent always converges. A very low $\alpha$ guarantees …
answered Jun 12 '16 by usεr11852 says Reinstate Monic
14
votes
numericals out) and that your optimization is unconstrained. Let me stress, you are a bit unluckily cause between 25-30 and 100 variables it is a bit of twilight zone when it comes to choosing between large … or small scale optimization routines. Having said that though, nothing is lost. Given that even first order derivative are expensive to get that kind off kills the idea of Newton's method. You might …
answered Apr 5 '13 by usεr11852 says Reinstate Monic
1
vote
A strong and relatively straightforward routine to use for box-constraints is BOBYQA; it is available in R through the minqa package. It is the default optimiser for the lme4 package when it comes to …
answered Aug 29 '16 by usεr11852 says Reinstate Monic
7
votes
and Sahinidis, 2013 review paper: "Derivative-free optimization: A review of algorithms and comparison of software implementations" in the Journal of Global Optimisation seems to be your best bet for … something exhaustive. Within R the minqa package that provide derivative-free optimization by quadratic approximation (QA) routines. The package contains some of Powell's most famous "optimisation …
answered Mar 24 '16 by usεr11852 says Reinstate Monic
2
votes
Three suggestions: Would you consider looking CERES solver by Google? Being somewhat naive on this, would you consider using a Stochastic Gradient descent, possibly with some form of regularization …
answered Oct 14 '12 by usεr11852 says Reinstate Monic
1
vote
Yes, it is reasonable to have this boosting effect. This effect means that (to steal the words of C. Dyer) the learning algorithm "'pays attention' to rare but informative features". It is the direc …
answered Oct 23 '15 by usεr11852 says Reinstate Monic
4
votes
Given you have a fixed number $N$ of data (ie. you have equal number of reading on both dimensions) you could immediately use: The square-root rule rounded-down ($\sqrt{N}$), (ie. the Excel-way :) ) …
answered Sep 6 '14 by usεr11852 says Reinstate Monic
1
vote
Hessians (eg. Newton's method) before touching more advanced concepts like constrained optimisation, combinatorial optimization, etc. Evolutionary algorithms are primarily (meta-)heuristics approaches …
answered Sep 24 '18 by usεr11852 says Reinstate Monic
8
votes
Simulated Annealing (SA) is a very simple algorithm in comparison with Bayesian Optimization (BO). Neither method assumes convexity of the cost function and neither method relays heavily on gradient … where you landed before, you don't know where you will land next. It is a typical Markov Chain approach. You do not model any strong assumptions about the underlaying solution surface. MCMC optimization
answered Feb 19 '16 by usεr11852 says Reinstate Monic
2
votes
This might come as a slight anti-climax but essentially prior to fitting we remove any dependent columns of $X$ (or $XW$ in the case of a weighted task) so that $X$ is of full rank. For each IRLS iter …
answered Aug 7 '17 by usεr11852 says Reinstate Monic
14
votes
$\|x\|_2$ is the Euclidean norm of the vector $x$; $\|x\|_2^2$ is the squared Euclidean norm of $x$. Note that as the Euclidean norm is probably the mostly commonly used norm people routinely abbrevia …
answered Dec 15 '15 by usεr11852 says Reinstate Monic
3
votes
It is perfectly fine to do so. One might argue that optimising certain structural hyperparameters like the cost function (e.g. Gini index vs Entropy) is an extreme case of cherry-picking - we pick a …
2
votes
only gradient information you might as well try to utilize it to the max. Check how a Line Search works and what Wolfe Conditions are; step-selection (in simple cases) is not that hard to include in an optimization algorithm. :) …
answered Apr 11 '13 by usεr11852 says Reinstate Monic

15 30 50 per page