# Tag Info

1 vote
Accepted

### 1D cluster - Jenks optimization - Finding optimal number

This seems to be a two stage problem: first, identify the number of clusters and then, secondly, optimally perform the clustering. For the first part, I'd suggest Cluster Validation by Prediction ...
• 4,599
Accepted

### Should folds in k-fold CV actually be representative?

This is a reasonable question but it might be conflating 2 things. Should $k$ be chosen to make [something] representative? Should the-way-we-assign-data-to-each-fold be chosen to make [something] ...
• 1,439
1 vote

### Why do we use Acquisition Functions?

The purpose of Bayesian optimization is to find global minima of functions that have many local minima. More typical optimizers are "local," in the sense that they follow some procedure ...
• 78.2k

### Equality-constrained least-squares when the matrix is singular

If it is possible to obtain a unique solution to $Ax=b$ subject to the constraints $Cx =d,$ then you can obtain it with Lagrange multipliers. To be explicit, suppose $A$ is an $n\times p$ matrix, $x$ ...
• 287k
1 vote

### Finding the global minimum of a nonnegative, black-box, polynomial spline

It's not entirely clear to me what you do and don't know in your optimisation, but if I understand correctly, you don't even know how many piecewise parts there are to your function. If the number of ...
• 94.4k
1 vote
Accepted

### Support vector machine, complementary slackness and marginal hyperplane

Theoretically, no. You can see this in terms of smoothly adjusting $C$ - at some point $x_i$ may be on the margin but with $\alpha_i = 0$. Practically, yes. In particular, note that stopping ...
• 1,259

### How to handle weighted examples in stochastic gradient descent (with mini-batches)?

Gradient descent is about following the derivatives (gradients). Recall that the derivative rule for calculating the derivative of a function $f$ times a constant $c$ is just  \frac{\partial}{\...
• 113k
1 vote
Accepted

### How to perform fine balance matching in R for several covariates?

rcbalance() doesn't actually produce fine balance on the included covariates; it produces "refined" balance, which is slightly different in that it ...
• 21.8k
1 vote
Accepted

### Why use transpose of nabla in gradient descent

Fundamentally this is just a notational issue. Assuming $x_k$ and $d_k$ are multidimensional "vectors" (say $n$ dimensional), the standard is often that all vectors are "columns", ...

### Why use transpose of nabla in gradient descent

Note that if f is scalar, you need an inner product to make a scalar on the right hand side. Generally nabla will be a vector
• 5,176

### Simulated Annealing Parameter Tuning

Just looking through the code: SA function is always called with the same value of temperature, $T=8$. The statement ...
• 1,580
Accepted

### Combinations from different sets with weightings

This can be expressed as a binary linear program. Provided the total number of unique combinations is not prohibitively large, it can be solved efficiently. When multiple solutions exist, usually ...
• 287k

### Why do we use Acquisition Functions?

In uncertain scenarios, where Bayesian optimization is employed, we are facing the exploration-exploitation tradeoff. The acquisition function is a solution for it. As you noticed, Bayesian ...
• 113k

### Examples in the Real World where Evolutionary Algorithms/Genetic Algorithms Outperform other Classes of Optimization Algorithms

It's still an open field of research, but there are many examples. Hyperparameter tuning is an example of a noisy and expensive function that can be handled by evolutionary algorithms. Neural ...
• 113k