In statistics this refers to selecting an estimator of a parameter by maximizing or minimizing some function of the data. One very common example is choosing an estimator which maximizes the joint density (or mass function) of the observed data referred to as Maximum Likelihood Estimation (MLE).

learn more… | top users | synonyms (2)

1
vote
1answer
42 views

How can I implement lasso in R using optim function

As you know lasso is a popular variable selection method of the form of $ (y-x\beta)'(y-X\beta)+\lambda \sum_i|\beta_i| $ the first is that it is possible to use optim() function in R to minimize ...
1
vote
1answer
40 views

Solving a scrambled image puzzle with a genetic algorithm

I want to solve a puzzle such as this one: by using a genetic algorithm. When the number of pieces grow, and maybe some are rotated, the number of combinations become overwhelming. I am hoping that ...
0
votes
0answers
11 views

Intuitive explanation of the proximal operator

Can someone please give an intuitive explanation of the proximal operator? I understand that it is somehow related or is a generalization of projection, but other than that, I'm unable to grasp it.
0
votes
1answer
20 views

Data point that minimizes all four parameters [duplicate]

First of all, I must say I'm in no way a data scientist or anything; I just happened to come across a problem, and I am attempting to use statistics to find the best solution. The problem is about ...
0
votes
1answer
25 views

How to solve nonlinear optimization problem in R

Suppose I have a set of data and reason to believe the following relation holds y ~ a0 + a1*x1 + a2*x2 + a3*log(x3) How can I use R to solve for the coefficients {a0, ... a3}, supposing I want to ...
0
votes
0answers
8 views

Meta-parameter search for elastic net regularization of general objective function

In their 2004 paper on elastic net regularization, Zou and Hastie present an efficient method for finding the meta-parameters by folding the $L_2$-regularization component into the OLS problem and ...
0
votes
0answers
41 views

stochastic network optimization

I'd like to optimize the flow of materials through a network. There are vertices (i.e. physical locations) and edges (i.e. links between the physical locations). Inputs: locations transactional ...
0
votes
1answer
34 views
+50

Calibrating Generalized Hyperbolic distribution in R - which parameters are valid and allow for a numerical calculation of absolute moments

I am using the R-package ghyp in order to calibrate and model. In fact my coding is based on this paper. I know that I could do quite a robust fit using ...
0
votes
0answers
13 views

Why does Co-ordinate descent work? [closed]

If it works does it mean that "function is convex if it's convex in any direction"?
0
votes
0answers
40 views

Estimating correlation hyperparameters of a Gaussian Process

I have an actual function that I need to simulate using a GP model. I've not done this before so I'm unclear of the steps. I have used the true function at different values of the inputs ($\vec X1, ...
1
vote
0answers
48 views

help with constrained optimization setup in R

I have a function which parameters I wish to optimize. As a simplified example, the function looks something like this: ...
1
vote
0answers
40 views

Finding a global minimum of non-convex quasi-smooth function that is costly to evaluate

I have a bounded non-convex function in 10-dimensional space. The function is quasi-smooth, you can imagine a histogram, here is an illustration, it just shows the idea and not related to my ...
0
votes
1answer
21 views

Multimodal objective optimization

What is the meaning of multimodal objective optimization? Could you provide an example? What is the difference when compared with multi-objective optimization, which would provide solutions along ...
1
vote
0answers
10 views

Basis for a certain class of iterative algorithms [closed]

I have seen the following algorithm trick in a few places. Suppose that you have some closed form equation that you would like to solve, of the form $A=F(A)$, where $F$ may be rather complicated and ...
0
votes
1answer
25 views

find the optimal correspondence matrix

I have two sets of points and I find with different methods, different correspondence matrices which shows which point in one set correspond to other point in the other set. How could I find the ...
1
vote
0answers
15 views

Multiple objective allocation function

I have an allocation problem where, for a given good, I have buyer $i$ willing to buy up to quantity $b_{s,i}$ and seller $j$ willing to sell up to quantity $s_{s,i}$. There are $N$ buyers and $M$ ...
0
votes
0answers
52 views

Newton Raphson Over-estimates Parameters

I have implemented an almost plain vanilla algorithm to find the MLE estimates of 3 parameters in a log-likelihood function (in R.) When I test my algorithm with some simulated data it does pretty ...
1
vote
0answers
118 views

Solve systems of non-linear equations

How could try to solve this system of linear equations. I made a code in R trying to solve it, that might make me suggestions please. I want to estimate $\lambda$, $\beta$ and $\alpha$. ...
0
votes
0answers
33 views

Optimally distributing sports tickets to customers

I work for a business that has a certain number of sports tickets to distribute to customers and potential customers every year, and I've been asked to develop a model that will give insight into ...
1
vote
1answer
34 views

Recommended packages for numerical optimization with symbolic calculus

I'd like to train a model $\widehat{y}_i = F(x_i, \theta)$, by minimizing the sum of a loss function, $L(\widehat{y}_i, y_i, \theta)$. I'd like to input $\{x_i, y_i\}, F, L$ into a software package ...
0
votes
0answers
12 views

What should be the fitness function while using Particle Swarm optimisation

I am using Particle Swarm Optimisation for optimising the parameters of a Neural network (for multi-class classification problem). But what should be the fitness function for it ? I have tried ...
2
votes
1answer
45 views

How to find all optima in an optimization problem?

I have an optimization problem where several optima can exist at different input values, and I need to find as many as possible. As an example consider the cross-in-tray function, which has four ...
0
votes
0answers
16 views

Discrete optimization with a very large solution neighborhood to explore

I have a problem whose feasible (discrete) solutions can measured by a cost function. I am thinking of using some optimization technique to get better solutions from a rough initial approximation. I ...
1
vote
1answer
28 views

How to visualize a set of many optimizations of posterior simulations of an objective function?

I started by fitting a model: $y = f(X) + \epsilon$. The model includes random effects and coefficients -- there is a lot of heterogeneity in the population (and the data is longitudinal). I then ...
0
votes
0answers
13 views

Multiplicative gradient descent?

The normal gradient descent is additive: $w_{t+1}=w_t-\lambda_t\nabla f(w_t)$, but is there a multiplicative gradient descent that looks something like $w_{t+1}=w_t[-\lambda_t\nabla f(w_t)]$? I know ...
0
votes
0answers
33 views

Is there any optimization algorithms for these problems?

I have modeling the problem with the following equation: $$ \min_{X} L(X)=f(X)-\alpha g(X) + \beta k(X) $$ where $\alpha \gt 0, \beta \gt 0$, and, in my cases, I found the $f(X)-\alpha g(X) \lt 0$ and ...
0
votes
0answers
18 views

Reconstruct a vector with a known vector and residual

I observe $\vec y$ and know $\vec x$. I assume that $\vec y$ mostly consists of $\vec x$, with some added residual $\vec r$. This gives me the problem $\vec y = a\vec x + \vec r$, where $a \in [0, ...
1
vote
0answers
30 views

Lagrange Multipliers in practice

Say we want to minimize the function $f^2({\bf{x}})$, under the constraint $g({\bf{x}})=0$. The classic solution (Method I) is to introduce a Lagrange Multiplier, and solve: $$\frac{\partial ...
2
votes
2answers
88 views

Optimal bin width for two dimensional histogram

There are lots of rules for selecting an optimal bin width in a 1D histogram (see for example) I'm looking for a rule that applies the selection of optimal equal-bin widths on two-dimensional ...
1
vote
0answers
48 views

Optimization in R vs Python, constrained, unconstrained and automatic differentiation?

I am an economics/stat guy who uses quite a bit of optimization (maximum likelihood, simulated maximum likelihood), constrained optimization (mathematical programming w/ equilibrium conditions), ...
0
votes
0answers
23 views

What is the best Global Optimization algorithm to deal with integer values (Binary)

I want to use a global optimization algorithm to find the optimum set of values for n variables (n>100) and all variables take binary values (0 or 1). I am thinking in Genetic Algorithm, Particle ...
3
votes
2answers
138 views

Under what circumstances is the log likelihood function of a point process concave?

I am trying to understand under what circumstances the log likelihood function of a point process concave. Assume that the process can be defined by a conditional intensity function and that the log ...
0
votes
0answers
14 views

Definition of the deterministic annealing method

iI have ran into a shape matching problem and one term which I read about is deterministic annealing. I learnt that it would help to convert discrete problems, e.g. ...
0
votes
0answers
12 views

Feature selection based on cost function

Suppose that we are searching for best features using an optimization algorithm for a classification model (MLP,SNM,Regression,etc...). We should set a cost ...
1
vote
0answers
31 views

Optimize number of layers and neurons with an optimization algorithm

I have a neural network that i want optimize number of hidden layers and neurons in every layer using an optimization algorithm like ...
0
votes
0answers
36 views

How to derive this problem with soft-thresholding method?

The problem is defined as $$ \min_{x} \Bigg\{ a{\|x\|}^2+\frac{b}{2}{\|x-c\|}^2 \Bigg\} $$ where $x\in R^{n \times 1}, c \in R^{n \times 1}$ and $a,b$ are scalars. Equations 2.5 to 2.8 of this paper ...
2
votes
0answers
46 views

Finding the most “uniform” or “least concentrated” density function, subject to moment constraints

Background I want to find a probability measure for a continuous random variable, subject to moment constraints, that is maximally "uniform", as defined below: Definition: Maximally Uniform ...
2
votes
1answer
55 views

Linear Regression with $L_2$: Different penalty strengths yield the same parameters?

Suppose I have model 1: $$Y=aX$$ where $X$ is $n\times1$, consisting of a single feature. suppose I fit this model with $L_2$ penalty with coefficient $\lambda=1$ : $$ \underset{a}{\min} ...
1
vote
1answer
47 views

Holt-Winters and Abnormal termination in LNSRCH

I try to fit data with Holt-Winters function in R. Nevertheless, i am getting the following message: ...
4
votes
2answers
62 views

Do optimization techniques map to sampling techniques?

From any generic sampling algorithm, one can derive an optimization algorithm. Indeed, to maximize an arbitrary function $f: \textbf{x} \rightarrow f(\textbf{x})$, it suffices to draw samples from $g ...
0
votes
1answer
70 views

Best optimization package for employee scheduling problem? [closed]

I am looking to solve the optimization problem described below. Which optimization software package would be best suited for this, considering the requirements specified below? Requirements: 1) Can ...
0
votes
1answer
64 views

Convert continues number to integer number in optimization algorithms in MATLAB

I'm using a continuous optimization algorithm for optimizing neural network's number of neurons in first and second layers besides feature selection so I used this structure for converting continues ...
2
votes
1answer
49 views

MAP estimate of posterior parameters

I have a setup where the joint posterior is written as: $$ P(w, \lambda, \phi \vert y) = P(\phi) \times P(w \vert \lambda) \times P(\lambda) \times \prod_{i=1}^{N}P(y_i \vert w_i, \phi, \lambda) $$ ...
3
votes
1answer
61 views

finding global minima of a random forest estimator

I have a random forest regression model with 1000 trees, having 16 parameters (using python scikit-learn). The estimator can predict a target value with cross validated r2 score of 0.87 +/- 0.03. I ...
1
vote
1answer
39 views

EM for Mixtures of Bernoulli (M-step)

When applying the M-step for a mixture of Bernoulli distributions, one of the parameters in our maximization is the Bernoulli parameter $\mu_{k}$, where $k$ is the index of the "mixture component", ...
1
vote
1answer
20 views

Optimise selection from a set, with constraints

I would like to use R to solve a problem I have. I don't even know what to call a problem of this kind and I'm finding Googling difficult. My guess is that this kind of problem already has R ...
0
votes
0answers
44 views

Parameter optimization of SVM

Currently I am using SVM to perform some classification task. I use libSVM with Matlab interface. From the practical guide of SVM (Link), we know that there are two parameters need to be tuned, namely ...
3
votes
1answer
45 views

Constrained Maximization and Likelihood Ratio Tests for Nested Linear Models

Suppose $\boldsymbol \beta \in \mathbb{R}^k$ is a vector of coefficients for a generalized linear model with $g \left[ E(Y|X) \right] = X\beta$ for a link function $g$ and I wish to test the composite ...
0
votes
1answer
57 views

What statistical method to correct systematic error in the output of a economic optimization model?

I am working with an economic optimization model which attempts to model the dynamics of a certain commodity market (prices, quantities, production etc.) for different frequencies (monthly, quarterly, ...
0
votes
0answers
18 views

Binary event probability optimization

I have a relatively small sample of binary events (50-100 events) that occurred during a time of day (the success rate is closely related to the time of day). I'm grouping these events into hour ...