In statistics this refers to selecting an estimator of a parameter by maximizing or minimizing some function of the data. One very common example is choosing an estimator which maximizes the joint density (or mass function) of the observed data referred to as Maximum Likelihood Estimation (MLE).

learn more… | top users | synonyms (2)

1
vote
0answers
8 views

Constrained regression for binary dependent variable

I would like to discuss the methodology of the following case: I have a data for several patients over several years for 5 factors describing the health of a particular patient. Every factor consists ...
0
votes
1answer
19 views

Nonlinear square optimization task in matlab [on hold]

let us suppose that we have following task:Find the optimal value of weights so that minimize following equation where var-means variance of given x1 variable, also we have constraint that ...
0
votes
0answers
7 views

Function in R that can gives optimal value of each variabe to maximise output of a linear equation [on hold]

i have linear regression equation of which i want the optimal values of each variable to maximise my Y. Is there a function(may be optim) in R, that takes linear equation and a dataset, with historic ...
0
votes
0answers
26 views

Estimating Gamma MLE with left truncated data (using R and maxLik)

I'm trying to find the maximum likelihood estimation of the parameters of a Gamma distributed random variable using maxLik. The following code explain what I did: ...
0
votes
0answers
29 views

Statistical optimization Model for which person should be assigned a lead within each state

I am working on creating an optimization model based on sales on a particular website. The system assigns the leads to different sales people. I want to generate a model doesn't randomly assign leads ...
0
votes
0answers
8 views

Is there any software package to minimize the weighted total variation for image recovery? [closed]

I would like to recover my original image using the following weighted total variation minimization. I have the implemented package for unweighted but i have no idea how to solve this one using ...
0
votes
0answers
6 views

Find a maximum value for input data

I have a set of 20x20000 input variables and 1x20000 target values. I want to find a best combination of 5 variables out of 20 on a condition that other 15 variables are constant to get the best ...
0
votes
0answers
11 views

Multinomial Logistic Regression aka Softmax Regression

In optimization point of view of generalized linear modeling, there is a transfer function that maps a linear score to a final target. There is also a loss function that is minimized in training to ...
0
votes
0answers
30 views

Scaling overlapping subsets, optimizing nearness [closed]

For a set of two thousand xyz points, I am multiply-scaling the z values. The points exist in overlapping subsets. Each subset is scaled by a float variable. In all there are anywhere from ...
0
votes
0answers
10 views

Strategy for estimating a more complex hidden markov model (HMM)

I have a HMM in mind where emission probabilities change over time (not dependent on state). For example, suppose I have two states and four possible emissions. If in state 1, emission probabilities ...
4
votes
1answer
23 views

Bayesian Optimization for a Stochastic Target that changes over time

Let's say there is a single slot machine that: costs zero to play can only be played once per day has a payout that is conditionally normal and is a function of the date and time. I want to use ...
-1
votes
0answers
35 views

two parameter optimization in R [closed]

I have to find the maximum of accuracy (of classifier) which is depending on only two parameters and my parameters are as i.e., p1 ranges from 1 to 400 and p2 ranges from 666 to 1000. Please help me ...
0
votes
0answers
8 views

fast way to train a classifier on different but overlapping features

I am training a linear classifier repeatedly on different set of overlapping features. I have a 3D grid of features, each time features from a small sphere from a grid are used to train a classifier, ...
0
votes
0answers
10 views

Some clarifications about Stochastic and Online Gradient descent

Stochastic and Online gradient descent are ofte used as synonimous, the reason is because both use a 1-sample estimate of the gradient in the parameters update law. Infact assume we have a process ...
0
votes
1answer
19 views

How do I calculate the tipping point of over/under odds in Football?

I am trying to understand how game odds work. One scenario I came across was the over/under scores for football (soccer) games in the form of a table like this: ...
0
votes
0answers
16 views

Statistical analysis of the results calculated from a stochastic global optimization

I recently use the Basinhopping algorithm [1], a Monte Carlo Markov Chain variant, to find the global minimum of numerical functions (which are not necessarily smooth). Though the global optimization ...
1
vote
1answer
90 views

Newton's method for regression analysis without second derivative

In regression analysis, instead of gradient descent, Newton's method can be used for minimizing the cost function. However, in Newton's method, we need to calculate second derivative too. For ...
0
votes
0answers
13 views

Genetic algorithm for solving the un-certain dimension problem

Introduction Last week, I have learned basic genetic programming using Python to solve a simple problem. I introduce it here: There is a city need for air quality monitoring network. The ...
4
votes
1answer
37 views

In neural nets, why use gradient methods rather than other metaheuristics?

In training deep and shallow neural networks, why are gradient methods (e.g. gradient descent, Nesterov, Newton-Raphson) commonly used, as opposed to other metaheuristics? By metaheuristics I mean ...
0
votes
4answers
122 views

Why is the gradient the best direction to move in?

When optimizing a convex function, doing an update like: $$w_{t+1} =w_{t}+ c\ \nabla(f(w)) $$ is recommended. Why is moving along $\nabla(f(w))$ the fastest way to move closer to the goal? What's the ...
0
votes
0answers
10 views

R: Constrained Optimisation: Set limit to number of non-zero parameters [migrated]

Say I am trying to optimise the parameters of a function given some data. Say the parameter is a 3-element vector. Using the constrOptim function I could set the conditions that the sum of the ...
1
vote
2answers
63 views

Lower-bound for total-variation between two sub-gaussian random variables

Problem setup Let $X$ and $Y$ random variables on the real line with following properties: $X$ and $Y$ are $\sigma^2$ sub-gaussian $E[X] = 0$ and $E[Y] = \Delta$. Question Whats the lower ...
1
vote
0answers
30 views

Sparsity on the simplex

Say I want to minimize a convex function on the probability simplex. How is it possible to encourage the sparsity of the solution (while keeping the problem convex)? Since using the sparsity ...
3
votes
1answer
53 views

Optimization of stochastic computer models

This is a tough topic for myself to google since having the words optimization and stochastic in a search almost automatically defaults to searches for stochastic optimization. But what I really want ...
1
vote
0answers
13 views

what are some of the success stories of applying optimization in computer vision?

I'm trying to find several examples that demonstrate the success of optimization methods in application to computer vision and robotics. In particular, I'm interested in problems I can formulate from ...
0
votes
0answers
24 views

Why is Training Error lower than Testing Error during the First Epoch?

I am training a deep neural network for classification (specifically, a convolutional neural network for object recognition). I use mini-batches for my training, because I cannot fit the entire ...
0
votes
0answers
14 views

Hyperparameter optimization on large dataset

I have a huge dataset and want to carry out regression, such as gradient boosting. The problem is that the dataset is huge and hyperparameter optimization is computational expensive, especially I use ...
1
vote
0answers
32 views

How do you pronounce BOBYQA?

This question might seem subjective, but I ask because there is some precedent on this site. I and my colleagues have always pronounced the name of this optimizer as "bobby cue ay" (which also gives ...
1
vote
0answers
37 views

Gradient descent using Newtons method

I am implementing gradient descent for regression using newtons method as explained in the 8.3 section of the Machine Learning A Probabilistic Perspective (Murphy) book. I am working with two ...
0
votes
0answers
25 views

Optimization without gradient [duplicate]

I have a non-convex function of several parameters, that takes real values. I am looking for a good algorithm to find local minima. I am reasonably confident I can come up with descent starting ...
1
vote
0answers
53 views

Divergence in gradient descent

I am trying to find a function h(r) that minimises a functional H(h) by a very simple gradient descent algorithm. The result of H(h) is a single number. (Basically, I have a field configuration in ...
1
vote
0answers
21 views

How can I compute the FTRL proximal algorithm in a distributed way? [closed]

I want to run the algorithm to multiple nodes since the training data is to big to run on a single. I plan to implement it on Spark.
3
votes
0answers
39 views

How to parameterize coefficient matrix to restrict eigenvalues?

Consider the $r-$dimensional autoregression $$ y_t = Ay_{t-1} + v_t, v_t \overset{iid}{\sim}N(0,\Sigma). $$ It is well known that if all eigenvalues of $A$ have modulus less than unity then this ...
0
votes
1answer
26 views

Gradient descent for a noisy system

I have a system with tuning parameter $w$. To evaluate this system I use cost function $f(w)$. I try finding the optimum value for $w$ using Gradient Descent starting from $w_0$. The problem with ...
0
votes
0answers
12 views

Inexact line search in Gradient descent

In general setting of gradient descent algorithm we have, \begin{equation} x_{n+1}=x_n−\alpha d_n, \end{equation} where $\alpha$ is the step size and $d_n$ is the gradient evaluated at the point ...
0
votes
0answers
27 views

Gradient descent or mean-field update for variational Bayes

I am using variational Bayes to perform inference for a graphical model. In particular, I am using the mean-field approach such that the approximate posterior factorises over the parameters of ...
5
votes
1answer
54 views

Finding optimal correspondences between objects given two square distance matrices

I would like to find the optimal correspondences between two systems of objects based on the distances between objects WITHIN the two systems. So, the input to the algorithm would be two square ...
0
votes
0answers
12 views

DoE and Parallel Coordinate Plot

First off, I'm not too technical; however, I know what I need to do more or less, but don't know how to put the pieces together. I need to create a Design of Experiments, perhaps with R Studio. With ...
2
votes
0answers
20 views

Combinatorial optimization: split features into several subsets to maximize overall score

I tried to reformulate my original problem (that is quite difficult to explain) to a simpler one. Please, take it into account when you may think that the final goal doesn't make too much sense. ...
3
votes
1answer
40 views

Most suitable optimizer for the Gaussain process likelihood function

The Gaussian process (GP) log Likelihood function can be expressed as Where K is a positive definite covariance matrix. The hyperparameters can be obtained through maximizing the likelihood ...
2
votes
2answers
32 views

Can I estimate the optimal proportions of ingredients in a blend?

I'm trying to find the optimal blend of 4 ingredients, to maximize liking of a beverage. I can write a function for liking such as: Liking is on a scale of 0 - 1 (0% - 100%). I conducted 10 ...
0
votes
0answers
13 views

How to make educated guess about movement of people through graph?

I have data about weekly counts of people on entry points (orange circles on the picture below) and need to make educated guess about their counts at destination points (marked by green stars). I know ...
0
votes
0answers
14 views

Constrained optimization

I was looking at a colleagues solution to an optimization problem, and I was intrigued by his approach. The problem is defined as : \begin{equation*} \begin{aligned} & ...
18
votes
1answer
494 views

Understanding “almost all local minimum have very similar function value to the global optimum”

In a recent blog post by Rong Ge, it was said that: It is believed that for many problems including learning deep nets, almost all local minimum have very similar function value to the global ...
1
vote
0answers
13 views

Need to solve a “pseudo-metabolic” network for rates

Sorry about the re-post from Biology section... following problem: I have data on a certain complex association network (from monomers 1 ... 8 to complexes of all combinations, such as, 12 ... 13 ... ...
8
votes
0answers
113 views

Restricted maximum likelihood with less than full column rank of $X$

This question deals with restricted maximum likelihood (REML) estimation in a particular version of the linear model, namely: $$ Y = X(\alpha)\beta + \epsilon, \epsilon\sim N_n(0, \Sigma(\alpha)), $$ ...
0
votes
0answers
23 views

Optimizing a model for a limited budget

I am building models to predict probability of failures against a list of approximately 500K assets. I want to optimize my models for maximum predictive performance on a fixed (limited) number of ...
6
votes
1answer
154 views

Loss function Approximation With Taylor Expansion

As an example, take the objective function of the XGBOOST model on the $t$'th iteration: $$\mathcal{L}^{(t)}=\sum_{i=1}^n\ell(y_i,\hat{y}_i^{(t-1)}+f_t(\mathbf{x}_i))+\Omega(f_t)$$ where $\ell$ is ...
1
vote
0answers
24 views

Problem with initial guess in Newton-Raphson iteration method

I'm working on estimating the four parameters of Exponentiated Modified Weibull Extension Distribution introduced by Sarhan and Apaloo (2013) with the Maximum Likelihood Estimation (MLE). Because the ...
0
votes
0answers
33 views

orthogonal constraints in maxEnt methods

In a typical entropy maximization problem, in which we wish to maximize the entropy of a distribution subject to several moment constraints, how important is it that the moment constraints are ...