In statistics this refers to selecting an estimator of a parameter by maximizing or minimizing some function of the data. One very common example is choosing an estimator which maximizes the joint density (or mass function) of the observed data referred to as Maximum Likelihood Estimation (MLE).

learn more… | top users | synonyms (2)

0
votes
0answers
6 views

Where is the Vowpal Wabbit SGD objective function specified? [on hold]

I am new to Vowpal Wabbit, and have been reviewing everything I can find on it. I am interested in using SGD to optimize a fairly large problem with a few hundred parameters several hundred prediction ...
0
votes
0answers
24 views

Linear - Quadratic optimization for system of objectives

I have two distinct data sets, $\{x^{\mu},J^{\mu}\}$, $\mu=1,\ldots,n$ and $\{x^{\nu},V^{\nu}\}$, $\nu=1,\ldots,m$ that also include uncertainties $\delta J^{\mu}$ and $\delta V^{\nu}$. In these I fit ...
2
votes
1answer
25 views

Why are stochastic optimization algorithms able to find global minima (PSO and Genetic algorithm)

Why do these two methods, the particle swarm optimization (PSO) and the genetic algorithm find global minima (or are at least able to). And my second question is that these both algorithms are based ...
2
votes
0answers
34 views

How to find a cost function with only a statistical measure of success?

Using the U.S.A. as a loose analogy, we have search algorithms that find the names and number of States adjacent to a given State (containing a selected city). The goal is to minimize the number of ...
2
votes
0answers
27 views

Finding the optimal combination of independent variables for a constrained dependent variable

I'm currently working on power plant time series data and my main objective is finding out the optimal combination of independent variables which would keep "SO2 concentration (dependent variable) ...
0
votes
0answers
32 views

Arraning arriving Trains as good as possible to minimize the number of Tracks needed [closed]

The problem I like to solve is very similar to the following: There are a number of trains arriving at different times, and staying for different duration at a rail station. e.g.: Train 1 arrives ...
0
votes
0answers
7 views

constrained linear combination fitting

I have a linear combination that I want to optimize. I defined it as a unconstrained convex optimization. The problem is that I need my params to be in 0-1 range, and thought I could normalize the ...
0
votes
0answers
17 views

Constrained optimization of non linear functions in R [closed]

My objective is to maximize sales by reallocating my spend under certain constraints. For each key the sales is given by alpha*spend^beta. The total sales is the the sum of sales across all keys and ...
0
votes
0answers
10 views

Matlab optimize expression with L-infinity norm [closed]

I am trying to implement the following minimization problem in matlab $min_{A} ||X - XA||^{2}_{F} + \lambda \sum^{d}_{i=1}||a_j||_{\infty}$ where $a_j$ is the $j^{th}$ row, $A,X \in \mathbb{R}^{d ...
0
votes
0answers
19 views

Factor analysis with 2-norm equality constraint

I'm interested in the interpretation of the solution to the factor analysis problem with a 2-norm equality constraint on the columns of the loadings matrix. I plan to decompose $\mathbf{X}_i \in ...
1
vote
1answer
25 views

Constrained global optimization question

Given an example: Employee A,B,C can work on three tasks 1,2,3. The value created by each employee on each task are: A:(9,10,11) B:(4,5 ,10) C:(1,3 ,5) Ideally, A,B,C all work on task 3 will create ...
0
votes
0answers
35 views

How does the logistic regression with L-BFGS have to be initialized?

I've performed a logistic regression with L-BFGS on R and noticed that if I changed the initialization, the model retuned was different. Here is my dataset (390 obs. of 14 variables, Y is the target ...
1
vote
0answers
25 views

When is logistic regression minimizing under squared error loss the same as maximizing binomial likelihood?

Implementing logistic regression and getting different results depending on whether I minimize squared error or maximize log likelihood. When are the two equivalent?
1
vote
0answers
41 views

Method to minimize a quadratic form

How should I minimize a quadratic form $g(x)^TA(x)^{-1}g(x)$ with respect to $x$, where $g$ is a vector which depends on the vector $x$? This quadratic form is obtained from a quasi likelihood ...
0
votes
1answer
31 views

Non-linear model fitting

I would like to fit a non-linear model that looks like the following: $V(g)=a*A(g)/(b*B(g)+c*C(g))$, where $g$ represents a gene, $a$, $b$ and $c$ are coefficients of $A(g)$, $B(g)$, $C(g)$, which ...
0
votes
0answers
28 views

Trying to fit single layer neural net with R's nls (nonlinear least squares) function

Working on building a neural network modeling frame using graph objects in R. I have a data set on passengers of the Titanic, modeling binary "survived" variable against continuous "fare" and "age" ...
1
vote
0answers
30 views

How do you do constrained non-linear least squares in R [migrated]

I am fitting a non-linear least squares model in R. I wish to minimize $(Y - f(Xb))^2$ where $f$ is a nonlinear monotone differentiable function, $X$ is a set of features and $b$ is the parameter ...
0
votes
2answers
36 views

How can I make this biological relation into a glm model?

I have a biological relation: Y/m = (X * b) / (1 + X * b) where Y and X are variables, m and b are parameters. m is greater than Y, and everything is greater than 0. I have some training data with ...
3
votes
1answer
116 views

What is the intuitive (geometric?) meaning of minimizing the log determinant of a matrix?

I have come across optimization problems which seek a positive semi-definite matrix $A$ that minimizes some possibly non-convex function that includes the addition of $1/(\text{dimension}) * \log \det ...
0
votes
1answer
24 views

Suffer from Local Optima

So here is my trouble: I wanted to test whether my estimation method is correct, so what I did was to simulate a data set with a group of parameters: (a=200, b=0.3, c=0.4, d=0.5, for example). If my ...
1
vote
0answers
18 views

Choose $m$ out of $n$ distributions s.t. the union of them likely contains top $k$ elements

I have $n$ sets of items. Each item in each set has a certain score. I want to select top $k$ items out of all available (i.e., out of the union of $n$ sets). However, explicitly calculating the union ...
0
votes
1answer
30 views

Gradient of loss function for (non)-linear prediction functions

$ \newcommand{\y}{\mathbf{y}} \newcommand{\wv}{\mathbf{w}} \newcommand{\xv}{\mathbf{x}} \newcommand{\loss}{L(\wv;\xv, y)} $ I'm trying to clear up the calculation of the gradient of a loss function, ...
0
votes
0answers
9 views

Coordinate descent method and collocation

Is it possible to use the coordinate descent/search method with a collocation approach ? My guess is no, because there isn't any discretization that can be applied on this method because it is a ...
0
votes
0answers
10 views

multivariate weighted sum of squares

I have some multivariate continuous data (multiple dependent variables) produced by an underlying model, I would like to estimate the parameters of the model that produced the data. To do this is ...
-1
votes
0answers
29 views

How this Monte-Carlo method called(name) optimization of functions with multiple parameters [duplicate]

I have a description of a Monte Carlo method and don't know if it is a sequential monte -carlo, dynamic monte-carlo? What should I be looking for? Do you know similar methods? This is optimization ...
0
votes
2answers
78 views

Monte carlo optimisation (find maximum of function with multiple parameters)

UPDATE 4 UPDATE I JUST NEED TO know name of method(because there are hundreds of mmc methods) I have a description of a Monte Carlo method and don't know if it is a sequential monte -carlo, dynamic ...
2
votes
1answer
36 views

Prove the loglikelihood is strictly concave for ABO allele frequency blood type data

I am working through the problems in Kenn Lange's book Numerical Analysis for Statisticians. I am going to try and do all of the problems in the book, though none of them are specifically assigned for ...
0
votes
1answer
23 views

Finding the optimum result of an expensive process

I have an algorithm $A$ that takes parameters $\theta$ and returns a real number $x$: $A(\theta) = x$. I want to find the optimum value of $A(\theta)$ for values of $\theta$ within a fixed range. For ...
0
votes
0answers
31 views

how to make or prepare range file in svm-scale in libsvm using matlab

Respected all, I am using LIBSVM, for scaling the input data svm scale function is used. The syntax is 'svm-scale -l -1 -u 1 -s range train > train.scale' or svm-scale -s scaling_parameters ...
4
votes
1answer
35 views

Regularization and projection onto the $l_*$ ball

I'm trying to understand how regularization works in term of projections onto a $l_*$ ball, and Euclidean projection onto the simplex. I'm not sure I understand what we mean when we project the ...
2
votes
1answer
51 views

Optimal Binning with respect to a given response variable

I'm looking for optimal binning method (discretization) of a continuous variable with respect to a given response (target) binary variable and with maximum number of intervals as a parameter. ...
5
votes
1answer
97 views

Estimation with MLE and returning the score/gradient (QMLE)

I am estimating a simple AR(1) process by the ML approach. I also wish to compute the Quasi MLE standard errors, which is given by the sandwich form of the Hessian and the Score (see for example the ...
0
votes
0answers
11 views

Empirical likelihood method to Find the order of a parameter given a set of constraints [migrated]

Firstly we assume that $X_1,...X_n$ are order statistics($X_i\leq X_{i+1}$) from an i.i.d sample of random variables and let $r$ be integer and $r\geq1$. Start with the Equation (1) as following ...
0
votes
1answer
43 views

Lagrange Multiplier for fuzzy clustering with size constrains

I'm trying to solve a clustering problem with size constrains. Minimize $J=\sum_{i=1}^c\sum_{j=1}^n {{u_i}_j}^2{d_i}_j$ $d_{ij}$ is the distance from each element to it's cluster center. Usually ...
0
votes
0answers
32 views

Quasi-Newton Accelerator (QN1) for EM Algorithm

I am trying to implement what is called a "very simple to implement" accelerator for the EM algorithm. Specifically I am talking about the QN1 algorithm, described here, and am using a multivariate ...
0
votes
0answers
11 views

Non - Linear, Non-Smooth Optimization in Excel [migrated]

I am trying to solve a non-linear, non-smooth optimization in excel. Both GRG and Evolutionary algorithms are not able to give reasonable results(they are not converging in certain cases). The number ...
1
vote
3answers
69 views

How to use an optimization solver to get t-stats and p-values for the estimates?

I calculate a data log likelihood (evaluated at a set of parameters to be estimated), and my task is to find the set of parameters that maximize my log likelihood. My problem is: thought there are a ...
3
votes
0answers
29 views

Heuristics streaming data matching

I have an index composed by thousands of documents. Slightly modified copies of those documents are sent to my application in small chunks, and I need to check, from those chunks, which document has ...
0
votes
0answers
23 views

Checking error in optimisation problem

My apologies if this problem is trivial -- I do not have much experience with statistical methods! Hopefully someone can point me in the right direction. Background: I am currently working on an ...
0
votes
1answer
28 views

Online gradient descent for strongly convex function

Given that our loss function is $\alpha$ strongly convex function which means $\mbox(\nabla f(\mathbf{x})-\nabla f(\mathbf{y}))^{T}(\mathbf{x}-\mathbf{y})\geq \alpha||\mathbf{x}-\mathbf{y}||_{2}^{2} ...
4
votes
1answer
87 views

Is an SVM's (maximum) likelihood uniquely defined as a function of hyperparameters?

I think that I must be reading this paragraph (below) incorrectly. Note that both types of evidence that we have defined in general depend on the inverse noise level $C$ and the kernel $K(x, ...
0
votes
1answer
25 views

Coordinate vs. gradient descent

I was wondering what the different use cases are for the two algorithms, Coordinate Descent and Gradient Descent. I know that coordinate descent has problems with non-smooth functions but it is used ...
1
vote
0answers
17 views

Extensions to distributed Stochastic Gradient Descent

In the conclusions of the paper for distributed Stochastic Gradient Descent using mini-batches the authors discuss a problem of the proposed method regarding the synchrony requirements. Specifically, ...
0
votes
0answers
32 views

How do you determine neural network loss function when there are multiple outputs?

This great Youtube tutorial taught me how to fit a neural network with one output. To apply back-propagation, you first find the Jacobian of the loss function with respect to the weights. ...
0
votes
1answer
35 views

For a quadratic form to minimize with a L2 regularization term, is the gradient of the solution collinear to the solution?

Say you minimize a quadratic form f with a L2 regularization term (g = f + L2_term). The solution of minimizing g is x*. Is the gradient of f applied to x* collinear to x* as the figure below ...
0
votes
0answers
6 views

Worried about hidden costs of using this loss function to fit weights

I have the following model: $$\frac{Y}{T} = f(X \beta)$$ where beta is a vector of weights, and Y and T - Y are greater than 0. I want to fit the $\beta$ vector using the loss function $$ ...
1
vote
0answers
21 views

Expectation Propagation - Computing mean and variance of error function

I'm still trying to wrap my head around computing the moments for the expectation propagation algorithm and whether I can use it for the following example: say i have a product of distributions which ...
0
votes
1answer
59 views

Neural networks: how can convex optimization produce different weights each time?

I am training a multilayer perceptron with a logistic activation function by backpropagation. The weights are not unique - each time I redo the fit, I get a different set of weights. However the ...
2
votes
1answer
52 views

L-BFGS-B converging in iteration 0 in R [closed]

I am trying to fit an F distribution to a given set using optim's L-BFGS-B method. For some reason, it is always converging at iteration 0, which obviously doesn't ...
1
vote
1answer
76 views

Advantages of taking the logarithm to minimize the likelihood

In regression/classification problem, we are often interested in minimizing a cost function with respect to the parameters of the model. In many cases, the cost function is the negative likelihood. To ...