In statistics this refers to selecting an estimator of a parameter by maximizing or minimizing some function of the data. One very common example is choosing an estimator which maximizes the joint density (or mass function) of the observed data referred to as Maximum Likelihood Estimation (MLE).

learn more… | top users | synonyms (2)

0
votes
0answers
7 views

Find the set of K elements between n that maximize the total distance [migrated]

Given a set $Q$ of $n$ points, we want to find the subset $S_{max}\subset Q$ of $k$ elements that maximize the total distance between them. $S_{max} = \max_S\sum_{\mbox{$\begin{array}{c} i,j\in S\\ ...
0
votes
0answers
18 views

marginal likelihood in linear bayesian regression (in weight-space)

I want to tune the hyperparameters namely the target deviance $\sigma_y$ and weight deviance $\sigma_w$ in bayesian linear regression. The posterior distribution in level-1 inference which is ...
0
votes
0answers
10 views

How to minimize setup cost in linear programming model using R [migrated]

Suppose we have a matrix m: > m C1 C2 M1 70 80 M2 50 61 M3 45 40 with machines M1-M3 and customers C1-C2. The values indicated the product volume that each ...
0
votes
2answers
70 views

Optimization of Complicated Function with Two Random Variables

I am trying to find $x$ that will minimize the following expression which involves two sources of randomness. I am stuck and not even sure where to start. Any suggestions would be appreciated. Please ...
1
vote
1answer
20 views

How to chose a store by optimizing my costs?

Let's say that I wanted to chose between two grocery stores (store $a$ and store $b$) in my area. They have the same items, and they both charge a variable price / cost for each product ($a_1$, $b_1$, ...
1
vote
0answers
15 views

How to optimize a generalized trace problem in dimensionality reduction

I know how to solve this problem in dimensionality reduction. $argmax_{X}$ $Trace[XLX^T]$ with $XX^T=I$ ,where $L$ is symmetric, $X$ is unitary, and $I$ is identity matrix. But I'd like to know how ...
1
vote
1answer
15 views

Solving “n” equations with 3 unknowns

I'm new to R and I'm trying to solve a system of equations. I have about 380 equations where i have 3 unknowns per equation. I can use three equations and solve by using "solve()" and it works great. ...
1
vote
1answer
23 views

Estimating error for parameters from multiple regression with linear constraints

I am working on a multiple linear regression problem where I would like to constrain only some of the parameters to non-negative values. There have been discussions of how to solve for the parameters ...
0
votes
0answers
13 views

when to stop this convex optimizations algorithm?

I am reading the article with title "metric learning with collaping classes" lately http://papers.nips.cc/paper/2947-metric-learning-by-collapsing-classes.pdf. See this thread (what is 1/0 in this ...
1
vote
1answer
90 views

Optimization of a Convex Function involving Standard Normal CDF and PDF

Could someone provide closed form solutions, if any, and steps to get there for the following optimization problem? Please note this function has been shown to be a convex function and hence a minimum ...
2
votes
1answer
36 views

How does one do Stochastic Gradient Descent (SGD) on an objective function that has a regularizer?

I know that for Stochastic Gradient Descent, one picks a data point $(x_n, y_n)$ at random from the training set $S_N$ and then updates the parameter of the model in question. If the cost function ...
1
vote
1answer
24 views

Why is it necessary to divide by the number of samples when optimizing squared error?

In a lot of different optimization problems, and with particular regard to gradient descent, we use the mean squared error as a loss function. In the formulation of mean squared error, you divide by ...
2
votes
0answers
87 views

Brent's method in the R optim implementation always returns the same local minimum [migrated]

I'm trying to minimise the function shown above. I'm searching between (-1,1). I use the following code ...
0
votes
0answers
7 views

How to demonstrate connection between performance and difference between real and optimal values of the parameters

I have the data from the experiment with 140 participants. I model the process of decision-making during the experiment and estimate the values of two parameters (A and B). Then, using simulations, I ...
1
vote
1answer
25 views

PSO Clustering using R using Repplab package

I wish to try clustering a matrix of numerical data using swarm intelligence. (Matrix is 28000 X 53 and sparse). I'm working in R and found the REPPlab package and used the EPPlab function. My ...
0
votes
0answers
52 views

Finding the optimal threshold parameter

Assume we are penalizing the least squares by the hard thresholding penalty: $argmin_\theta 2^{-1}(z_2-\theta)^2 + p_\lambda(|\theta|)$ where $p_\lambda(|\theta|)$ is the hard thresholding penalty ...
0
votes
0answers
17 views

How to optimize costly, smooth, multidimensional, varying scale function with flat regions and slight noise

I am trying to optimize hyperparameters of a complex model. Each iteration takes roughly 30s (during which the lower level model is run many times). I believe the underlying function to be generally ...
0
votes
0answers
17 views

What Technique to use to Optimize Future Prices?

I work for a company that sells different levels of ad packages on its website in various markets. These ad packages are sold contractually for specific periods of time (e.g. 3 months, 6 months, 1 ...
0
votes
0answers
18 views

What if we use mean and standard deviation in Stahel-Donoho outlier measure?

I need to use an outlyingness measure in an optimization problem which is already complex. So I need a simple measure of outlyingness. I didn't find any except Stahel-Donoho outlyingness measure. In ...
3
votes
1answer
28 views

Ensuring parameters of log linear model sum to 1

I am training a log-linear model with parameters $\theta$ using SGD. I want to ensure that my parameters will end up being probabilities i.e. $\sum_i \theta_i = 1$. One way to do this is by using ...
0
votes
1answer
60 views

Hessian for linear regression with regularization

I'm using matlab to solve a regularized linear regression via the fminunc() function. The cost function is from the standford machine learning class. It's pretty slow so and I think it could be sped ...
0
votes
0answers
16 views

What are some major theories on picking the right number for the sample window size in time series analysis?

for example the number of samples to run the moving average, or the number of samples for sequential hypothesis testing. Or if there is a control scheme going on what is the best time window for an ...
0
votes
0answers
23 views

Tikhonov regularizes least square dual function

I'd like to find the dual problem for least squares with Tikhonov regularization. For now, I have the primal problem expressed as minimize $||Ax-b||_2^2 + \gamma||x||_2^2$. I'm introducing a dummy ...
0
votes
1answer
88 views

How to minimize class weight vector of Random Forest Classifier using CV

What I'd like to do is optimize the class weights of a Random Forest Classifier (using python and the sklearn library) for multiclass classification, in which different misclassification errors have ...
1
vote
0answers
59 views

Hyperparameter tuning in Gaussian Process Regression

I am trying to tune the hyperparameters of the gaussian process regression algorithm I've implemented. I simply want to maximize the log marginal likelihood given by the formula ...
0
votes
0answers
31 views

Optimisation reproduces initial values? [migrated]

I am trying to maximise the following function with optimx in R ...
0
votes
0answers
30 views

Linear - Quadratic optimization for system of objectives

I have two distinct data sets, $\{x^{\mu},J^{\mu}\}$, $\mu=1,\ldots,n$ and $\{x^{\nu},V^{\nu}\}$, $\nu=1,\ldots,m$ that also include uncertainties $\delta J^{\mu}$ and $\delta V^{\nu}$. In these I fit ...
2
votes
1answer
32 views

Why are stochastic optimization algorithms able to find global minima (PSO and Genetic algorithm)

Why do these two methods, the particle swarm optimization (PSO) and the genetic algorithm find global minima (or are at least able to). And my second question is that these both algorithms are based ...
2
votes
0answers
38 views

How to find a cost function with only a statistical measure of success?

Using the U.S.A. as a loose analogy, we have search algorithms that find the names and number of States adjacent to a given State (containing a selected city). The goal is to minimize the number of ...
2
votes
0answers
35 views

Finding the optimal combination of independent variables for a constrained dependent variable

I'm currently working on power plant time series data and my main objective is finding out the optimal combination of independent variables which would keep "SO2 concentration (dependent variable) ...
0
votes
0answers
11 views

constrained linear combination fitting

I have a linear combination that I want to optimize. I defined it as a unconstrained convex optimization. The problem is that I need my params to be in 0-1 range, and thought I could normalize the ...
0
votes
0answers
25 views

Factor analysis with 2-norm equality constraint

I'm interested in the interpretation of the solution to the factor analysis problem with a 2-norm equality constraint on the columns of the loadings matrix. I plan to decompose $\mathbf{X}_i \in ...
1
vote
1answer
32 views

Constrained global optimization question

Given an example: Employee A,B,C can work on three tasks 1,2,3. The value created by each employee on each task are: A:(9,10,11) B:(4,5 ,10) C:(1,3 ,5) Ideally, A,B,C all work on task 3 will create ...
0
votes
0answers
50 views

How does the logistic regression with L-BFGS have to be initialized?

I've performed a logistic regression with L-BFGS on R and noticed that if I changed the initialization, the model retuned was different. Here is my dataset (390 obs. of 14 variables, Y is the target ...
1
vote
0answers
29 views

When is logistic regression minimizing under squared error loss the same as maximizing binomial likelihood?

Implementing logistic regression and getting different results depending on whether I minimize squared error or maximize log likelihood. When are the two equivalent?
1
vote
0answers
48 views

Method to minimize a quadratic form

How should I minimize a quadratic form $g(x)^TA(x)^{-1}g(x)$ with respect to $x$, where $g$ is a vector which depends on the vector $x$? This quadratic form is obtained from a quasi likelihood ...
0
votes
1answer
35 views

Non-linear model fitting

I would like to fit a non-linear model that looks like the following: $V(g)=a*A(g)/(b*B(g)+c*C(g))$, where $g$ represents a gene, $a$, $b$ and $c$ are coefficients of $A(g)$, $B(g)$, $C(g)$, which ...
0
votes
0answers
35 views

Trying to fit single layer neural net with R's nls (nonlinear least squares) function

Working on building a neural network modeling frame using graph objects in R. I have a data set on passengers of the Titanic, modeling binary "survived" variable against continuous "fare" and "age" ...
0
votes
2answers
40 views

How can I make this biological relation into a glm model?

I have a biological relation: Y/m = (X * b) / (1 + X * b) where Y and X are variables, m and b are parameters. m is greater than Y, and everything is greater than 0. I have some training data with ...
3
votes
1answer
125 views

What is the intuitive (geometric?) meaning of minimizing the log determinant of a matrix?

I have come across optimization problems which seek a positive semi-definite matrix $A$ that minimizes some possibly non-convex function that includes the addition of $1/(\text{dimension}) * \log \det ...
0
votes
1answer
26 views

Suffer from Local Optima

So here is my trouble: I wanted to test whether my estimation method is correct, so what I did was to simulate a data set with a group of parameters: (a=200, b=0.3, c=0.4, d=0.5, for example). If my ...
1
vote
0answers
19 views

Choose $m$ out of $n$ distributions s.t. the union of them likely contains top $k$ elements

I have $n$ sets of items. Each item in each set has a certain score. I want to select top $k$ items out of all available (i.e., out of the union of $n$ sets). However, explicitly calculating the union ...
1
vote
1answer
46 views

Gradient of loss function for (non)-linear prediction functions

$ \newcommand{\y}{\mathbf{y}} \newcommand{\wv}{\mathbf{w}} \newcommand{\xv}{\mathbf{x}} \newcommand{\loss}{L(\wv;\xv, y)} $ I'm trying to clear up the calculation of the gradient of a loss function, ...
0
votes
0answers
12 views

Coordinate descent method and collocation

Is it possible to use the coordinate descent/search method with a collocation approach ? My guess is no, because there isn't any discretization that can be applied on this method because it is a ...
0
votes
0answers
15 views

multivariate weighted sum of squares

I have some multivariate continuous data (multiple dependent variables) produced by an underlying model, I would like to estimate the parameters of the model that produced the data. To do this is ...
0
votes
2answers
106 views

Monte carlo optimisation (find maximum of function with multiple parameters)

UPDATE 4 UPDATE I JUST NEED TO know name of method(because there are hundreds of mmc methods) I have a description of a Monte Carlo method and don't know if it is a sequential monte -carlo, dynamic ...
2
votes
1answer
42 views

Prove the loglikelihood is strictly concave for ABO allele frequency blood type data

I am working through the problems in Kenn Lange's book Numerical Analysis for Statisticians. I am going to try and do all of the problems in the book, though none of them are specifically assigned for ...
0
votes
1answer
24 views

Finding the optimum result of an expensive process

I have an algorithm $A$ that takes parameters $\theta$ and returns a real number $x$: $A(\theta) = x$. I want to find the optimum value of $A(\theta)$ for values of $\theta$ within a fixed range. For ...
0
votes
0answers
69 views

how to make or prepare range file in svm-scale in libsvm using matlab

Respected all, I am using LIBSVM, for scaling the input data svm scale function is used. The syntax is 'svm-scale -l -1 -u 1 -s range train > train.scale' or svm-scale -s scaling_parameters ...
4
votes
1answer
37 views

Regularization and projection onto the $l_*$ ball

I'm trying to understand how regularization works in term of projections onto a $l_*$ ball, and Euclidean projection onto the simplex. I'm not sure I understand what we mean when we project the ...