In statistics this refers to selecting an estimator of a parameter by maximizing or minimizing some function of the data. One very common example is choosing an estimator which maximizes the joint density (or mass function) of the observed data referred to as Maximum Likelihood Estimation (MLE).

learn more… | top users | synonyms (2)

0
votes
0answers
7 views

The optimization problem of soft margin Support Vector Machine: How to interpret?

I try to understand what exactly we are trying to optimize in the case of Support Vector Machine problem, which supports soft margins. The original problem is posed first as, without soft margins ...
0
votes
0answers
18 views

Polynomial fitting for Shading Correction

I am reading the book Optimization for Computer Vision, and the first example of optimization is a regression for shading correction, in which the author proposes the following polynomial: $p_s(c,x) ...
1
vote
1answer
14 views

Is this problem convex ? (regularization term on xTw)

Suppose we want to solve the following: $$ \min_{w} f(x^Tw, y) + \lambda g(x^Tw) $$ with $f$ a (logistic) loss and $g$ something like a variance. Is this a convex optimization problem ? What are ...
0
votes
0answers
4 views

Lagrange multiplier method [migrated]

I am doing some data mining algorithm self learning tutorial. I came up with a problem which I need your help to solve. In order to minimize the resource consumption, a car manufacturer considers how ...
0
votes
0answers
6 views

easy and difficult sample point

In my dataset, I can assume that there are two groups of samples. For one group, my model can easily learn the labels and for the other group my model has a poor performance in prediction of the ...
0
votes
0answers
8 views

Classification problem with constraints

I am trying to solve a classification problem with constraints and need advice on how I should approach it. Here's the problem: Given N observations, FLAG_j, j=1,..,N (this is a binar variable), and ...
0
votes
1answer
25 views

Dimension Reduction

I have a n*m matrix, the rank of matrix (r) is near to min(m,n) I want to minimize the rank by removing some of the rows or columns to get r << min(m,n) The goal is to achieve least rank for ...
0
votes
0answers
11 views

Bayesian optimization for known objective function with high dimension

I was wondering if one can use Bayesian Optimization algorithm for KNOWN, high-dimensional, expensive objective functions? If the answer is yes how efficient is that in terms of the quality of the ...
3
votes
0answers
156 views

Maximum likelihood estimation involving both probabilities and probability densities

Note: based on suggestions in the comments, I have rewritten this question. Please refer to the history for the original version. In general my question regards how to compute likelihoods in mixed ...
0
votes
0answers
45 views

Encouraging 'de-duplication' (ie: nullifying duplication) in an evolutionary optimization problem

I have a chronological production process with 24 cycles. Each production cycle has 5 of 8 conveyor belts in operation in each production cycle, whilst 3 of 8 belts are always at rest for ...
2
votes
0answers
7 views

Norm-bounded input

What is norm-bounded input? The expression is used in section 4.4 of 'Building High-level Features Using Large Scale Unsupervised Learning' by Le et al. I can find papers using the term, but not any ...
2
votes
1answer
49 views

How do I force the L-BFGS-B to not stop early? Projected gradient is zero

I'm trying to use the SciPy implementation of the fmin_l_bfgs_b algorithm using the following code: ...
1
vote
0answers
37 views

Unable to understand joint pdf and EM

I am unable to understand how the density function is derived in this paper Semiblind System Identification with Symbolic Chaotic Sequences The Authors have ...
1
vote
1answer
22 views

Unnatural clustering with known clusters shapes and optimization criteria

My question is similar to this question Clustering with shape prior, but with additional information. The second answer suggests a mixture model approach to this problem, which is something like ...
2
votes
1answer
47 views
0
votes
1answer
20 views

Time series tracking queue optimization problem

In order to track prices of many different products from different sources, I must optimally schedule a group of trackers dedicated to price collection (ie. collect one price at a time for each ...
0
votes
0answers
7 views

Optimisation of Fans In An Account

I have daily facebook data (past 3-4 months) of a company. I know how many fans they gained per day, fans they lost per day, engagements and so on. These are split into 'paid for', 'free from the ...
0
votes
0answers
30 views

optim() for multi variable returns values on the boundary in R

I would like to use function optim() in R to minimise the target function. The two optimised parameters both have constrains. I have created a test sampel data. ...
0
votes
1answer
35 views

SVM parameters clarification

James et al. in An introduction to the statistical learning (p. 351) claim that the solution to the support vector classifier problem involves only the inner products of the observations. They ...
2
votes
0answers
19 views

Perturbation of binned random variables

Setup: Assume that we have an interval coded random variable $T$ which takes values in the set of half-open intervals $\{[L_1, U_1), \ldots, [L_K, U_K]\}$, where $U_k$ and $L_k$ are the upper and ...
3
votes
1answer
41 views

Using quadratic programming to fit a piecewise linear model plus seasonality

I am reading this paper on fitting an L1TF model to data using quadratic programming. Section 7.4 states how one could add seasonality to the model however it doesn't go very far into it. I am trying ...
0
votes
1answer
16 views

Lagrange: Difference between minimizing and maximizing

The following seems straight forward: 1) Maximize f(x1, x2) = x1x2 subject to h(x1, x2) ≡ x1 + 4x2 = 16. Form the Lagrangian: L(x1, x2) = x1x2 − λ (x1 + 4x2 − 16) The first order conditions are (1) ...
0
votes
0answers
12 views

Faster gradient descent convergence by transforming the gradient?

If we modify the gradient descent update for a convex objective function $f(\boldsymbol{\theta})$ from $\boldsymbol{\theta}_{t+1} = \boldsymbol{\theta}_t - \nabla f(\boldsymbol{\theta}_t)$ to ...
0
votes
0answers
5 views

Optimizing grid of sources and multipliers

I have a 35x35 grid of cells. There are two possible states for cells: "source" and a "multiplier". The center 25x25 cells may be either "source" or "multiplier". The rest of the cells must be ...
0
votes
0answers
24 views

Optimize parameters?

I've been back-testing a trading strategy that has two parameters (both are # of days to look back), and I've tested the system for robustness by comparing my 10-year Sharpe ratio based on approx ...
0
votes
0answers
27 views

forcing certain parameters to be skipped during optim in R

I have a code which tests each possible order of ARIMA and selects the best model by choosing the one with the absolute minimum sum of lags from the PACF graph. The code then proceeds to add weight to ...
0
votes
1answer
28 views

Constraining norms with inequalities [closed]

I have time-series data for N stocks. sample.data<-rep(10,rnorm(100)), where each column shows the returns of different stocks over time. I am trying to ...
0
votes
0answers
24 views

Exponential distribution of interval censored data MLE using optim on R error

I have a set of data which is interval censored and I wish to calculate its maximum likelihood error using optim. This is my code: ...
2
votes
1answer
53 views

Linear regression with an inequality constraint

I am looking for an efficient way of finding a linear fit $Mx = y$ subject to an inequality constraint: $\frac{|x_2|}{\sqrt{x_3^2 + x_4^2}} \geq a$, with $a \geq 1$. The rectangular matrix $M$ is ...
1
vote
1answer
39 views

Why is k-medians typically used with Manhattan rather than Euclidean distance?

K-medians is typically used with Manhattan distance rather than Euclidean distance. Why is this?
0
votes
0answers
4 views

Optimal intertemporal allocation of bets based on estimated returns

Each day, an investor predicts the day's stock-market return in the morning, generating a point estimate and prediction interval, and chooses to bet an amount $B_t$ on the market that day based on her ...
3
votes
0answers
34 views

Why is 'relaxed LASSO' different from LASSO?

If we start with a set of data $(X,Y)$, apply LASSO to it and obtain a solution $\beta^L$, we can apply LASSO again to the data set $(X_S, Y_S)$, where $S$ is the set of non-zero indexes of $\beta^L$, ...
0
votes
1answer
45 views

Are there optimization methods for k-NN parameter $k$?

Currently I am just go through from the min to the max, and determine $k$ by the performance. I am wondering if there's optimization approaches for selecting $k$? I am aware of there's question in ...
1
vote
1answer
33 views

Optimizing for target metrics in Weka

I'm a PhD student in Information Retrieval with some limited experience in ML. We've been working on a binary classification task with weka (I'm using weka programmatically via Java), specifically ...
0
votes
0answers
45 views

Regression Analysis and Optimization in R

I have the following table: ...
0
votes
0answers
26 views

Why we even try to minimize a loss function, which is non-convex, in matrix factorization?

In matrix factorization (especially under the scenario of recommendation system), we often try to factorize a matrix Y into two low rank matrices: $Y=U\cdot V^T$ If we assume there $m$ instances in ...
0
votes
0answers
13 views

fitting a non-linear curve with one parameter

I have an equation: $\ddot{x}+(\delta+\epsilon\cos{t})x=0$ known as the Mathieu equation.The $\delta-\epsilon$ parameter space of this equation looks something like The red lines in this diagram ...
0
votes
0answers
14 views

What happens if you replace the sampling procedure in MCMC with maximization when fitting a HMM?

When using Markov chain monte carlo to fit hidden Markov models, after you use the forward algorithm to obtain the posterior distribution, you sample the hidden states for the current observation. ...
0
votes
0answers
13 views

Optimizing selection from varying sets

On pages of website(s) I have a set of potential messages to choose from and only one or two slots to show them in. (think 'this product is on sale' or 'this product is new'). On each page the set ...
1
vote
0answers
35 views

Solving for GLM coefficients w/ Newton-Raphson

Note : This is a question about a homework problem I am facing. I have some data that is to be modeled with a logistic regression model. I am supposed to do two things: (1) use newton-raphson ...
0
votes
0answers
19 views

How important is the quality of solutions to NP-hard problems arising in machine learning problems?

Machine learning and inference problems give rise to intractable problems. For instance the exact inference in Bayes nets is an NP-hard problem. At the same time there are polynomially tractable ...
1
vote
1answer
61 views

How can I implement lasso in R using optim function

As you know lasso is a popular variable selection method of the form of $ (y-x\beta)'(y-X\beta)+\lambda \sum_i|\beta_i| $ the first is that it is possible to use optim() function in R to minimize ...
1
vote
1answer
59 views

Solving a scrambled image puzzle with a genetic algorithm

I want to solve a puzzle such as this one: by using a genetic algorithm. When the number of pieces grow, and maybe some are rotated, the number of combinations become overwhelming. I am hoping that ...
0
votes
0answers
14 views

Intuitive explanation of the proximal operator

Can someone please give an intuitive explanation of the proximal operator? I understand that it is somehow related or is a generalization of projection, but other than that, I'm unable to grasp it.
0
votes
1answer
22 views

Data point that minimizes all four parameters [duplicate]

First of all, I must say I'm in no way a data scientist or anything; I just happened to come across a problem, and I am attempting to use statistics to find the best solution. The problem is about ...
0
votes
1answer
31 views

How to solve nonlinear optimization problem in R

Suppose I have a set of data and reason to believe the following relation holds y ~ a0 + a1*x1 + a2*x2 + a3*log(x3) How can I use R to solve for the coefficients {a0, ... a3}, supposing I want to ...
0
votes
0answers
18 views

Meta-parameter search for elastic net regularization of general objective function

In their 2004 paper on elastic net regularization, Zou and Hastie present an efficient method for finding the meta-parameters by folding the $L_2$-regularization component into the OLS problem and ...
0
votes
0answers
42 views

stochastic network optimization

I'd like to optimize the flow of materials through a network. There are vertices (i.e. physical locations) and edges (i.e. links between the physical locations). Inputs: locations transactional ...
0
votes
1answer
51 views

Calibrating Generalized Hyperbolic distribution in R - which parameters are valid and allow for a numerical calculation of absolute moments

I am using the R-package ghyp in order to calibrate and model. In fact my coding is based on this paper. I know that I could do quite a robust fit using ...
0
votes
1answer
61 views

Estimating correlation hyperparameters of a Gaussian Process

I have an actual function that I need to simulate using a GP model. I've not done this before so I'm unclear of the steps. I have used the true function at different values of the inputs ($\vec X1, ...