Tagged Questions

In statistics this refers to selecting an estimator of a parameter by maximizing or minimizing some function of the data. One very common example is choosing an estimator which maximizes the joint density (or mass function) of the observed data referred to as Maximum Likelihood Estimation (MLE).

learn more… | top users | synonyms (2)

0
votes
0answers
8 views

How to find the optimal estimates for a function [on hold]

I have a function such like this: ...
0
votes
0answers
9 views

Fitting strategy for nonlinear least squares with penalty

I have a model I'd like to fit which has the form $Y = \alpha + \beta_1 * f(x_1; \theta_1) + \beta_2 * f(x_2; \theta_2)$ The $f$'s are complicated, nonlinear, and non convex transformations applied ...
0
votes
1answer
14 views

Lagrange: Difference between minimizing and maximizing

The following seems straight forward: 1) Maximize f(x1, x2) = x1x2 subject to h(x1, x2) ≡ x1 + 4x2 = 16. Form the Lagrangian: L(x1, x2) = x1x2 − λ (x1 + 4x2 − 16) The first order conditions are (1) ...
0
votes
0answers
12 views

Faster gradient descent convergence by transforming the gradient?

If we modify the gradient descent update for a convex objective function $f(\boldsymbol{\theta})$ from $\boldsymbol{\theta}_{t+1} = \boldsymbol{\theta}_t - \nabla f(\boldsymbol{\theta}_t)$ to ...
0
votes
0answers
5 views

Optimizing grid of sources and multipliers

I have a 35x35 grid of cells. There are two possible states for cells: "source" and a "multiplier". The center 25x25 cells may be either "source" or "multiplier". The rest of the cells must be ...
0
votes
0answers
31 views

What's the most efficient way to sort a list of relative values

I've searched long and hard and have yet to find a viable solution for the following problem. Thank you in advance for your time. Let's suppose I have a list of entities {a,b,c,.. aa,ab,.. zy,zz} ...
0
votes
0answers
22 views

Optimize parameters?

I've been back-testing a trading strategy that has two parameters (both are # of days to look back), and I've tested the system for robustness by comparing my 10-year Sharpe ratio based on approx ...
0
votes
0answers
24 views

forcing certain parameters to be skipped during optim in R

I have a code which tests each possible order of ARIMA and selects the best model by choosing the one with the absolute minimum sum of lags from the PACF graph. The code then proceeds to add weight to ...
0
votes
1answer
24 views

Constraining norms with inequalities [closed]

I have time-series data for N stocks. sample.data<-rep(10,rnorm(100)), where each column shows the returns of different stocks over time. I am trying to ...
0
votes
0answers
12 views

Exponential distribution of interval censored data MLE using optim on R error

I have a set of data which is interval censored and I wish to calculate its maximum likelihood error using optim. This is my code: ...
2
votes
1answer
48 views

Linear regression with an inequality constraint

I am looking for an efficient way of finding a linear fit $Mx = y$ subject to an inequality constraint: $\frac{|x_2|}{\sqrt{x_3^2 + x_4^2}} \geq a$, with $a \geq 1$. The rectangular matrix $M$ is ...
1
vote
1answer
29 views

Why is k-medians typically used with Manhattan rather than Euclidean distance?

K-medians is typically used with Manhattan distance rather than Euclidean distance. Why is this?
0
votes
0answers
4 views

Optimal intertemporal allocation of bets based on estimated returns

Each day, an investor predicts the day's stock-market return in the morning, generating a point estimate and prediction interval, and chooses to bet an amount $B_t$ on the market that day based on her ...
3
votes
0answers
32 views

Why is 'relaxed LASSO' different from LASSO?

If we start with a set of data $(X,Y)$, apply LASSO to it and obtain a solution $\beta^L$, we can apply LASSO again to the data set $(X_S, Y_S)$, where $S$ is the set of non-zero indexes of $\beta^L$, ...
0
votes
1answer
40 views

Are there optimization methods for k-NN parameter $k$?

Currently I am just go through from the min to the max, and determine $k$ by the performance. I am wondering if there's optimization approaches for selecting $k$? I am aware of there's question in ...
1
vote
1answer
22 views

Optimizing for target metrics in Weka

I'm a PhD student in Information Retrieval with some limited experience in ML. We've been working on a binary classification task with weka (I'm using weka programmatically via Java), specifically ...
0
votes
0answers
43 views

Regression Analysis and Optimization in R

I have the following table: ...
0
votes
0answers
21 views

Why we even try to minimize a loss function, which is non-convex, in matrix factorization?

In matrix factorization (especially under the scenario of recommendation system), we often try to factorize a matrix Y into two low rank matrices: $Y=U\cdot V^T$ If we assume there $m$ instances in ...
0
votes
0answers
9 views

fitting a non-linear curve with one parameter

I have an equation: $\ddot{x}+(\delta+\epsilon\cos{t})x=0$ known as the Mathieu equation.The $\delta-\epsilon$ parameter space of this equation looks something like The red lines in this diagram ...
0
votes
0answers
14 views

What happens if you replace the sampling procedure in MCMC with maximization when fitting a HMM?

When using Markov chain monte carlo to fit hidden Markov models, after you use the forward algorithm to obtain the posterior distribution, you sample the hidden states for the current observation. ...
0
votes
0answers
13 views

Optimizing selection from varying sets

On pages of website(s) I have a set of potential messages to choose from and only one or two slots to show them in. (think 'this product is on sale' or 'this product is new'). On each page the set ...
1
vote
0answers
32 views

Solving for GLM coefficients w/ Newton-Raphson

Note : This is a question about a homework problem I am facing. I have some data that is to be modeled with a logistic regression model. I am supposed to do two things: (1) use newton-raphson ...
0
votes
0answers
19 views

How important is the quality of solutions to NP-hard problems arising in machine learning problems?

Machine learning and inference problems give rise to intractable problems. For instance the exact inference in Bayes nets is an NP-hard problem. At the same time there are polynomially tractable ...
1
vote
1answer
48 views

How can I implement lasso in R using optim function

As you know lasso is a popular variable selection method of the form of $ (y-x\beta)'(y-X\beta)+\lambda \sum_i|\beta_i| $ the first is that it is possible to use optim() function in R to minimize ...
1
vote
1answer
50 views

Solving a scrambled image puzzle with a genetic algorithm

I want to solve a puzzle such as this one: by using a genetic algorithm. When the number of pieces grow, and maybe some are rotated, the number of combinations become overwhelming. I am hoping that ...
0
votes
0answers
12 views

Intuitive explanation of the proximal operator

Can someone please give an intuitive explanation of the proximal operator? I understand that it is somehow related or is a generalization of projection, but other than that, I'm unable to grasp it.
0
votes
1answer
22 views

Data point that minimizes all four parameters [duplicate]

First of all, I must say I'm in no way a data scientist or anything; I just happened to come across a problem, and I am attempting to use statistics to find the best solution. The problem is about ...
0
votes
1answer
26 views

How to solve nonlinear optimization problem in R

Suppose I have a set of data and reason to believe the following relation holds y ~ a0 + a1*x1 + a2*x2 + a3*log(x3) How can I use R to solve for the coefficients {a0, ... a3}, supposing I want to ...
0
votes
0answers
13 views

Meta-parameter search for elastic net regularization of general objective function

In their 2004 paper on elastic net regularization, Zou and Hastie present an efficient method for finding the meta-parameters by folding the $L_2$-regularization component into the OLS problem and ...
0
votes
0answers
42 views

stochastic network optimization

I'd like to optimize the flow of materials through a network. There are vertices (i.e. physical locations) and edges (i.e. links between the physical locations). Inputs: locations transactional ...
0
votes
1answer
46 views

Calibrating Generalized Hyperbolic distribution in R - which parameters are valid and allow for a numerical calculation of absolute moments

I am using the R-package ghyp in order to calibrate and model. In fact my coding is based on this paper. I know that I could do quite a robust fit using ...
0
votes
0answers
48 views

Estimating correlation hyperparameters of a Gaussian Process

I have an actual function that I need to simulate using a GP model. I've not done this before so I'm unclear of the steps. I have used the true function at different values of the inputs ($\vec X1, ...
1
vote
0answers
43 views

Finding a global minimum of non-convex quasi-smooth function that is costly to evaluate

I have a bounded non-convex function in 10-dimensional space. The function is quasi-smooth, you can imagine a histogram, here is an illustration, it just shows the idea and not related to my ...
0
votes
1answer
22 views

Multimodal objective optimization

What is the meaning of multimodal objective optimization? Could you provide an example? What is the difference when compared with multi-objective optimization, which would provide solutions along ...
1
vote
0answers
10 views

Basis for a certain class of iterative algorithms [closed]

I have seen the following algorithm trick in a few places. Suppose that you have some closed form equation that you would like to solve, of the form $A=F(A)$, where $F$ may be rather complicated and ...
0
votes
1answer
27 views

find the optimal correspondence matrix

I have two sets of points and I find with different methods, different correspondence matrices which shows which point in one set correspond to other point in the other set. How could I find the ...
1
vote
0answers
18 views

Multiple objective allocation function

I have an allocation problem where, for a given good, I have buyer $i$ willing to buy up to quantity $b_{s,i}$ and seller $j$ willing to sell up to quantity $s_{s,i}$. There are $N$ buyers and $M$ ...
0
votes
0answers
70 views

Newton Raphson Over-estimates Parameters

I have implemented an almost plain vanilla algorithm to find the MLE estimates of 3 parameters in a log-likelihood function (in R.) When I test my algorithm with some simulated data it does pretty ...
1
vote
0answers
122 views

Solve systems of non-linear equations

How could try to solve this system of linear equations. I made a code in R trying to solve it, that might make me suggestions please. I want to estimate $\lambda$, $\beta$ and $\alpha$. ...
0
votes
0answers
35 views

Optimally distributing sports tickets to customers

I work for a business that has a certain number of sports tickets to distribute to customers and potential customers every year, and I've been asked to develop a model that will give insight into ...
1
vote
1answer
36 views

Recommended packages for numerical optimization with symbolic calculus

I'd like to train a model $\widehat{y}_i = F(x_i, \theta)$, by minimizing the sum of a loss function, $L(\widehat{y}_i, y_i, \theta)$. I'd like to input $\{x_i, y_i\}, F, L$ into a software package ...
0
votes
0answers
13 views

What should be the fitness function while using Particle Swarm optimisation

I am using Particle Swarm Optimisation for optimising the parameters of a Neural network (for multi-class classification problem). But what should be the fitness function for it ? I have tried ...
2
votes
1answer
47 views

How to find all optima in an optimization problem?

I have an optimization problem where several optima can exist at different input values, and I need to find as many as possible. As an example consider the cross-in-tray function, which has four ...
0
votes
0answers
18 views

Discrete optimization with a very large solution neighborhood to explore

I have a problem whose feasible (discrete) solutions can measured by a cost function. I am thinking of using some optimization technique to get better solutions from a rough initial approximation. I ...
1
vote
1answer
28 views

How to visualize a set of many optimizations of posterior simulations of an objective function?

I started by fitting a model: $y = f(X) + \epsilon$. The model includes random effects and coefficients -- there is a lot of heterogeneity in the population (and the data is longitudinal). I then ...
0
votes
0answers
13 views

Multiplicative gradient descent?

The normal gradient descent is additive: $w_{t+1}=w_t-\lambda_t\nabla f(w_t)$, but is there a multiplicative gradient descent that looks something like $w_{t+1}=w_t[-\lambda_t\nabla f(w_t)]$? I know ...
0
votes
0answers
33 views

Is there any optimization algorithms for these problems?

I have modeling the problem with the following equation: $$ \min_{X} L(X)=f(X)-\alpha g(X) + \beta k(X) $$ where $\alpha \gt 0, \beta \gt 0$, and, in my cases, I found the $f(X)-\alpha g(X) \lt 0$ and ...
0
votes
0answers
19 views

Reconstruct a vector with a known vector and residual

I observe $\vec y$ and know $\vec x$. I assume that $\vec y$ mostly consists of $\vec x$, with some added residual $\vec r$. This gives me the problem $\vec y = a\vec x + \vec r$, where $a \in [0, ...
1
vote
0answers
30 views

Lagrange Multipliers in practice

Say we want to minimize the function $f^2({\bf{x}})$, under the constraint $g({\bf{x}})=0$. The classic solution (Method I) is to introduce a Lagrange Multiplier, and solve: $$\frac{\partial ...
2
votes
2answers
114 views

Optimal bin width for two dimensional histogram

There are lots of rules for selecting an optimal bin width in a 1D histogram (see for example) I'm looking for a rule that applies the selection of optimal equal-bin widths on two-dimensional ...