In statistics this refers to selecting an estimator of a parameter by maximizing or minimizing some function of the data. One very common example is choosing an estimator which maximizes the joint density (or mass function) of the observed data referred to as Maximum Likelihood Estimation (MLE).

learn more… | top users | synonyms (2)

0
votes
0answers
7 views

optimization - cost function matlab [on hold]

suppose I have a signal x1 always positive of length N = 1000. % signal x1 x=rand(1,1000); x(x<0)=0.01; I have also other 599 signals x2, x3,...,x600 (they ...
1
vote
0answers
11 views

Compare optimization algorithms with stop criteria

I'm comparing Harmony Search and Genetic Algorithm for a specific finance case study. I set a early stopping criteria. If we don't have improvement in 1/5 of total iterations of algorithm, It will ...
0
votes
0answers
13 views

Is there an advantage to using Metropolis-Hastings as opposed to Dijkstra's algorithm for pathfinding? [on hold]

I recently read a paper regarding the use of Metropolis-Hastings in finding shortest paths, ...
0
votes
0answers
17 views

Most-efficient/effective Incentive Scheme Design to Minimize Loan Default Probability [on hold]

Here is an open-ended, hypothetical question regarding the optimization of a loan incentive scheme. Any and all suggestions/plans are welcome. Please ask any clarifying questions if you wish: A loan ...
0
votes
0answers
25 views

How to solve equation by computer package? [closed]

I want to solve f(x)=0.9*exp(2000*x)+0.1*exp(15000*x)-5290*x-1 equation, by any computer package. I tried Excel Goal-Seek analysis, but it is not good for value ...
2
votes
0answers
43 views

Maximizing combined log likelihood from different dataset using r

I have observed data $y$. And I have a function that gives me estimated $\hat{y} = f(x,\hat{P})$ where P is the parameter I want to estimate. I was able to optim() command in R to get maximized ...
0
votes
1answer
20 views

Non linear optimization with objective function as a string [closed]

I am looking for a package to help me solve some non linear optimisation problems with constraints. The objective function f to optimize is given as a string. For example, f is given as the following ...
0
votes
0answers
33 views

Are there ways to improve Levenberg-Marquardt backpropogation performance in Neural Networks?

When using Levenberg-Marquardt optimization for a function approximation problem, the performance and speed generally trumps that of the gradient descent. I am approximating the functions cos(n * Pi) ...
1
vote
2answers
25 views

Recommended/estimated number of radial basis functions in RBFN

thank you for taking the time to read my question. I am attempting to make a Radial Basis Function Network to see if a relationship exists between input/output data that I have been collecting. I ...
1
vote
0answers
9 views

How can I find the bounds that gets Simulated Annealing to converge?

According to Wikipedia on Simulated Annealing, For any given finite problem, the probability that the simulated annealing algorithm terminates with a global optimal solution approaches 1 as the ...
1
vote
1answer
18 views

Confidence that Optimum Lies Within a Given Region

I have an ordinary least squares regression equation. Through calculus or simulation, I can find the combination of explanatory values that maximize the estimated mean response; I can also establish a ...
2
votes
3answers
98 views

Machine learning for discovering formulas

I have a vector of desired values that I want to fit based in some generated predictors. The tricky part is that I wish to have the explicit formula. For example, giving the below input I would like ...
0
votes
0answers
5 views

Seek C/C++ library to solve linear regression problem [migrated]

Is there some off-the-shelf C/C++ library to solve the following problem: Thanks in advance!
1
vote
1answer
19 views

How to find $\arg\max$ of a neural network?

Let's say I have a neural network $f$ that takes input $\vec x \in \mathbb {R}^n$ and produces output $f(\vec x) \in \mathbb{R}$. How can I find $\hat x = \underset{\vec x}{\arg\max} \; f(\vec x)$?
0
votes
0answers
14 views

Online EM?? Did any one implement online EM using matlab?

In Online EM algorithm in each iteration for each sample one estimates sufficient statistics and adds it to overall sufficient statistics and in maximization step one maximizes parameters to MLE for ...
0
votes
0answers
13 views

maximizing matrix diagonal elements in R [migrated]

Given a matrix (or table in R ) ...
4
votes
2answers
59 views

Expectation maxmisation algorithm increases true likelihood at each iteration

I've heard that the EM algorithm ensures that the true likelihood is non-decreasing at each iteration of the algorithm, but I'm not sure why this is the case. I've provided a basic plot which I ...
3
votes
1answer
57 views

What are the differences between various R quadratic programming solvers?

I am looking for a package to help me solve some quadratic optimisation problems and I see there are at least half a dozen different packages. According to this page: QP (Quadratic programming, ...
3
votes
1answer
24 views

What is the solution to this minimization problem?

I'm encountering the following minimization problem in my research: $$\hat b = \underset{b}{\arg\min} \sum_i^n \left( \log \frac{a_i}{b} \right)^2$$ I could iteratively optimize, but I think that ...
1
vote
0answers
17 views

Under what assumptions can parameter estimate uncertainty be estimated from the Hessian?

Given a model with some parameters, some data it's attempting to reproduce, and a distance function to quantify how well the predictions correspond to the data, I can fit parameters via a general ...
0
votes
1answer
15 views

Total Variation Denoising help

I am trying to work through the "Mathematical exposition for 1D digital signals" in the wikipedia entry for Total Variation Denoising (TVD). I am familiar with Lagrange multipliers. However, I cant ...
1
vote
0answers
14 views

Likelihood convexification

I am doing constrained vector optimization using a non-convex non-linear likelihood function. My problem is of the following form: $$\begin{align*}\hat Q &= \underset{\vec Q}{\arg\min} -\log ...
0
votes
0answers
20 views

Implementing evolutionary algorithms

We're trying to to minimize the following functions using Multi Objective Evolutionary Algorithms : $\mathrm{minimize}~\lambda_q(1-a)E(N)E(X)+\lambda_xE(N)E(\max(X/a-R,0))$ ...
0
votes
1answer
39 views

Clarification about Perceptron Rule vs. Gradient Descent vs. Stochastic Gradient Descent implementation

I experimented a little bit with different Perceptron implementations and want to make sure if I understand the "iterations" correctly. Rosenblatt's original perceptron rule As far as I understand, ...
0
votes
0answers
17 views

What is Tau and Omega in the Black Litterman model?

I'm looking into the BL model when it comes to portfolio optimization, and I'm having a hard time trying to understand each one. I've read on several papers that Omega is the covariance matrix, but I ...
0
votes
2answers
67 views

R optim function - Setting constraints for individual parameters

I am looking to minimize a function using optim as follows: ...
0
votes
1answer
60 views

R - Constrained optimization for a function taking a matrix

I would like to do constrained optimization for a function which takes a matrix as input. The matrix is sparse, representing a weighted adjacency matrix , and only the weights shall be subject to ...
2
votes
1answer
123 views

Constrained assignment problem (Linear Programming, Genetic Algorithm, etc…)

I'm looking for advice on how I should approach a specific problem. I have about 1000 shops that I have to assign to about 20 different supply centers out of a possible 28, and I'm trying to pick the ...
1
vote
1answer
30 views

On the usage of numerical optimization technique to maximize log-likelihood

Literature and resources say that when the ML log-likelhood does not have a closed form expression, then we can use Newton-Raphson and other optimization techniques. My Question is: During estimation ...
0
votes
1answer
62 views

Selecting optimization algorithms

I am in the process of developing a multi-state model (using the msm package in R), and have been encountering the error message initial value in 'vmmin' is not finite This error occurs ...
0
votes
0answers
42 views

Choose the correct regression model

The relation of two measurement values x and y might be linear mod1 <- lm(x~y) ...
0
votes
0answers
14 views

SGD weight learning for mixture of generative models

Let's say that I've learned $k$ generative models $\mathbf{G}=G_i$ in some ways from my data. I'd like to create a mixture from them in order to have something like model averaging in this form: ...
3
votes
1answer
69 views

Designing Asymmetric regression (assymettric loss for regression)

I have a hybrid classification/regression problem.The predicted value can be assumed to be centred around 0. I want to penalize the predictor more, if the predicted value and actual value have ...
1
vote
1answer
18 views

Finding all solutions of a multidimensional nonlinear equation

I have a problem in which I have two (though there can be more) multivariate Gaussian functions very close to each other and I want to find whether being next together causes the sum of these two ...
1
vote
1answer
86 views

Nonlinear Autoregressive model parameter estimation from time series

I'm working on a nonlinear multivariate autoregressive model of order 1 (markovian). It is a discrete-time dynamical system which models exchange of mass between compartments in a compartmental model ...
0
votes
0answers
26 views

Maximum likelihood method

I try to calibration the parameters $\theta$ of my probabilistic model from available (limited) information at some point $\mathbf{x}_i, i=(1,...,n)$ on a 3D space defined by $[0, M]^3 \subset ...
0
votes
1answer
122 views

What is the connection between regularization and the method of lagrange multipliers ?

To prevent overfitting people people add a regularization term (proportional to the squared sum of the parameters of the model) with a regularization parameter $\lambda$ to the cost function of linear ...
0
votes
0answers
14 views

Pairwise elastic net Algorithms

I am interested in minimizng the following $f(x)=x^{T}Rx + \sum_{j=1}^{N}\sum_{i=1}^{N}A_{i,j}|x_{i}||x_{j}| +\sum_{i=1}^{N} |x_{i}|$ where the $R,A$ are positive definite and $A_{i,j}>0$ I ...
0
votes
0answers
7 views

confidence intervals in multivariate nonlinear model [duplicate]

I am recently start to use this site which is very helpful for learning purpose.I am basically engineer and facing the problem that is related to construction of confidence intervals .Mathematically ...
0
votes
0answers
27 views

Marketing offer optimization

I would like to set up a campaign for 2 products (2 offers each) Product A - offer 1 - 25% off Product A - offer 2 - 50% off Product B - offer 3 - 25% off Product B - offer 4 - 50% off Data ...
5
votes
1answer
289 views

Default lme4 optimizer requires lots of iterations for high-dimensional data

TL;DR: lme4 optimization appears to be linear in the number of model parameters by default, and is way slower than an equivalent ...
2
votes
0answers
111 views

A smarter way for evaluating combinations of samples to optimize an overall score

I have the following problem (which I simplified for clarity) that I want to find k out of n samples (here: rows) that yield the maximum score (sum of the values in ...
2
votes
1answer
65 views

how to differentiate matrix

How can I differentiate the following by $\mathbf{W}$ ? \begin{equation} \mathbf{Y} = (\mathbf{W}^T\mathbf{x} + b)^2 \end{equation} Where $\mathbf{W} \in \mathcal{R}^{d\times D}$ and ...
3
votes
1answer
53 views

maximum a posteriori vs squared loss

I am unclear about max a posteriori and squared loss. Let me assume I have $N$ images and $\mathbf{y}_i$ is the label of the image $i$, where, $\mathbf{y}_i\in \mathbb{R}^{C\times 1}$ - a binary ...
4
votes
2answers
195 views

Statistical analysis in the cryptoanalysis of the Enigma cipher machine (safeguarding of the intelligence source in Ultra)

During the second world war the British were doing cryptanalysis of the German Enigma cipher machine and subsequent signals intelligence. One of the issue was about the safeguarding of the source of ...
1
vote
1answer
60 views

optimal down payment estimation in credit scoring

Knowing I can estimate the risk of default, via logistic regression, of a consumer on a small loan... what would be the best way to estimate the optimal down-payment amount to ask for in order to ...
0
votes
0answers
34 views

Optimal number of workers for small business model

I'm new here. I formed an analytical model of a small scale business where the expense can be defined as $C_L=(1-T).(1-1/n)$ and production rate can be defined as $R=1/(1-T+T/n)$. Where ...
2
votes
0answers
94 views

Minimizing KL divergence from a given distribution, according to a graph

Given $n$ discrete random variables $X_1,...,X_n$, a distribution $p$ on $X=(X_1,...,X_d)$ and a DAG (Directed Acyclic Graph) $G$ on $\{1,...,d\}$, which is the distribution $q$ factorizing with $G$ ...
0
votes
1answer
14 views

Compare convergence of optimization methods

I need to quantify how 2 optimization methods differ in convergence. When training a neural network I get the following plots, which show an error function after each gradient update. I think the ...
2
votes
1answer
130 views

How do CNN's avoid the vanishing gradient problem

I have been reading alot about convoloutional neural networks and was wondering how they avoid the vanishing gradient problem. I know deep belief networks stack single level auto-encoders or other ...