# Tagged Questions

In statistics this refers to selecting an estimator of a parameter by maximizing or minimizing some function of the data. One very common example is choosing an estimator which maximizes the joint density (or mass function) of the observed data referred to as Maximum Likelihood Estimation (MLE).

24 views

### Is EM feasible when there is no closed form maximization of the expectation of log likelihood?

In every example I've seen of expectation maximization, the E step concludes with an expression of the expectation of log likelihood ( $Q(\theta | \theta^{(t)})$ ) for which a maximum w.r.t. $\theta$ ...
35 views

### Does constrained EM algorithm work with bad initial inputs?

When trying to perform constrained optimization using EM algorithm, does EM work if the initial solution (x0) violates the constraints?
18 views

### Pythagorean Theorem to Optimize Multiple Variables?

New to stats, so I'm not sure if this is an already established thing or something I just made up that feels good. I have a list of board games that I'm interested in buying based on their price, ...
33 views

### Maximum likelihood estimation, how to derive the hessian

I am reading a paper and trying to understand how the authors estimated the standard errors of a set of parameter estimates $[\delta \ \ \phi \ \ \Sigma]$. Below is the loglikelihood function (sorry I ...
55 views

### What are common ill-conditioned problems/mistakes in linear regression? [closed]

Many problems are ill-conditioned and cannot be reliably solved using double precision computer systems. Here is what I know that could happen in linear model, that the problem is ill-conditioned and ...
21 views

### Summation constraint matrix

I'm trying to create an assignment script using the R package Rsymphony, but I can't seem to figure out how to convert the summations into an actual constraint matrix. I tried to figure it out based ...
140 views

### How do iterative methods for solving maximizing likelihood problems work?

Does anyone know about the computational iteration processes for maximum likelihood estimation? If these set of equations cannot be solved practically then how does the computer solve them?
19 views

### Numerical optimization of Monte-Carlo simulated moments

I hope not to post an off-topic related question here as it is related to programming, however, I am faced to a specific problem which I 1) tackle in a statistically oriented language and 2) which I ...
16 views

### Scheduling grid optimization

I am trying to optimize the programming of multiple TV channels for a given week. For each show (a day, a time and a TV show) it is possible to forecast in advance the number of people that will watch ...
9 views

### Repeated sampling via stochastic approximation to estimate noise

In stochastic approximation algorithms, one is interested in finding zeroes or extrema of functions which cannot be computed directly and that we can only observe via noisy observations. To accomplish ...
38 views

### Step-by-step example of reverse-mode automatic differentiation

Not sure if this question belongs here, but it's closely related to gradient methods in optimization, which seems to be on-topic here. Anyway, feel free to migrate if you think some other community ...
14 views

### Alternative Applications of Portfolio Optimization [closed]

What other statistical optimizations in the natural and social sciences require the maximization of the difference between the mean and the variance? In other words have an objective function (...
469 views

### Why are symmetric positive definite (SPD) matrices so important?

I know the definition of symmetric positive definite (SPD) matrix, but want to understand more. Why are they so important, intuitively? Here is what I know. What else? For a given data, Co-...
12 views

### Inbound/Outbound calls integer programming

For modeling on call center schedule optimization with inbound calls, I can use library(lpSolve) from R to get the solutions by ...
19 views

42 views

### Finding bounds for function parameter

I have a function $f$, for the sake of simplicity it may be unidimensional function (but what if it's multi-dimensional?). I am interested in finding some value parameter $\theta$ that maximizes it. I ...
26 views

### Is the EM-algorithm the same thing that variational inference in LDA?

I am new in the probabilistic topic modeling, and I need to understand deeply the LDA process, I understand what want to do the inference process in LDA, and I understand too that there is 2 "types" ...
65 views

### Find best parameter values: minimise some while maximizing others (in R)?

I have several variables (A, B, C) which are independent statistical measures. Each of these vary for different values of my function's parameter X. The plots of A,B,C look something like this: I ...
95 views

### Constrained optimization in R

My data is categorized by two different parameters (say F having n groups and S having m groups) and I want to get a relationship between the two. For example $F =${$f_1 , f_2 , f_3$} = {$10,10,5$} ...
10 views

### Choosing a cluster with low variance and many data points

I have data points that have been grouped using k-means clustering. Some of these clusters may only have one data point, which would give them a variance of zero. But I am more interested in the ...
9 views

### How can I cluster data points according to the local minima they belong to?

I'm using the genetic algorithm for hyperparameter optimisation. My loss function is the cross-validated loss, that means I can evaluate my loss function but I don't know how it looks like (the shape)....
8 views

### Method for a continuous variable for each sub-level of an ordinal variable?

I have three variables. A dependent variable (y) which is continuous, one dependent variable that is ordinal (x1), and one continuous variable which is bound (between 0 and 1, x2). I would like to ...
32 views

### Optimization depending on initial start values

I'm running a maximum likelihood of a logit regression, but the estimated parameters value and the loglikelihood value are depending on the value of the algorithm's start. For example, if my start is ...
26 views

### Optimizing a function available only through (monte-carlo) stochastic approximation

I am working on a problem where I want to estimate the maximum of a density that I can, in practice, evaluate (pointwisely) using a Monte-Carlo approach (because of intractable integrals). Obviously, ...
94 views

### Convergence of L-BFGS in non-convex settings

Is it true that generally L-BFGS may not converge in non-convex settings even if learning rate is really small? For example here L-BFGS diverges, but there are theoretical guarantees on its local ...