In statistics this refers to selecting an estimator of a parameter by maximizing or minimizing some function of the data. One very common example is choosing an estimator which maximizes the joint density (or mass function) of the observed data referred to as Maximum Likelihood Estimation (MLE).

learn more… | top users | synonyms (2)

1
vote
0answers
14 views

Optimize number of layers and neurons with an optimization algorithm

I have a neural network that i want optimize number of hidden layers and neurons in every layer using an optimization algorithm like ...
0
votes
0answers
29 views

How to derive this problem with soft-thresholding method?

The problem is defined as $$ \min_{x} \Bigg\{ a{\|x\|}^2+\frac{b}{2}{\|x-c\|}^2 \Bigg\} $$ where $x\in R^{n \times 1}, c \in R^{n \times 1}$ and $a,b$ are scalars. Equations 2.5 to 2.8 of this paper ...
1
vote
0answers
35 views

Finding the most “uniform” or “least concentrated” density function, subject to moment constraints

Background I want to find a probability measure for a continuous random variable, subject to moment constraints, that is maximally "uniform", as defined below: Definition: Maximally Uniform ...
2
votes
1answer
50 views

Linear Regression with $L_2$: Different penalty strengths yield the same parameters?

Suppose I have model 1: $$Y=aX$$ where $X$ is $n\times1$, consisting of a single feature. suppose I fit this model with $L_2$ penalty with coefficient $\lambda=1$ : $$ \underset{a}{\min} ...
1
vote
1answer
11 views

Holt-Winters and Abnormal termination in LNSRCH

I try to fit data with Holt-Winters function in R. Nevertheless, i am getting the following message: ...
4
votes
2answers
56 views

Do optimization techniques map to sampling techniques?

From any generic sampling algorithm, one can derive an optimization algorithm. Indeed, to maximize an arbitrary function $f: \textbf{x} \rightarrow f(\textbf{x})$, it suffices to draw samples from $g ...
0
votes
1answer
30 views

Best optimization package for employee scheduling problem? [closed]

I am looking to solve the optimization problem described below. Which optimization software package would be best suited for this, considering the requirements specified below? Requirements: 1) Can ...
0
votes
1answer
41 views

Convert continues number to integer number in optimization algorithms in MATLAB

I'm using a continuous optimization algorithm for optimizing neural network's number of neurons in first and second layers besides feature selection so I used this structure for converting continues ...
2
votes
1answer
41 views

MAP estimate of posterior parameters

I have a setup where the joint posterior is written as: $$ P(w, \lambda, \phi \vert y) = P(\phi) \times P(w \vert \lambda) \times P(\lambda) \times \prod_{i=1}^{N}P(y_i \vert w_i, \phi, \lambda) $$ ...
2
votes
1answer
43 views

finding global minima of a random forest estimator

I have a random forest regression model with 1000 trees, having 16 parameters (using python scikit-learn). The estimator can predict a target value with cross validated r2 score of 0.87 +/- 0.03. I ...
1
vote
1answer
24 views

EM for Mixtures of Bernoulli (M-step)

When applying the M-step for a mixture of Bernoulli distributions, one of the parameters in our maximization is the Bernoulli parameter $\mu_{k}$, where $k$ is the index of the "mixture component", ...
1
vote
1answer
19 views

Optimise selection from a set, with constraints

I would like to use R to solve a problem I have. I don't even know what to call a problem of this kind and I'm finding Googling difficult. My guess is that this kind of problem already has R ...
0
votes
0answers
28 views

Parameter optimization of SVM

Currently I am using SVM to perform some classification task. I use libSVM with Matlab interface. From the practical guide of SVM (Link), we know that there are two parameters need to be tuned, namely ...
3
votes
1answer
39 views

Constrained Maximization and Likelihood Ratio Tests for Nested Linear Models

Suppose $\boldsymbol \beta \in \mathbb{R}^k$ is a vector of coefficients for a generalized linear model with $g \left[ E(Y|X) \right] = X\beta$ for a link function $g$ and I wish to test the composite ...
0
votes
1answer
41 views

What statistical method to correct systematic error in the output of a economic optimization model?

I am working with an economic optimization model which attempts to model the dynamics of a certain commodity market (prices, quantities, production etc.) for different frequencies (monthly, quarterly, ...
0
votes
0answers
13 views

Binary event probability optimization

I have a relatively small sample of binary events (50-100 events) that occurred during a time of day (the success rate is closely related to the time of day). I'm grouping these events into hour ...
0
votes
1answer
30 views

Conceptual question on optimization

What is the intuition and the physical meaning of the mathematical expression in convex optimization? When using optimization algorithms like particle swarm or genetic algorithm, do they have ...
1
vote
1answer
30 views

Finding optimal combination of parameters for clustering

I have a spreadsheet with one object per line. Each column contains values that are parameters of my objects (let's say length, width, height, weight, color). I can classify objects based on color and ...
1
vote
0answers
26 views

When does l1 regularisation give a sparse solution?

I was maximising a likelihood function, which is convex. I know that the system has a K-sparse solution. I wanted to know the conditions (or some sufficient conditions) on the likelihood function ...
0
votes
0answers
10 views

optimisation procedures before training SVM

I'm using the LIBSVM in Java for classification with 200 documents in inputs. I build/train the SVM using the same input training data. My response time for preprocessing of documents (tokenization, ...
0
votes
0answers
16 views

Analysis of full factorial with categorical dependent variable and blocking?

I'm working on a research project for which there is some proprietary information that I can't provide here. However, I will do my best to lay out as much information as I can. In this project we ...
0
votes
0answers
13 views

How to use regression analysis to set an optimal price

I am working on a side project with very small dataset where i am trying to figure out the optimal price i should set for a transaction fee (something like payPal). Currently i am using an arbitrary ...
0
votes
1answer
25 views

Constraint optimisation

Consider the constraint optimization $\text{argmin}_{\beta}(f(\beta)+\lambda g(\beta))$ can someone define $\beta(\lambda)$. That is, what is the relationship between $\lambda$ and $\beta$?.
0
votes
1answer
23 views

Marquardt Loglikelihood Calculation in Eviews

I paper I am trying to replicate used Eviews to estimate their state space model (by maximizing the associated maximum likelihood). They used the BHHH and Marquardt algorithms. My question is given ...
1
vote
0answers
27 views

How to fight underfitting in deep neural net

When I started with ANN I thought I'd have to fight overfitting as the main problem. But in practice I can't even get my NN to pass the 20% error rate barrier. I can't even nearly beat my score on ...
1
vote
1answer
35 views

Finding optimal hyperplane

I have a set of vectors $\{V_i\}$ in $n$-dimensional space. There is a number corresponded to each vector $\alpha_i = f(V_i)$ ($\alpha_i$ can be negative). I want to find a hyperplane which would ...
3
votes
0answers
110 views

Better estimator of expected sum than mean

I am trying to find the optimal estimator for the maximal expected $\Sigma X_i$ where $X_i$ is sampled from an unknown distribution which is chosen to be maximal. To clarify and simplify, there are ...
0
votes
1answer
29 views

How to use SVD for dimensionality reduction to reduce columns specifically?

My original data has many more columns (features) than rows (users). I'm trying to reduce the features of my SVD (I need all of the rows). I found one method of doing so in a book called "Machine ...
0
votes
1answer
27 views

What method to use for cluster identification ?

This question is from a confused novice. I have a data set with where each point is located in a 2-D space defined by two objectives (say, X and Y). I wish to identify a set of points from this space ...
0
votes
1answer
85 views

Drunken cockroach - Trying to meet expected value

Imagine that you have $1000 that you can split however you want. You bet in a cockroach run, but it is not the finish that's interesting. You can bet for the cockroach to go left or right, and you ...
0
votes
0answers
27 views

Standard Errors of Transformed Variables

I am carrying out an MLE where some I use a log transformation on the variance parameters which are being optimized. When I calculate the standard errors (se) the se of the transformed variables is ...
0
votes
0answers
13 views

Error running optim function with STAR from book example

I'm running an example of Smooth Transition AR (STAR) Model from the book "Analysis of financial time series, 3rd edition" by Tsay, in section 4.1.3. The script is as follows: ...
2
votes
1answer
41 views

Geometric interpretation of penalized linear regression II

An older question gives an intuitive explanation of how penalized linear regression works, using two separate contours: one for the least square objective, one for the penalty term (i.e. ...
0
votes
0answers
17 views

Fisher Scoring v/s Coordinate Descent for MLE in R

R base function glm() uses Fishers Scoring for MLE, while the glmnet uses the coordinate descent method to solve the same equation ? Coordinate descent is more time efficient than Fisher Scoring as ...
5
votes
1answer
170 views

Stochastic Programming with MCMC

I have just started learning about MCMC (using PyMC), and it seems to be a hammer that can be used to solve a large class of inference and optimization problems. While I understand that there are ...
0
votes
0answers
16 views

Difference between a stochastic and deterministic optimization problem?

I am reading a lot about classifying optimization problems. But i cannot find a real life example that treads the difference between a stochastic and deterministic optimization problem.
0
votes
0answers
18 views

Combining BHHH and Levenberg Marquardt

I already asked a question related to this here: When is Maximum Likelihood the same as Least Squares I know understand how Levenberg Marquardt (LM) can be applied to the objective function. In ...
0
votes
1answer
32 views

Sum of weights in portfolio theory is not equal to 1

I'm trying to understand basic portfolio theory using R. As far as I understood, the sum of the weights of assets must be equal to 1 . But in this link, that teaches how to compute the efficient ...
4
votes
1answer
65 views

When is Maximum Likelihood the same as Least Squares

In this paper on p315: http://www.ssc.upenn.edu/~fdiebold/papers/paper55/DRAfinal.pdf They explain that they use Levenberg Marquardt (LM) (along with BHHH) to maximize the likelihood. However as I ...
1
vote
1answer
27 views

Running simulated annealing optimization multiple times and averaging results?

I am using a simulated annealing algorithm to optimize a cost function. Given the fact that simulated annealing is a stochastic algorithm and will not give the same results each time you run it, I was ...
2
votes
0answers
36 views

Bayesian bandits with delayed rewards

I looked into the topic of bayesian bandits in order to create a simple testing tool for headline optimizations. UCB1 seemed easy enough until I discovered that there is probably a problem with the ...
0
votes
0answers
23 views

Efficient solution of fmincg without providing gradient?

I'm working on multiclass logistic regression model with a large number of features (numFeatures>100). Using a maximum likelihood estimation-based cost function and ...
0
votes
2answers
32 views

Evolution strategies in libsvm

I'm working on protein multi-classification problem. I'm using libsvm and the edit distance kernel. This kernel depends from a parameter (gamma). I'm able to get the best parameters (gamma and C) ...
1
vote
1answer
118 views

Maximum likelihood estimation for state space models using BFGS

I have written some code that can do Kalman filtering (using a number of different Kalman-type filters [Information Filter et al.]) for Linear Gaussian State Space Analysis for an n-dimensional state ...
0
votes
1answer
86 views

Need clarity on alpha, beta, gamma optimization in Triple Exponential Smoothing Forecast

I asked a variation of this question, but I want to be more direct. Take the exact same Triple Exponential Smoothing Model (Holt-Winters with a moving level, trend, and seasonal component)--- Would ...
0
votes
0answers
11 views

Best approach to relabelling to produce greatest concordance

A Bayesian clustering algorithm I am using assigns a group for each sample that is input. For different random seeds different results are obtained and the labelling of each group is different. I ...
3
votes
1answer
70 views

Clustering without a distance matrix

I've recently completed a project where I used scikit-learn's DBSCAN module to find clusters in relatively short strings of text. I used a custom string similarity ...
1
vote
0answers
14 views

Prioritization based on three factors

Background: Sales reps visit doctor and detail about a product/drug. One visit is termed as one call. In return he writes the prescriptions to doctors prescribing that particular drug. Problem ...
1
vote
2answers
55 views

Likelihood vs. noise kernel hyperparameter in GPML Toolbox

I'm using GPML toolbox by C.E.Rasmussen to solve the basic GP regression problem (presented in the book) with noisy observations. That is to say, estimate the underlying function $f$ of a static noisy ...
2
votes
1answer
141 views

Sparse Autoencoder [Hyper]parameters

I have just started using the autoencoder package in R. http://cran.r-project.org/web/packages/autoencoder/index.html Inputs to the autoencode() function include lambda, beta, rho and epsilon. What ...