In statistics this refers to selecting an estimator of a parameter by maximizing or minimizing some function of the data. One very common example is choosing an estimator which maximizes the joint density (or mass function) of the observed data referred to as Maximum Likelihood Estimation (MLE).

learn more… | top users | synonyms (2)

0
votes
0answers
10 views

Analysis of full factorial with categorical dependent variable and blocking?

I'm working on a research project for which there is some proprietary information that I can't provide here. However, I will do my best to lay out as much information as I can. In this project we ...
0
votes
0answers
11 views

How to use regression analysis to set an optimal price

I am working on a side project with very small dataset where i am trying to figure out the optimal price i should set for a transaction fee (something like payPal). Currently i am using an arbitrary ...
0
votes
0answers
13 views

Approximation of Gauss Hypergeometric function [closed]

I have a non-convex optimization problem due to this Hypergeometric function: $_2F_1(a,k;a+b;z)$. I am looking for an approximation of this function with convex function(s).Any idea ? Thanks!
0
votes
1answer
25 views

Constraint optimisation

Consider the constraint optimization $\text{argmin}_{\beta}(f(\beta)+\lambda g(\beta))$ can someone define $\beta(\lambda)$. That is, what is the relationship between $\lambda$ and $\beta$?.
0
votes
1answer
20 views

Marquardt Loglikelihood Calculation in Eviews

I paper I am trying to replicate used Eviews to estimate their state space model (by maximizing the associated maximum likelihood). They used the BHHH and Marquardt algorithms. My question is given ...
1
vote
0answers
18 views

How to fight underfitting in deep neural net

When I started with ANN I thought I'd have to fight overfitting as the main problem. But in practice I can't even get my NN to pass the 20% error rate barrier. I can't even nearly beat my score on ...
1
vote
1answer
35 views

Finding optimal hyperplane

I have a set of vectors $\{V_i\}$ in $n$-dimensional space. There is a number corresponded to each vector $\alpha_i = f(V_i)$ ($\alpha_i$ can be negative). I want to find a hyperplane which would ...
3
votes
0answers
97 views

Better estimator of expected sum than mean

I am trying to find the optimal estimator for the maximal expected $\Sigma X_i$ where $X_i$ is sampled from an unknown distribution which is chosen to be maximal. To clarify and simplify, there are ...
0
votes
1answer
23 views

How to use SVD for dimensionality reduction to reduce columns specifically?

My original data has many more columns (features) than rows (users). I'm trying to reduce the features of my SVD (I need all of the rows). I found one method of doing so in a book called "Machine ...
0
votes
1answer
22 views

What method to use for cluster identification ?

This question is from a confused novice. I have a data set with where each point is located in a 2-D space defined by two objectives (say, X and Y). I wish to identify a set of points from this space ...
0
votes
1answer
83 views

Drunken cockroach - Trying to meet expected value

Imagine that you have $1000 that you can split however you want. You bet in a cockroach run, but it is not the finish that's interesting. You can bet for the cockroach to go left or right, and you ...
0
votes
0answers
25 views

Standard Errors of Transformed Variables

I am carrying out an MLE where some I use a log transformation on the variance parameters which are being optimized. When I calculate the standard errors (se) the se of the transformed variables is ...
-5
votes
0answers
26 views

Work And Hour Optimization [closed]

Problem is to optimize the "Optimize Work" column such that sum of "Optimized Hours (i.e [Optimize Work]* [Hours(Per Item)])" column remain equal to 120 and "Optimize Work" can maximum reach up to ...
0
votes
0answers
12 views

Error running optim function with STAR from book example

I'm running an example of Smooth Transition AR (STAR) Model from the book "Analysis of financial time series, 3rd edition" by Tsay, in section 4.1.3. The script is as follows: ...
2
votes
1answer
39 views

Geometric interpretation of penalized linear regression II

An older question gives an intuitive explanation of how penalized linear regression works, using two separate contours: one for the least square objective, one for the penalty term (i.e. ...
0
votes
0answers
14 views

Fisher Scoring v/s Coordinate Descent for MLE in R

R base function glm() uses Fishers Scoring for MLE, while the glmnet uses the coordinate descent method to solve the same equation ? Coordinate descent is more time efficient than Fisher Scoring as ...
3
votes
0answers
46 views

Stochastic Programming (e.g. LP) with MCMC

I have just started learning about MCMC (using PyMC), and it seems to be a hammer that can be used to solve a large class of inference and optimization problems. While I understand that there are ...
0
votes
0answers
10 views

Difference between a stochastic and deterministic optimization problem?

I am reading a lot about classifying optimization problems. But i cannot find a real life example that treads the difference between a stochastic and deterministic optimization problem.
0
votes
0answers
15 views

Combining BHHH and Levenberg Marquardt

I already asked a question related to this here: When is Maximum Likelihood the same as Least Squares I know understand how Levenberg Marquardt (LM) can be applied to the objective function. In ...
0
votes
1answer
28 views

Sum of weights in portfolio theory is not equal to 1

I'm trying to understand basic portfolio theory using R. As far as I understood, the sum of the weights of assets must be equal to 1 . But in this link, that teaches how to compute the efficient ...
4
votes
1answer
60 views

When is Maximum Likelihood the same as Least Squares

In this paper on p315: http://www.ssc.upenn.edu/~fdiebold/papers/paper55/DRAfinal.pdf They explain that they use Levenberg Marquardt (LM) (along with BHHH) to maximize the likelihood. However as I ...
1
vote
1answer
23 views

Running simulated annealing optimization multiple times and averaging results?

I am using a simulated annealing algorithm to optimize a cost function. Given the fact that simulated annealing is a stochastic algorithm and will not give the same results each time you run it, I was ...
2
votes
0answers
35 views

Bayesian bandits with delayed rewards

I looked into the topic of bayesian bandits in order to create a simple testing tool for headline optimizations. UCB1 seemed easy enough until I discovered that there is probably a problem with the ...
0
votes
0answers
8 views

Efficient solution of fmincg without providing gradient?

I'm working on multiclass logistic regression model with a large number of features (numFeatures>100). Using a maximum likelihood estimation-based cost function and ...
0
votes
2answers
30 views

Evolution strategies in libsvm

I'm working on protein multi-classification problem. I'm using libsvm and the edit distance kernel. This kernel depends from a parameter (gamma). I'm able to get the best parameters (gamma and C) ...
1
vote
1answer
109 views

Maximum likelihood estimation for state space models using BFGS

I have written some code that can do Kalman filtering (using a number of different Kalman-type filters [Information Filter et al.]) for Linear Gaussian State Space Analysis for an n-dimensional state ...
0
votes
1answer
48 views

Need clarity on alpha, beta, gamma optimization in Triple Exponential Smoothing Forecast

I asked a variation of this question, but I want to be more direct. Take the exact same Triple Exponential Smoothing Model (Holt-Winters with a moving level, trend, and seasonal component)--- Would ...
0
votes
0answers
11 views

Best approach to relabelling to produce greatest concordance

A Bayesian clustering algorithm I am using assigns a group for each sample that is input. For different random seeds different results are obtained and the labelling of each group is different. I ...
-1
votes
0answers
36 views

Curve Fitting - Objective functions

This is more of an open ended question: I am currently doing some non-linear curve fitting of different types of curves to some noisy data (gaussian and lorentzian peaks). I use the simplest ...
3
votes
1answer
50 views

Clustering without a distance matrix

I've recently completed a project where I used scikit-learn's DBSCAN module to find clusters in relatively short strings of text. I used a custom string similarity ...
1
vote
0answers
12 views

Prioritization based on three factors

Background: Sales reps visit doctor and detail about a product/drug. One visit is termed as one call. In return he writes the prescriptions to doctors prescribing that particular drug. Problem ...
1
vote
2answers
39 views

Likelihood vs. noise kernel hyperparameter in GPML Toolbox

I'm using GPML toolbox by C.E.Rasmussen to solve the basic GP regression problem (presented in the book) with noisy observations. That is to say, estimate the underlying function $f$ of a static noisy ...
2
votes
1answer
95 views

Sparse Autoencoder [Hyper]parameters

I have just started using the autoencoder package in R. http://cran.r-project.org/web/packages/autoencoder/index.html Inputs to the autoencode() function include lambda, beta, rho and epsilon. What ...
2
votes
1answer
19 views

SVM Training: Working Set Selection

This is related to Joachims's 1998 paper on training SVMs (link to paper). In 11.3, I understand how the term $V(\mathbf d)$ arises as a result of a first order approximation, and why it needs to be ...
0
votes
0answers
9 views

Measures to take if activation in neural network gets saturated

I am coding an auto encoder with mnist data. It have 784 inputs, 50 hidden units, 784 outputs. Since outputs are huge in number, when error gets propagated during gradient descent, error of hidden ...
0
votes
0answers
8 views

Learning a multivariate polynomial with dependent coefficients

I have a polynomial of the form $K^2((a-i)^2 + (b-j)^2 + c^2) = (ct)^2$ where $a,b,c,t$ are unknowns. I have multiple observation points for the values of $i,j,\&\ K$. Can I use some technique ...
1
vote
0answers
23 views

EM Algorithm - Expectation w.r.t. a subset of current parameters

Suppose I want to make inference on a parameter vector $\theta $=$(\theta_{1},\theta_{2},\theta_{3})$ and I have some missing data $Y_{mis}$. I would like to use the EM algorithm to find the mode of ...
2
votes
1answer
70 views

Is there a package that I can use in order to get rules for a target outcome in R

For example In this given data set I would like to get the best values of each variable that will yield a pre-set value of "percentage" : for example I need that the value of "percentage" will be ...
2
votes
2answers
47 views

Naive Line Search for Gradient Descent

I'm trying to understand the Line Search approach to Gradient Descent (http://en.wikipedia.org/wiki/Line_search). It seems that a naive implementation would ...
3
votes
2answers
67 views

Gradient descent based minimization algorithm that doesn't require initial guess to be near the global optimum

The problem with gradient descent algorithms, e.g. the Levenberg–Marquardt algorithm, is that they will converge into a minimum that is nearest to the initial guess, so when starting from different ...
0
votes
0answers
13 views

TSP optimisation problem

What is the number of possible routes for a symmetric TSP? is it (n-1)!/2 ? Because sometimes I am finding that it is n! and sometimes (n-1)! and /2 since it is symmetric
0
votes
0answers
35 views

Minimum Number of Variables for Effective use of the Genetic Algorithm

From my understanding of the genetic algorithm the population consists of individuals, where each individual is a potential solution made up of "genes", and each gene is a variable. So for a cost ...
6
votes
1answer
112 views

Gaussian mixture regression in higher dimensions

Problem: I have a discrete representation of a surface/height-map $z = f(x,y)$ that i want to model as a mixture of gaussians (please take probability distributions out of your mind for a moment). ...
0
votes
0answers
25 views

Maximize Sortino Ratio in R

I'm trying to find portfolio weights, which will maximize my Sortino Ratio. I have weekly log-returns on 16 stocks. I've used all day yesterday to install and implement the following example on my own ...
0
votes
0answers
21 views

What statistical model for price optimisation

I'm not an expert so please be understanding if I don't formalize the problem in a professional/correct way. I have a data set with more than 160.000 offers that were made to the users of a website. ...
-1
votes
1answer
60 views

NP hard implementation optimisation using Monte Carlo method

I need to implement an algorithm ( or find an implementation) and optimise it using Monte Carlo method. This must be an NP hard such as the Travelling Salesman problem or the Knapsack problem. How can ...
1
vote
0answers
16 views

computing KL divergence: M projections for arbitrary distributions

Background I have a generative model for a process that can be described as follows: $$ y = t(x, w) + e $$ where $x$ and $y$ observations of a set of random variables which are related by a ...
1
vote
0answers
30 views

Wilcoxon-Mann-Whitney as a loss function

I'm reading a paper where the authors are using Wilcoxon-Mann-Whitney loss function while minimizing an objective function. As the authors say in the paper, the role of the loss function is to give a ...
1
vote
0answers
45 views

Alternatives to constrained non-linear optimization: a case study with Gaussian curve fitting

I have a set of data points distributed like a bell-shape curve with varying amplitudes and widths. I need to parameterize these data points and a Gaussian with amplitude, width and offset parameters ...
2
votes
2answers
166 views

Kalman Filtering, Smoothing and Parameter Estimation for State Space Models in R and C#

I am in the process of writing an open source State Space Analysis suite in C# (for fun). I have implemented a number of different Kalman-Based Filters (Kalman Filter, Information Filter and the ...