0
votes
0answers
17 views

How to find all optima in an optimization problem?

I have an optimization problem where several optima can exist at different input values, and I need to find as many as possible. As an example consider the cross-in-tray function, which has four ...
0
votes
1answer
34 views

Conceptual question on optimization

What is the intuition and the physical meaning of the mathematical expression in convex optimization? When using optimization algorithms like particle swarm or genetic algorithm, do they have ...
5
votes
1answer
181 views

Stochastic Programming with MCMC

I have just started learning about MCMC (using PyMC), and it seems to be a hammer that can be used to solve a large class of inference and optimization problems. While I understand that there are ...
1
vote
0answers
14 views

Prioritization based on three factors

Background: Sales reps visit doctor and detail about a product/drug. One visit is termed as one call. In return he writes the prescriptions to doctors prescribing that particular drug. Problem ...
2
votes
1answer
27 views

SVM Training: Working Set Selection

This is related to Joachims's 1998 paper on training SVMs (link to paper). In 11.3, I understand how the term $V(\mathbf d)$ arises as a result of a first order approximation, and why it needs to be ...
0
votes
0answers
10 views

Learning a multivariate polynomial with dependent coefficients

I have a polynomial of the form $K^2((a-i)^2 + (b-j)^2 + c^2) = (ct)^2$ where $a,b,c,t$ are unknowns. I have multiple observation points for the values of $i,j,\&\ K$. Can I use some technique ...
1
vote
0answers
37 views

Wilcoxon-Mann-Whitney as a loss function

I'm reading a paper where the authors are using Wilcoxon-Mann-Whitney loss function while minimizing an objective function. As the authors say in the paper, the role of the loss function is to give a ...
1
vote
1answer
30 views

The role of the regularization parameter while optimizing a function

I'm reading a paper where the aim is to optimize the following function: $h()$ is a loss function and $\lambda$ is a regularization parameter. The idea in $h()$ is that whenever $p_l-p_d > 0$ ...
0
votes
0answers
19 views

How the optimal separating Hyperplane can be constructed for a Support Vector Machine

i refer to http://www.markowetz.org/florian/FlorianMarkowetz_DiplMath_thesis.pdf on page 31-35 he tries to reformulate Vapniks proof that the primal an dual problem for the maximal margin hyperplane ...
0
votes
0answers
48 views

ML algorithm to find optimal control parameter

I have a training dataset $(X, y) \rightarrow z$. Where $X$ is an $n$th dimensional real vector, $y$ is an integer number in $\{1, 2, 3\}$, and $z$ is a real number. I am looking for machine learning ...
0
votes
0answers
16 views

Choosing the values of a proper subset of features to maximise regression tree output

Suppose I have a regression tree and feature set $X$. Suppose that the feature set is composed of $X:=\{X_0,X_1,...,X_{100}\}$, where each $X_i \sim N(0,\sigma^2)$. Suppose that ...
0
votes
0answers
20 views

Segmenting an interval sensibly

Is there a canonical/recommended approach to or algorithm for splitting up an interval with the intent of minimizing the number of segments while keeping a high accuracy? It is essentially an ...
2
votes
2answers
130 views

Example how maximizing and minimizing a function can be equivalent?

I don't understand how sometimes given an optimization problem, a function could get its optimal solution by minimizing or sometimes just by reformulation it becomes maximizing. Can you please give me ...
0
votes
2answers
69 views

Minimizing the norm of a vector of parameters

I'm reading a paper that defines a function $f_w(x)$ that takes input $x$ and parameters $w$ and a set of constraints. There are also training data. The aim is to find the set of parameters $w$ that ...
0
votes
1answer
117 views

Goldfarb Idnani quadratic solver

I am implementing the Support Vector Regression (SVR) algorithm by means of quadratic programming. In order to do that, I am using an optimization library that contains a quadratic solver based on the ...
0
votes
0answers
28 views

Yet another question on Elo++ updating rule

I am back to my early love, rating systems. I was reading again the paper of Sysmanis: How I won the "Chess Ratings - Elo vs the Rest of the World" Competition And I come to a new doubt. He is ...
1
vote
0answers
25 views

How different is training a factor graph with discriminative features?

Many of the people define graphical models with factors, each with 'conditional probability tables' (CPT) and perform inference on them. But more realistic case is when you can't define full ...
0
votes
0answers
17 views

what does Task in multitask learning mean?

I'm new in multitask learning. I didn't exactly understand what is Tasks in multitask learning ? every task are single training data or what ?
0
votes
0answers
37 views

intuitive understanding of max-min in optimization

I have problem in understanding max-min in optimization. I know what does it mean to maximize obejctive function or minimize it, I don't really understand what does it mean to have objective function ...
5
votes
0answers
90 views

Compressed sensing: Optimization in $L_1$ norm and total variation with fourier coefficients

I'm reading the article Robust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Frequency Information (Candes, Romberg and Tao, 2004). In this article they are talking ...
1
vote
0answers
40 views

Role of orthogonal constraints (X^TX=I) in linear discriminant analysis?

What is the role of the orthogonal constraints in Linear Discriminant Analysis? Why would it work/not work if the fraction of the traces is maximized without that constraint? Wouldn't its still ...
2
votes
2answers
73 views

Optimization through 'alternating' descent

While analysing a particular iterative method which reaches the fixed points of a multivariate function f(x, y) (x and y are N-dimensional vectors in this case), I was able to reformulate it in the ...
0
votes
2answers
89 views

How to find parameters in multivariate space efficiently?

I am trying to optimize sklearn.linear_model.SGDRegressor, and I was wondering if people could point me in which direction I should try to optimize? Personal experience and literature are both ...
0
votes
0answers
37 views

self organizig map with spectral clustering

I'm looking for implementation of self organizing map which for the clustering part uses spectral clustering rather than k-means. Does anyone knows something about it ?
2
votes
1answer
125 views

In stochastic gradient descent, is there only one update to $\theta$ for each iteration?

I have read that the update equation for stochastic gradient descent is as shown below, for each iteration, k. Does one iteration correspond to one training example? So for each example is there only ...
0
votes
1answer
156 views

Linear Discriminant (decision region) singly connected and convex?

I am reading about linear discriminants, and have encountered a phrase that I have no idea about. The phrase says that a decision region constructed in a certain way, is "singly connected and convex", ...
2
votes
1answer
153 views

Issues with implementing neural network

I am trying to use neural network to learn a non linear function mapping input to outputs. However, I am having some issues with it. I used tansig activation function for the hidden layers and for the ...
1
vote
1answer
85 views

Optimization parameter for classification [closed]

I do not have enough knowledge about optimization. My problem is simple. Lets say I have 100 classes. Each class contains some instances (images). The feature vector obtained from a particular image ...
1
vote
1answer
94 views

Variable cost - benefit analysis on random forest

I have a random forest being trained with n vectors each with m variables. Each variable has a cost based on how much time it takes to compute it (m1 might take 1 unit while m2 might take 100, making ...
1
vote
0answers
125 views

Why does k-NN perform better than SVR and linear regression?

I have a data set used in a regression with 30 attributes and 30K instances. I am trying out a bunch of algorithms (SMO regression, Linear Regression and K-NN) but it was quite surprising to see that ...
1
vote
1answer
91 views

Smart way to search through a very large parameter space

I have a system whose performance is based on a rather large parameter set (200 parameters, lets say, of which each can take a very wide range of values). There are tests to evaluate the performance ...
0
votes
1answer
77 views

Minimize a function with respect to a matrix

I have two sets of vectors, A and B. Vectors from set A live in an m-dimensional space, ...
3
votes
0answers
44 views

Reducing the dimension of an embedding

Let $O \in \mathbb R^{p\times m}$ be a data matrix of observations. Suppose we are given a model $\mu : \mathbb R^n \rightarrow \mathbb R^m$ which is able to approximately fit the observations. Fix ...
2
votes
1answer
279 views

Forecasting optimization techniques in fantasy baseball

I am currently trying to build a better forecasting model for my fantasy baseball roster. I currently am using commonly accepted projected season statistics (ZiPS from Fangraphs) to determine the ...
5
votes
2answers
213 views

How can one show a Kmeans solution is unique?

Suppose we are given a distribution P and a constant K. We wish to minimize the kmeans objective w.r.t centers ${C1,..Ck}$: What constraints on $P$ are known to imply that the optimal solution is ...
2
votes
1answer
248 views

Kmeans on “symmetric” data

A set is said to be fully-symmetric if for every x in it, negating one of its components results in y such that y is in the set as well. A set is said to be semi-symmetric if for every x in it, ...
0
votes
1answer
216 views

How do we usually select the best combination of parameters of a machine learning model (for a given dataset)?

Am I wrong, or the standard way of optimizing a machine learning model is by evaluating the algorithm over the (initial) dataset for all possible combinations of parameters, and then pick up the one ...
2
votes
1answer
215 views

In non-negative matrix factorization, are the coefficients of features comparable?

I'm using Alternating Nonnegative Least Squares Matrix Factorization Using Projected Gradient. The result (I use 2 as rank) is like this: ...
2
votes
0answers
262 views

Kernel SVM in primal training with Stochastic Gradient Descent

In short: I am currently reading Online Learning with Kernels (http://books.nips.cc/papers/files/nips14/AA33.pdf) for fun and I can't figure out how he got to equation 8 from equations 6 and 7. The ...
3
votes
1answer
131 views

SVM optimization problem

I think I understand the main idea in support vector machines. Let us assume that we have two linear separable classes and want to apply SVMs. What SVM is doing is that it searches a hyperplane ...
1
vote
1answer
500 views

Algorithm for price optimization [closed]

I'm trying to figure out a way for calculating price optimization in a commerce environment. In other words, I'm trying to analyze how a company can increase revenue and profitability by analyzing ...
1
vote
0answers
48 views

Enforcing sparsity on probability

I am trying to induce a probability distribution $Q$ by optimizing an objective function and am wondering how can one encourage sparsity for $Q$ while keeping the optimization convex. In particular, ...
2
votes
2answers
183 views

Learning classifier system frameworks

For some time already I am studiyng and applying machine learning methods. Recently for some particular problem, where methods like SVM, RF, neural nets etc. failed I got limited success with genetic ...
1
vote
0answers
129 views

How to understand / visualize the error surface in online learning algorithms

I have a question about the shape of the error surface for online gradient descent algorithms. Take into account that I am trying to translate my specific question into a more general and idealized ...
3
votes
1answer
141 views

On Elo++ updating rule

I am not sure if this is the correct place to ask this kind of question, I hope it is. I am studying this paper on an improvement of Elo rating system called Elo++. On page 4 the author states that he ...
2
votes
0answers
289 views

Genetic algorithms, genetic programming or machine learning algorithms for solving this problem

I have a problem that consists of finding the optimal solution based on the following criteria: Logic for identifying that event A has occurred (i.e. "find" logic that most accurately categorises an ...
2
votes
0answers
294 views

Efficient Portfolio Optimization Through Simulation

Apologies in advance for the (possibly?) poor terminology as I'm a bit of a novice in the field. I was torn whether to ask this on stackoverflow or here, so hope its the right place. Anyway, my ...
15
votes
5answers
974 views

Can you overfit by training machine learning algorithms using CV/Bootstrap?

This question may well be too open ended to get a definitive answer, but hopefully not. Machine learning algorithms, such as SVM, GBM, Random Forest etc, generally have some free parameters that, ...
0
votes
2answers
329 views

Finding weights for variables in kNN

I'm using euclidean distance for kNN. I have labeled data, I have took logarithm of some variables to make them look more like normaly distributed and scaled them all. And now I would like to multiply ...
14
votes
2answers
480 views

Are machine learning techniques “approximation algorithms”?

hi all recently there was a ML-like question over on cstheory stackexchange & I posted an answer recommending powells method, gradient descent or genetic algorithms or other "approximation ...