# Tagged Questions

17 views

### How to find all optima in an optimization problem?

I have an optimization problem where several optima can exist at different input values, and I need to find as many as possible. As an example consider the cross-in-tray function, which has four ...
34 views

### Conceptual question on optimization

What is the intuition and the physical meaning of the mathematical expression in convex optimization? When using optimization algorithms like particle swarm or genetic algorithm, do they have ...
181 views

### Stochastic Programming with MCMC

I have just started learning about MCMC (using PyMC), and it seems to be a hammer that can be used to solve a large class of inference and optimization problems. While I understand that there are ...
14 views

### Prioritization based on three factors

Background: Sales reps visit doctor and detail about a product/drug. One visit is termed as one call. In return he writes the prescriptions to doctors prescribing that particular drug. Problem ...
27 views

### SVM Training: Working Set Selection

This is related to Joachims's 1998 paper on training SVMs (link to paper). In 11.3, I understand how the term $V(\mathbf d)$ arises as a result of a first order approximation, and why it needs to be ...
10 views

### Learning a multivariate polynomial with dependent coefficients

I have a polynomial of the form $K^2((a-i)^2 + (b-j)^2 + c^2) = (ct)^2$ where $a,b,c,t$ are unknowns. I have multiple observation points for the values of $i,j,\&\ K$. Can I use some technique ...
37 views

### Wilcoxon-Mann-Whitney as a loss function

I'm reading a paper where the authors are using Wilcoxon-Mann-Whitney loss function while minimizing an objective function. As the authors say in the paper, the role of the loss function is to give a ...
30 views

### The role of the regularization parameter while optimizing a function

I'm reading a paper where the aim is to optimize the following function: $h()$ is a loss function and $\lambda$ is a regularization parameter. The idea in $h()$ is that whenever $p_l-p_d > 0$ ...
19 views

### How the optimal separating Hyperplane can be constructed for a Support Vector Machine

i refer to http://www.markowetz.org/florian/FlorianMarkowetz_DiplMath_thesis.pdf on page 31-35 he tries to reformulate Vapniks proof that the primal an dual problem for the maximal margin hyperplane ...
48 views

### ML algorithm to find optimal control parameter

I have a training dataset $(X, y) \rightarrow z$. Where $X$ is an $n$th dimensional real vector, $y$ is an integer number in $\{1, 2, 3\}$, and $z$ is a real number. I am looking for machine learning ...
16 views

### Choosing the values of a proper subset of features to maximise regression tree output

Suppose I have a regression tree and feature set $X$. Suppose that the feature set is composed of $X:=\{X_0,X_1,...,X_{100}\}$, where each $X_i \sim N(0,\sigma^2)$. Suppose that ...
20 views

### Segmenting an interval sensibly

Is there a canonical/recommended approach to or algorithm for splitting up an interval with the intent of minimizing the number of segments while keeping a high accuracy? It is essentially an ...
130 views

### Example how maximizing and minimizing a function can be equivalent?

I don't understand how sometimes given an optimization problem, a function could get its optimal solution by minimizing or sometimes just by reformulation it becomes maximizing. Can you please give me ...
69 views

### Minimizing the norm of a vector of parameters

I'm reading a paper that defines a function $f_w(x)$ that takes input $x$ and parameters $w$ and a set of constraints. There are also training data. The aim is to find the set of parameters $w$ that ...
117 views

I am implementing the Support Vector Regression (SVR) algorithm by means of quadratic programming. In order to do that, I am using an optimization library that contains a quadratic solver based on the ...
28 views

### Yet another question on Elo++ updating rule

I am back to my early love, rating systems. I was reading again the paper of Sysmanis: How I won the "Chess Ratings - Elo vs the Rest of the World" Competition And I come to a new doubt. He is ...
25 views

### How different is training a factor graph with discriminative features?

Many of the people define graphical models with factors, each with 'conditional probability tables' (CPT) and perform inference on them. But more realistic case is when you can't define full ...
17 views

I'm new in multitask learning. I didn't exactly understand what is Tasks in multitask learning ? every task are single training data or what ?
37 views

### intuitive understanding of max-min in optimization

I have problem in understanding max-min in optimization. I know what does it mean to maximize obejctive function or minimize it, I don't really understand what does it mean to have objective function ...
90 views

### Compressed sensing: Optimization in $L_1$ norm and total variation with fourier coefficients

I'm reading the article Robust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Frequency Information (Candes, Romberg and Tao, 2004). In this article they are talking ...
40 views

### Role of orthogonal constraints (X^TX=I) in linear discriminant analysis?

What is the role of the orthogonal constraints in Linear Discriminant Analysis? Why would it work/not work if the fraction of the traces is maximized without that constraint? Wouldn't its still ...
73 views

### Optimization through 'alternating' descent

While analysing a particular iterative method which reaches the fixed points of a multivariate function f(x, y) (x and y are N-dimensional vectors in this case), I was able to reformulate it in the ...
89 views

### How to find parameters in multivariate space efficiently?

I am trying to optimize sklearn.linear_model.SGDRegressor, and I was wondering if people could point me in which direction I should try to optimize? Personal experience and literature are both ...
37 views

### self organizig map with spectral clustering

I'm looking for implementation of self organizing map which for the clustering part uses spectral clustering rather than k-means. Does anyone knows something about it ?
125 views

### In stochastic gradient descent, is there only one update to $\theta$ for each iteration?

I have read that the update equation for stochastic gradient descent is as shown below, for each iteration, k. Does one iteration correspond to one training example? So for each example is there only ...
156 views

### Linear Discriminant (decision region) singly connected and convex?

I am reading about linear discriminants, and have encountered a phrase that I have no idea about. The phrase says that a decision region constructed in a certain way, is "singly connected and convex", ...
153 views

### Issues with implementing neural network

I am trying to use neural network to learn a non linear function mapping input to outputs. However, I am having some issues with it. I used tansig activation function for the hidden layers and for the ...
85 views

### Optimization parameter for classification [closed]

I do not have enough knowledge about optimization. My problem is simple. Lets say I have 100 classes. Each class contains some instances (images). The feature vector obtained from a particular image ...
94 views

### Variable cost - benefit analysis on random forest

I have a random forest being trained with n vectors each with m variables. Each variable has a cost based on how much time it takes to compute it (m1 might take 1 unit while m2 might take 100, making ...
125 views

### Why does k-NN perform better than SVR and linear regression?

I have a data set used in a regression with 30 attributes and 30K instances. I am trying out a bunch of algorithms (SMO regression, Linear Regression and K-NN) but it was quite surprising to see that ...
91 views

### Smart way to search through a very large parameter space

I have a system whose performance is based on a rather large parameter set (200 parameters, lets say, of which each can take a very wide range of values). There are tests to evaluate the performance ...
77 views

### Minimize a function with respect to a matrix

I have two sets of vectors, A and B. Vectors from set A live in an m-dimensional space, ...
44 views

### Reducing the dimension of an embedding

Let $O \in \mathbb R^{p\times m}$ be a data matrix of observations. Suppose we are given a model $\mu : \mathbb R^n \rightarrow \mathbb R^m$ which is able to approximately fit the observations. Fix ...
279 views

### Forecasting optimization techniques in fantasy baseball

I am currently trying to build a better forecasting model for my fantasy baseball roster. I currently am using commonly accepted projected season statistics (ZiPS from Fangraphs) to determine the ...
213 views

### How can one show a Kmeans solution is unique?

Suppose we are given a distribution P and a constant K. We wish to minimize the kmeans objective w.r.t centers ${C1,..Ck}$: What constraints on $P$ are known to imply that the optimal solution is ...
248 views

### Kmeans on “symmetric” data

A set is said to be fully-symmetric if for every x in it, negating one of its components results in y such that y is in the set as well. A set is said to be semi-symmetric if for every x in it, ...
216 views

### How do we usually select the best combination of parameters of a machine learning model (for a given dataset)?

Am I wrong, or the standard way of optimizing a machine learning model is by evaluating the algorithm over the (initial) dataset for all possible combinations of parameters, and then pick up the one ...
215 views

### In non-negative matrix factorization, are the coefficients of features comparable?

I'm using Alternating Nonnegative Least Squares Matrix Factorization Using Projected Gradient. The result (I use 2 as rank) is like this: ...
262 views

### Kernel SVM in primal training with Stochastic Gradient Descent

In short: I am currently reading Online Learning with Kernels (http://books.nips.cc/papers/files/nips14/AA33.pdf) for fun and I can't figure out how he got to equation 8 from equations 6 and 7. The ...
131 views

### SVM optimization problem

I think I understand the main idea in support vector machines. Let us assume that we have two linear separable classes and want to apply SVMs. What SVM is doing is that it searches a hyperplane ...
500 views

### Algorithm for price optimization [closed]

I'm trying to figure out a way for calculating price optimization in a commerce environment. In other words, I'm trying to analyze how a company can increase revenue and profitability by analyzing ...
48 views

### Enforcing sparsity on probability

I am trying to induce a probability distribution $Q$ by optimizing an objective function and am wondering how can one encourage sparsity for $Q$ while keeping the optimization convex. In particular, ...
183 views

### Learning classifier system frameworks

For some time already I am studiyng and applying machine learning methods. Recently for some particular problem, where methods like SVM, RF, neural nets etc. failed I got limited success with genetic ...
129 views

### How to understand / visualize the error surface in online learning algorithms

I have a question about the shape of the error surface for online gradient descent algorithms. Take into account that I am trying to translate my specific question into a more general and idealized ...
141 views

### On Elo++ updating rule

I am not sure if this is the correct place to ask this kind of question, I hope it is. I am studying this paper on an improvement of Elo rating system called Elo++. On page 4 the author states that he ...
289 views

### Genetic algorithms, genetic programming or machine learning algorithms for solving this problem

I have a problem that consists of finding the optimal solution based on the following criteria: Logic for identifying that event A has occurred (i.e. "find" logic that most accurately categorises an ...
294 views

### Efficient Portfolio Optimization Through Simulation

Apologies in advance for the (possibly?) poor terminology as I'm a bit of a novice in the field. I was torn whether to ask this on stackoverflow or here, so hope its the right place. Anyway, my ...
974 views

### Can you overfit by training machine learning algorithms using CV/Bootstrap?

This question may well be too open ended to get a definitive answer, but hopefully not. Machine learning algorithms, such as SVM, GBM, Random Forest etc, generally have some free parameters that, ...