0
votes
0answers
36 views

ML algorithm to find optimal control parameter

I have a training dataset $(X, y) \rightarrow z$. Where $X$ is an $n$th dimensional real vector, $y$ is an integer number in $\{1, 2, 3\}$, and $z$ is a real number. I am looking for machine learning ...
0
votes
0answers
14 views

Choosing the values of a proper subset of features to maximise regression tree output

Suppose I have a regression tree and feature set $X$. Suppose that the feature set is composed of $X:=\{X_0,X_1,...,X_{100}\}$, where each $X_i \sim N(0,\sigma^2)$. Suppose that ...
0
votes
0answers
19 views

Segmenting an interval sensibly

Is there a canonical/recommended approach to or algorithm for splitting up an interval with the intent of minimizing the number of segments while keeping a high accuracy? It is essentially an ...
2
votes
2answers
74 views

Example how maximizing and minimizing a function can be equivalent?

I don't understand how sometimes given an optimization problem, a function could get its optimal solution by minimizing or sometimes just by reformulation it becomes maximizing. Can you please give me ...
0
votes
1answer
44 views

Minimizing the norm of a vector of parameters

I'm reading a paper that defines a function $f_w(x)$ that takes input $x$ and parameters $w$ and a set of constraints. There are also training data. The aim is to find the set of parameters $w$ that ...
0
votes
1answer
44 views

Goldfarb Idnani quadratic solver

I am implementing the Support Vector Regression (SVR) algorithm by means of quadratic programming. In order to do that, I am using an optimization library that contains a quadratic solver based on the ...
0
votes
0answers
19 views

Yet another question on Elo++ updating rule

I am back to my early love, rating systems. I was reading again the paper of Sysmanis: How I won the "Chess Ratings - Elo vs the Rest of the World" Competition And I come to a new doubt. He is ...
1
vote
0answers
20 views

How different is training a factor graph with discriminative features?

Many of the people define graphical models with factors, each with 'conditional probability tables' (CPT) and perform inference on them. But more realistic case is when you can't define full ...
0
votes
0answers
14 views

what does Task in multitask learning mean?

I'm new in multitask learning. I didn't exactly understand what is Tasks in multitask learning ? every task are single training data or what ?
0
votes
0answers
30 views

intuitive understanding of max-min in optimization

I have problem in understanding max-min in optimization. I know what does it mean to maximize obejctive function or minimize it, I don't really understand what does it mean to have objective function ...
5
votes
0answers
70 views

Compressed sensing: Optimization in $L_1$ norm and total variation with fourier coefficients

I'm reading the article Robust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Frequency Information (Candes, Romberg and Tao, 2004). In this article they are talking ...
1
vote
0answers
38 views

Role of orthogonal constraints (X^TX=I) in linear discriminant analysis?

What is the role of the orthogonal constraints in Linear Discriminant Analysis? Why would it work/not work if the fraction of the traces is maximized without that constraint? Wouldn't its still ...
2
votes
2answers
65 views

Optimization through 'alternating' descent

While analysing a particular iterative method which reaches the fixed points of a multivariate function f(x, y) (x and y are N-dimensional vectors in this case), I was able to reformulate it in the ...
0
votes
2answers
72 views

How to find parameters in multivariate space efficiently?

I am trying to optimize sklearn.linear_model.SGDRegressor, and I was wondering if people could point me in which direction I should try to optimize? Personal experience and literature are both ...
0
votes
0answers
36 views

self organizig map with spectral clustering

I'm looking for implementation of self organizing map which for the clustering part uses spectral clustering rather than k-means. Does anyone knows something about it ?
1
vote
1answer
84 views

In stochastic gradient descent, is there only one update to $\theta$ for each iteration?

I have read that the update equation for stochastic gradient descent is as shown below, for each iteration, k. Does one iteration correspond to one training example? So for each example is there only ...
0
votes
0answers
97 views

Linear Discriminant (decision region) singly connected and convex?

I am reading about linear discriminants, and have encountered a phrase that I have no idea about. The phrase says that a decision region constructed in a certain way, is "singly connected and convex", ...
2
votes
1answer
123 views

Issues with implementing neural network

I am trying to use neural network to learn a non linear function mapping input to outputs. However, I am having some issues with it. I used tansig activation function for the hidden layers and for the ...
1
vote
1answer
72 views

Optimization parameter for classification [closed]

I do not have enough knowledge about optimization. My problem is simple. Lets say I have 100 classes. Each class contains some instances (images). The feature vector obtained from a particular image ...
1
vote
1answer
84 views

Variable cost - benefit analysis on random forest

I have a random forest being trained with n vectors each with m variables. Each variable has a cost based on how much time it takes to compute it (m1 might take 1 unit while m2 might take 100, making ...
1
vote
0answers
113 views

Why does k-NN perform better than SVR and linear regression?

I have a data set used in a regression with 30 attributes and 30K instances. I am trying out a bunch of algorithms (SMO regression, Linear Regression and K-NN) but it was quite surprising to see that ...
1
vote
1answer
77 views

Smart way to search through a very large parameter space

I have a system whose performance is based on a rather large parameter set (200 parameters, lets say, of which each can take a very wide range of values). There are tests to evaluate the performance ...
0
votes
1answer
70 views

Minimize a function with respect to a matrix

I have two sets of vectors, A and B. Vectors from set A live in an m-dimensional space, ...
0
votes
0answers
37 views

Is it possible for a learning algorithm to learn weights on a different scale?

I'm doing some simple regression. In my training I initialize my weights randomly. Then it converges to a minimum, but I noticed that depending on the initial weights, the algorithm seems to find the ...
3
votes
0answers
43 views

Reducing the dimension of an embedding

Let $O \in \mathbb R^{p\times m}$ be a data matrix of observations. Suppose we are given a model $\mu : \mathbb R^n \rightarrow \mathbb R^m$ which is able to approximately fit the observations. Fix ...
2
votes
1answer
201 views

Forecasting optimization techniques in fantasy baseball

I am currently trying to build a better forecasting model for my fantasy baseball roster. I currently am using commonly accepted projected season statistics (ZiPS from Fangraphs) to determine the ...
0
votes
0answers
45 views

Confusion related to L2 and L1 SVM

I have this confusion related to L1 and L2 svm. I was reading this paper I am attaching the screenshot and the part I didn't understand The part that I didn't understand how it was derived I ...
0
votes
0answers
41 views

coordinated dual descent method and sequential minimal optimization

Libsvm uses the sequential minimal optimization as its main solver while Liblinear uses coordinated dual descent method. What are the major differences between these two methods? Looks like both of ...
5
votes
2answers
196 views

How can one show a Kmeans solution is unique?

Suppose we are given a distribution P and a constant K. We wish to minimize the kmeans objective w.r.t centers ${C1,..Ck}$: What constraints on $P$ are known to imply that the optimal solution is ...
2
votes
1answer
241 views

Kmeans on “symmetric” data

A set is said to be fully-symmetric if for every x in it, negating one of its components results in y such that y is in the set as well. A set is said to be semi-symmetric if for every x in it, ...
0
votes
1answer
173 views

How do we usually select the best combination of parameters of a machine learning model (for a given dataset)?

Am I wrong, or the standard way of optimizing a machine learning model is by evaluating the algorithm over the (initial) dataset for all possible combinations of parameters, and then pick up the one ...
2
votes
1answer
166 views

In non-negative matrix factorization, are the coefficients of features comparable?

I'm using Alternating Nonnegative Least Squares Matrix Factorization Using Projected Gradient. The result (I use 2 as rank) is like this: ...
2
votes
0answers
200 views

Kernel SVM in primal training with Stochastic Gradient Descent

In short: I am currently reading Online Learning with Kernels (http://books.nips.cc/papers/files/nips14/AA33.pdf) for fun and I can't figure out how he got to equation 8 from equations 6 and 7. The ...
3
votes
1answer
119 views

SVM optimization problem

I think I understand the main idea in support vector machines. Let us assume that we have two linear separable classes and want to apply SVMs. What SVM is doing is that it searches a hyperplane ...
1
vote
1answer
396 views

Algorithm for price optimization [closed]

I'm trying to figure out a way for calculating price optimization in a commerce environment. In other words, I'm trying to analyze how a company can increase revenue and profitability by analyzing ...
1
vote
0answers
44 views

Enforcing sparsity on probability

I am trying to induce a probability distribution $Q$ by optimizing an objective function and am wondering how can one encourage sparsity for $Q$ while keeping the optimization convex. In particular, ...
1
vote
0answers
57 views

Representer theorem for vector-valued functions

Is there a representer theorem for loss-functions of the form $\sum_{i}(f(x_i \mathbb{.}),y_i)$ of the form where the output of $f(.)$ is a vector and the domain is also a vector. Also, there is a ...
2
votes
2answers
149 views

Learning classifier system frameworks

For some time already I am studiyng and applying machine learning methods. Recently for some particular problem, where methods like SVM, RF, neural nets etc. failed I got limited success with genetic ...
1
vote
0answers
106 views

How to understand / visualize the error surface in online learning algorithms

I have a question about the shape of the error surface for online gradient descent algorithms. Take into account that I am trying to translate my specific question into a more general and idealized ...
3
votes
1answer
122 views

On Elo++ updating rule

I am not sure if this is the correct place to ask this kind of question, I hope it is. I am studying this paper on an improvement of Elo rating system called Elo++. On page 4 the author states that he ...
2
votes
0answers
255 views

Genetic algorithms, genetic programming or machine learning algorithms for solving this problem

I have a problem that consists of finding the optimal solution based on the following criteria: Logic for identifying that event A has occurred (i.e. "find" logic that most accurately categorises an ...
2
votes
0answers
246 views

Efficient Portfolio Optimization Through Simulation

Apologies in advance for the (possibly?) poor terminology as I'm a bit of a novice in the field. I was torn whether to ask this on stackoverflow or here, so hope its the right place. Anyway, my ...
14
votes
5answers
809 views

Can you overfit by training machine learning algorithms using CV/Bootstrap?

This question may well be too open ended to get a definitive answer, but hopefully not. Machine learning algorithms, such as SVM, GBM, Random Forest etc, generally have some free parameters that, ...
0
votes
2answers
269 views

Finding weights for variables in kNN

I'm using euclidean distance for kNN. I have labeled data, I have took logarithm of some variables to make them look more like normaly distributed and scaled them all. And now I would like to multiply ...
14
votes
2answers
393 views

Are machine learning techniques “approximation algorithms”?

hi all recently there was a ML-like question over on cstheory stackexchange & I posted an answer recommending powells method, gradient descent or genetic algorithms or other "approximation ...
1
vote
0answers
343 views

Performance of the fuzzy c-means clustering algorithm

I have implemented a genetic algorithm for a fuzzy c-means clustering in Matlab. Its performance should be apriori better than that of the classic fuzzy c-means (fcm function in matlab). However, on ...
2
votes
1answer
672 views

Finding degree of polynomial in regression analysis

I am working on a machine learning project where I am trying to fit a curve on data. Unfortunately the date has somewhat high feature vector. So, I can't really plot them on a 2D or 3D space to guess ...
2
votes
0answers
74 views

Paired multiarm bandit

I have a set of independent experiments with different distributions and I'm trying to determine which has the highest mean payoff. I would like to treat this as a multi-arm bandit problem, but the ...
1
vote
1answer
114 views

Optimize a boolean function

I have some data that a downstream system needs an optimized function of boolean logic for. Essentially I have data similar to: ...
16
votes
2answers
633 views

How to choose between learning algorithms

I need to implement a program that will classify records into 2 categories (true/false) based on some training data, and I was wondering at which algorithm/methodology I should be looking at. There ...