The tag has no wiki summary.

learn more… | top users | synonyms

0
votes
0answers
3 views

Liblinear types of solver

There is many variants of type of solver in liblinear but I don't understand their differences.Which one I must choose? Also why data must be scaled? duo to some numerical issues? ...
2
votes
1answer
18 views

Interpretation of logloss value

Does anyone have a interpretation of a logloss value? Am I correct to assume that values closer to 0 and 1 are more likely to be an indication that the predicted value is incorrect? Thanks.
0
votes
1answer
33 views

Forcing a particular false positives rate in a learning algorithm

I have a learning algorithm that classifies points as 0 or 1 (haven't settled on which one to implement yet). Of the points I classify as 1, I want to ensure that the number of points correctly ...
3
votes
1answer
106 views

Comparable traing and test cross-entropies result in very different accuracies

Premises I'm training a convolutional neural network (ConvNet) on 51 subclasses in the ImageNet dataset. In order to keep an eye on overfitting, I have been suggested to plot training and testing ...
2
votes
0answers
33 views

What is a robust way to find the max of $n$ independent, non-identical random variates?

Suppose I observe $n$ random variates along with their variance (but not mean) and I'd like to select the one with the largest mean as frequently as possible. The procedure must be memoryless--you ...
1
vote
0answers
16 views

Relation between Scoring rule and Loss function in Parameter estimation and model selection?

Initially, I had only heard of MLE and use it for almost everything, e.g. point estimate and model selection (with some penalty). Then, MSE appeared, which seems to play the same role as MLE does. I ...
3
votes
1answer
36 views

Question about computing Bayes Error - with or without loss function?

I am new to Bayesian Decision Theory and don't understand the following concept: So from what I understood, the Bayes error is used to report the performance of a Bayes classifier in terms of the ...
0
votes
1answer
47 views

How to design a cost function that has different weights for different types of classification errors?

I'm trying to design a continuous loss function for a logistic classifier. Suppose I have the following confusion matrix: [tn fp fn tp] I want the loss ...
0
votes
0answers
28 views

Properties of the Hellinger distance as a loss function

I am thinking about different loss function for non-parametric estimators (of densities). It is often said that $L^2$ loss is sensitive to outliers and doesn't do a good job of representing the tail ...
2
votes
3answers
119 views

Importance of optimizing the correct loss function

I want to understand the importance of optimizing the correct loss function. Say that I am building a linear regression model $p$ for predicting some values $y_1,\ldots,y_n$. I choose to fit my ...
0
votes
0answers
13 views

Loss specific belief propagation

We know standard Belief Propagation finds the parameters which maximize probability of posterior. Is there any way to use BP, for loss specific inference? For example, let's say someone wants to ...
0
votes
0answers
25 views

Maximizing F1-measure, when you have an algorithm for minimizing another loss function

Let's say you have an algorithm for minimizing the following loss function: $$ loss = \sum_i l(y_i, f(x_i)) $$ Let's say you are in the binary classification case, and the ratio of negative to ...
2
votes
1answer
58 views

Minimalizing absolute cost function error

I got two questions. 1. I know that in predictive analytics contests, when faced to yes/no problems, with the absolute cost function $f(x) = \frac{1}{n}\sum_{i=1}^n\lvert x_i-\hat x_i\rvert$ the ...
3
votes
1answer
114 views

LASSO with L1 loss function

I've been trying to figure out a way to perform LASSO with L1 loss function (instead of the L2 loss) but have been completely dumfounded as to how. I've attempted to use the flare package's ...
1
vote
1answer
42 views

Is the F-1 score symmetric?

Below is the report of my out-of-bag precision, recall and f-1score when using ...
0
votes
0answers
43 views

Loss function of kernlab

I'm looking for the default loss function the ksvm()-function from R package 'kernlab' is using. My guess is the Hinge Loss but I cant find any reference or citation for it (There's nothing ...
4
votes
1answer
127 views

What is the loss function of hard margin SVM?

People says soft margin SVM use hinge loss function: $\max(0,1-y_i(w^\intercal x_i+b))$. However, the actual objective function that soft margin SVM tries to minimize is $$ ...
0
votes
0answers
41 views

Comparing different types of losses as functions of lambda?

The usual pictures we see when dealing with different loss functions look similar to this: Here we see y*f(x) on the x-axis with an error associated with it. Suppose I have a logistic regression ...
0
votes
1answer
79 views

What is the loss function for C - Support Vector Classification?

In article LIBSVM: A Library for Support Vector Machines there is written, than C-SVC uses loss function: $$ \frac{1}{2}w^Tw+C\sum\limits_{i=1}^l\xi_i$$ OK, I know, what is $w^Tw$. But what is ...
0
votes
0answers
37 views

Bayesian Expected Loss for mean F1-score loss function

So I have a multi-label classification problem where the exact number of labels in each test set example is unknown. The loss function is mean F1-score which is where p is the precision and r is ...
2
votes
0answers
54 views

Regression analysis with special cost function

I want to do regression analysis using a special cost function that penalizes the sign error more than the square of the error. For example, I have a number of monthly change observations that can ...
0
votes
0answers
80 views

Why isn't k-means optimized using gradient descent?

I know k-means is usually optimized using Expectation Maximization. However we could optimize its loss function the same way we optimize any other! I found some papers that actually use stochastic ...
2
votes
0answers
17 views

Risk bounds for Poisson regression

Defining risk of the estimator as the expected square loss, is there any work discussing the non-asymptotic risk bounds for Poisson regression based on maximum likelihood estimate (MLE)?
2
votes
0answers
52 views

Transforming an estimator to overestimate

I have an estimator $\mu^*$ of a the mean $\mu$ of a certain distribution that I obtained using a variational technique (basically just establishing a bound on $\mu$ and finding a trial function that ...
1
vote
0answers
50 views

SVM with quadratic loss

I've seen some statement where I got the impression that SVM with a quadratic loss is no more than having a kernel matrix where a multiple of the unit matrix is subtracted from the kernel. It was ...
0
votes
2answers
149 views

Minimizing the expected loss

I was wondering about the motivation behind the following definition of expected loss: $$E[L] = \sum_{k} \sum_{j} \int_{R_{j}} L_{kj} p(x, C_{k})dx$$ where $L_{kj}$ is the loss matrix, in which $j$ ...
3
votes
2answers
169 views

Minimizing variance of an estimator under sampling cost penalty

I have an estimator $t$, whose variance depends on the dimension of my sample $x_{1:n}$: $$ \text{Var}(t(x_{1:n})) = f(n). $$ Suppose that the form of $f(n)$ is known. I would like to determine what ...
1
vote
0answers
77 views

A “Gambler's Loss function”?

What is a good loss function for a predictive model used by gamblers? I've been reading a bit about loss functions recently. I've always just went with MSE (e.g., for a couple of neural network ...
4
votes
1answer
112 views

Expectation notations

In Statistical Decision Theory, one often studies the following two measures (from "The Bayesian Choice"): Average loss (aka the frequentist risk): $R\left(\theta,\delta\right) = ...
1
vote
0answers
57 views

Representer theorem for vector-valued functions

Is there a representer theorem for loss-functions of the form $\sum_{i}(f(x_i \mathbb{.}),y_i)$ of the form where the output of $f(.)$ is a vector and the domain is also a vector. Also, there is a ...
1
vote
0answers
62 views

Can I define an $R^2$-like measure in this way when predicting with exotic loss functions?

It is common in regression to see $R^2$ formulated as follows: $$R^2\equiv 1 - {SS_{\rm err}\over SS_{\rm tot}},$$ where $SS_\text{err}=\sum_i (y_i - f_i)^2$ and $SS_\text{tot}=\sum_i ...
2
votes
0answers
109 views

Notation in GBM package vignette: expected value of loss functions

Can anyone help with the understanding of this notation (and idea) from the vignette for GBM in R? It starts with the following: Question 1: I believe this is simply saying that we are looking for ...
11
votes
2answers
410 views

How to design and implement an asymmetric loss function for regression?

Problem In regression one usually computes the mean squared error (MSE) for a sample: $$ \text{MSE} = \frac{1}{n} \sum_{i=1}^n\left(g(x_i) - \widehat{g}(x_i)\right)^2 $$ to measure the quality of a ...
6
votes
1answer
771 views

L1 regression estimates median whereas L2 regression estimates mean?

So I was asked a question on which central measures L1 (i.e., lasso) and L2 (i.e., ridge regression) estimated. The answer is L1=median and L2=mean. Is there any type of intuitive reasoning to this? ...
2
votes
0answers
98 views

Asymptotics of 0-1 classification loss

I am interested in training a simple binary linear classifier. That is, I will find a vector of weights $\bf w$ such that I can predict the class of new example by the sign of $f(x) = w^T x$. I ...
2
votes
0answers
157 views

Cramer-Rao type bound for Information Gain

I am interested in the Bayes risk of some distribution $\pi$ $$ r(\pi) = \mathbb{E}_{\pi(x)}[ \mathbb{E}_{\Pr(y|d,x)}[L(x,\hat x(y|d))]], $$ where $L$ is some loss function and $\hat x$ is the ...
2
votes
1answer
68 views

Convexity of loss function with respect to the mean

Let $X\,$ be a non-negative r.v. with known pdf $f(x|\theta)$ but with a single unknown parameter $\theta$. Suppose that the mean $\mu$ can be used to uniquely determine the value of $\theta$, i.e. if ...
0
votes
1answer
289 views

Changing logistic regression's loss function

We're using logistic regression to predict events probability. Logistic regression tries to minimize the residual variance (sum of squared residuals). However, in our specific problem we would like ...
2
votes
0answers
126 views

Does cross validation work with asymmetric loss functions?

My simple question is does cross validation work with an asymmetric loss function? I cannot find docs on google to answer.
2
votes
3answers
203 views

Multiclass classification when class distribution is known

What is an example of an algorithm that, when i have a known distribution across discrete groups and I have some sort of model score that a person is in each group, assigns persons to groups such that ...
1
vote
1answer
96 views

Weighted loss function for non-random sample

When comparing a regression estimation method (Y vs X) I currently use a weighted squared loss function: $$ \int_{-\infty}^{\infty}(\hat{f}(x)-f(x))^2 \, \hat{p}(x) \, dx $$ Where $\hat{f}(x)$ ...
1
vote
2answers
314 views

Matching loss function for tanh units in a neural net

There's not much more I can add to the question. Googling has mostly turned up research papers on springerlink and other sites I don't have access to. Given a neural network model with $tanh(x)$ as ...
3
votes
1answer
376 views

Dual problem for L2 support vector machine

Here is the dual problem for L2 support vector machine: $$\max_{\alpha\in\mathbb{R}^{n}} 2\alpha^{T}y-\alpha^{T}\left(K+n\lambda Id_{\mathbb{R}^{n}}\right)\alpha$$ $$\forall i\in\left\{ ...
4
votes
0answers
305 views

Normalized sigmoid loss function for boosting?

It seems non-convexity of loss function is not such a problem for boosting with a normalized sigmoid loss function. Do you know any further work showing better results with this kind of boosting than ...
5
votes
2answers
278 views

Hinge loss with one-vs-all classifier

I'm currently looking at the unconstrained primal form of the one-vs-all classifier $$\sum\limits_{i=1}^{N_I} \sum\limits_{k=1,\atop k \neq y_i}^{N_K} L(1+ ...
5
votes
3answers
597 views

Gradient descent oscillating a lot. Have I chosen my step direction incorrectly?

I'm trying to run a basic gradient descent algorithm with a absolute loss function. I can get it to converge to a good solution by it requires a much lower step size and more iterations than had I ...
6
votes
2answers
2k views

Gradient of Hinge loss

I'm trying to implement basic gradient descent and I'm testing it with a hinge loss function ie $\max(0,1-y\ \mathbf{w.x})$. However, I'm confused about the gradient of the hinge loss. I'm under the ...
8
votes
5answers
1k views

What is the 'fundamental' idea of machine learning for estimating parameters?

The 'fundamental' idea of statistics for estimating parameters is maximum likelihood. I am wondering what is the corresponding idea in machine learning. Qn 1. Would it be fair to say that the ...
10
votes
4answers
290 views

Comprehensive overview of loss functions?

I am trying to get a global perspective on some of the essential ideas in machine learning, and I was wondering if there is a comprehensive treatment of the different notions of loss (squared, log, ...