The loss-functions tag has no wiki summary.
1
vote
0answers
36 views
Asymptotics of 0-1 classification loss
I am interested in training a simple binary linear classifier. That is, I will find a vector of weights $\bf w$ such that I can predict the class of new example by the sign of
$f(x) = w^T x$.
I ...
2
votes
0answers
63 views
Cramer-Rao type bound for Information Gain
I am interested in the Bayes risk of some distribution $\pi$
$$
r(\pi) = \mathbb{E}_{\pi(x)}[ \mathbb{E}_{\Pr(y|d,x)}[L(x,\hat x(y|d))]],
$$
where $L$ is some loss function and $\hat x$ is the ...
2
votes
1answer
35 views
Convexity of loss function with respect to the mean
Let $X\,$ be a non-negative r.v. with known pdf $f(x|\theta)$ but with a single unknown parameter $\theta$. Suppose that the mean $\mu$ can be used to uniquely determine the value of $\theta$, i.e. if ...
1
vote
1answer
145 views
Changing logistic regression's loss function
We're using logistic regression to predict events probability. Logistic regression tries to minimize the residual variance (sum of squared residuals). However, in our specific problem we would like ...
2
votes
0answers
79 views
Does cross validation work with asymmetric loss functions?
My simple question is does cross validation work with an asymmetric loss function?
I cannot find docs on google to answer.
2
votes
3answers
162 views
Multiclass classification when class distribution is known
What is an example of an algorithm that, when i have a known distribution across discrete groups and I have some sort of model score that a person is in each group, assigns persons to groups such that ...
1
vote
1answer
64 views
Weighted loss function for non-random sample
When comparing a regression estimation method (Y vs X) I currently use a weighted squared loss function:
$$
\int_{-\infty}^{\infty}(\hat{f}(x)-f(x))^2 \, \hat{p}(x) \, dx
$$
Where $\hat{f}(x)$ ...
1
vote
2answers
121 views
Matching loss function for tanh units in a neural net
There's not much more I can add to the question. Googling has mostly turned up research papers on springerlink and other sites I don't have access to.
Given a neural network model with $tanh(x)$ as ...
3
votes
1answer
216 views
Dual problem for L2 support vector machine
Here is the dual problem for L2 support vector machine:
$$\max_{\alpha\in\mathbb{R}^{n}} 2\alpha^{T}y-\alpha^{T}\left(K+n\lambda Id_{\mathbb{R}^{n}}\right)\alpha$$
$$\forall i\in\left\{ ...
1
vote
0answers
119 views
Normalized sigmoid loss function for boosting?
It seems non-convexity of loss function is not such a problem for boosting with a normalized sigmoid loss function. Do you know any further work showing better results with this kind of boosting than ...
5
votes
2answers
181 views
Hinge Loss with one-vs-all classifier
I'm currently looking at the unconstrained primal form of the one-vs-all classifier
$$\sum\limits_{i=1}^{N_I} \sum\limits_{k=1,\atop k \neq y_i}^{N_K} L(1+ ...
5
votes
3answers
372 views
Gradient descent oscillating a lot. Have I chosen my step direction incorrectly?
I'm trying to run a basic gradient descent algorithm with a absolute loss function. I can get it to converge to a good solution by it requires a much lower step size and more iterations than had I ...
4
votes
1answer
451 views
Gradient of Hinge loss
I'm trying to implement basic gradient descent and I'm testing it with a hinge loss function ie $\max(0,1-y\ \mathbf{w.x})$. However, I'm confused about the gradient of the hinge loss. I'm under the ...
6
votes
5answers
541 views
What is the 'fundamental' idea of machine learning for estimating parameters?
The 'fundamental' idea of statistics for estimating parameters is maximum likelihood. I am wondering what is the corresponding idea in machine learning.
Qn 1. Would it be fair to say that the ...
5
votes
3answers
132 views
Comprehensive overview of loss functions?
I am trying to get a global perspective on some of the essential ideas in machine learning, and I was wondering if there is a comprehensive treatment of the different notions of loss (squared, log, ...
