New answers tagged

1

The Wikipedia page has left out a few steps. When they say "the likelihood of the training set", they just mean "the probability of the training set given some parameter values": $$ L(\theta | x_1, ..., x_n) = p(x_1, ..., x_n | \theta) $$ The data $x_k$ in the training set is assumed conditionally independent, so that $$ p(x_1, ..., x_n | \theta) = \...


0

I would recommend you to use Dice loss when faced with class imbalanced datasets, which is common in the medicine domain, for example. Also, Dice loss was introduced in the paper "V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation" and in that work the authors state that Dice loss worked better than mutinomial logistic loss ...


1

I've thought about this a lot (I am OP), and I've come up with the following conclusion through my readings: Consider RV $X$ distributed wrt a density $f(x\mid\theta_0)$ (some observed) Here we assume the data can be generated through a parameterization (justify the presence of conditioning), and that in particular samples wrt $X$ are generated wrt a true ...


0

I think you are correct: strictly speaking there should be no $\mathbb{E}$ before the integral term (which, by the way, also appears in equation 1.2, but not in equation 2.1). There is one way to see it: the best estimate $\hat{\theta}$ of the ground truth value $\theta$ will be the one that will minimize the Kullback-Leibler divergence between the estimated ...


1

This is how I think about it: $$ D_{KL}(p(y_i | x_i) \:||\: q(y_i | x_i, \theta)) = H(p(y_i | x_i, \theta), q(y_i | x_i, \theta)) - H(p(y_i | x_i, \theta)) \tag{1}\label{eq:kl} $$ where $p$ and $q$ are two probability distributions. In machine learning, we typically know $p$, which is the distribution of the target. For example, in a binary classification ...


3

$L_2$ regularization is basically adding a parabola with a minimum at the origin to the loss surface. How steeply the parabola rises depends on the magnitude of the $L_2$ penalty. If the penalty is too large, then the effect of regularization will overwhelm the signal from the cross-entropy loss, because the shape of the surface is so distorted by the ...


Top 50 recent answers are included