Vector pointing in the direction where a function is growing fastest; its components are partial derivatives of this function. For questions about gradients in ecology, please use the [ecology] tag instead.

160 questions
Filter by
Sorted by
Tagged with
30 views

### How Gradient Descent is used for classification with Decision Trees?

I'm not able to see how do we use gradient descent to minimize the loss of binary classification with decision tree. What I understood is that we first have a model (decision tree) that try to predict ...
19 views

### Is my understanding and presentation of concept of Gradient Boosting correct?

Initially the model is trained with a training set $\{x_{i}, y_{i}\}_{i=1}^{n}$ by minimizing a differentiable loss function $L(y, F(x))$, and, is initialized with a constant value, \begin{align*} F_{...
45 views

### Is a high learning rate irrelevant when dropping the first or last tree in a GBDT with 100 trees?

Suppose we've trained a GBDT model with 100 trees with a fairly high learning rate. Consider two cases: We drop the first tree in the model We drop the last tree in the model We then compare models ...
11 views

### How can I obtain group (factor) level Hessian and gradient from GLMM in lme4? [closed]

Summary I am working on a problem where I want to determine group level contributions to the maximum likelihood in a GLMM. To do so, I need to access the Hessian and gradients at the group level. This ...
35 views

28 views

### Normalized steepest descent with nuclear/frobenius norm

In steepest gradient descent, we try to find a local minima to a loss function $f(\cdot)$ by the rule: $x_{t} = x - \alpha \triangledown_{x}f(x)$. I've found in textbooks that often we want to ...
46 views

210 views

### Multiclass gradient boosting: how to derive the initial guess, how to predict a probability

I have some questions regarding multiclass boosted-tree-algorithmus. Currently, I apply xgBoost as implemented in R to solve a multi-classification problem. According to StatQuest, for a simple two-...
7 views

### Gradient map based on probability of incident

I have a binary classification model that outputs the probability of an instance belonging to the positive or negative class. I already have the probability threshold by which an instance will be ...
12 views

I have attached an image of the mathematical description of calculating the gradient for the cost function from Pytorch. 1.) Is $\vec{y}$ the output of the network? 2.) What is $v$ in terms of a ...
98 views

### Valid use of differentiable almost everywhere functions, like hinge loss, in gradient optimization/learners, like SciKit-Learn's SGDClassifier?

So, my abstract question is: is it valid (in the sense that stable convergence is roughly expected) to use functions that are differentiable almost everywhere in the practical application of ...
19 views

### Calculate expected gradient length in one-vs-rest or one-vs-one scenario with SVM

According to the paper "Active learning with support vector machines" (LINK) by Kremer et al. from 2014 it is possible to apply the expected gradient length on Support Vector Machines (SVM). Is it ...
78 views

### Why don't we use the output of activation functions for explainability instead of Gradient?

Most of the explainability methods use Gradient for measuring the sensitivity of the model to the input features, why can't we use the activation functions themselves as a measure of sensitivity? ...
31 views

### Monte Carlo Gradient Estimator [closed]

How do we derive this Monte Carlo Estimator? \begin{equation} \nabla_{\phi}\mathbb{E}_{q_{\phi}(z)}[f(z)] = \mathbb{E}_{q_{\phi}(z)}[f(z) \nabla_{q_{\phi}(z)}\ln q_{\phi}(z)] \end{equation}
64 views

90 views

### Simplifying the Matrix Form of the Solution to Ridge Regression

I'm trying to understand how to obtain the solution to an objective function by solving for the parameter vector $\theta$ in ridge regression. I found an example here from Naomi which takes an example ...