We’re rewarding the question askers & reputations are being recalculated! Read more.

Vector pointing in the direction where a function is growing fastest; its components are partial derivatives of this function. For questions about gradients in ecology, please use the [ecology] tag instead.

124 questions
Filter by
Sorted by
Tagged with
318 views

### Is there Early Stop in Stochastic gradient descent?

Let's say I have 100 data points. If stochastic gradient didn't converge at first round of 100 data points, then I need to continue another round. Then if it converged at 50th data point, so total ...
942 views

### Standardizing numerical and encoding of categorical data for training boosted decision tree

Is there a "best practice" way of standardizing numerical and encoding of categorical data for training boosted decision tree? Both for classification and regression problems
8k views

### Is gradient boosting appropriate for data with low event rates like 1%?

I am trying gradient boosting on a dataset with event rate about 1% using Enterprise miner, but it is failing to produce any output. My question is, since it a decision tree based approach, is it even ...
749 views

### Why does the basic gradient descent not converge for this example?

I have a toy example for a linear regression of the form $$y=\beta_0 + \beta_1x_1 + \beta_2x_2$$ The data is: ...
4k views

### Deriving gradient of a single layer neural network w.r.t its inputs, what is the operator in the chain rule?

Problem is: Derive the gradient with respect to the input layer for a a single hidden layer neural network using sigmoid for input -> hidden, softmax for hidden -> output, with a cross entropy ...
1k views

328 views

### Defining grad in R's optim for MLE

I have a ML I want to maximize in R's function optim. I am currently using the method BFGS. The optim procedure is quite slow however, and I was hoping to speed up the process by specifying the ...
10k views

### Gradient of loss function for (non)-linear prediction functions

$\newcommand{\y}{\mathbf{y}} \newcommand{\wv}{\mathbf{w}} \newcommand{\xv}{\mathbf{x}} \newcommand{\loss}{L(\wv;\xv, y)}$ I'm trying to clear up the calculation of the gradient of a loss function, ...
69 views

I was rethinking the logic of solving constraint optimization problem. And I read many stuff in website like this and textbooks. But the following question is still unsolved (to me). We know that at ...
1k views

### Statistically testing for a significant difference between two slope values [closed]

I have five trend lines plotted in excel of number of prescriptions of a 5 different drugs over time (MM/YYYY) and I want to test the statistical significance of the difference between the slopes, to ...
3k views

### Implementing WARP Loss (Gradient Computation)

I am trying to implement the WARP Loss in Torch, as defined in the WSABIE paper: http://www.thespermwhale.com/jaseweston/papers/wsabie-ijcai.pdf The Algorithm is as follows: The Algorithm specifies ...
920 views

### Temporal convolution for NLP [closed]

I'm trying to follow Kalchbrenner et al. 2014 (http://nal.co/papers/Kalchbrenner_DCNN_ACL14) (and basically most of the papers in the last 2 years which applied CNNs to NLP tasks) and implement the ...
834 views

### Is the gradient computation in the word2vec implementation actually wrong?

In the paper "Efficient Estimation of Word Representations in Vector Space", it is stated that "All models are trained using stochastic gradient descent and backpropagation": http://arxiv.org/pdf/1301....
611 views

### Convert time scalar field to velocity vector field

I am trying to figure out if it's possible to create vector velocity field from some raster containing spatial distribution of scalar variable (timestamp of some mass-media event in my case). Here is ...