New answers tagged

0

Adding to montols answer: I think he is right on most points, except that, from my understanding, it is the learning rate 𝜖, not 𝜆, that controls for validity of the Taylor expansion(TE). This is because 𝜖 scales the final step size taken towards the TE-minimum and for small 𝜖 TE clearly becomes a better approximation. Moreover, since the Hessian is ...


0

Welcome Kipkogei Francis! The prediction step should normally provide you with two columns each with one predicted value for each class. If you provide a sample of your data and the code you have tried then it may be possible to solve your problem otherwise only general comments are possible.


0

In fact, the decision tree is equivalent with the deep neural network with the ReLU activation function. See the paper Oblique Decision Trees from Derivatives of ReLU Networks or the or discussion on decision tree and neural networks.


1

The shrinkage rate value $\eta$ (aka. learning rate) in the context gradient boosting does not mean anything particular individually. It helps us control the rate over which our prediction function is adapting its shape. $\eta$ is vaguely related to the size of the dataset. Everything else being equal, a larger dataset should require a "higher" $\...


5

Do the weak models all need to be of the same kind? Could I follow a decision stump with a small neural network, for instance? They do not have to be all of the same type, but there are good reasons for using simple models that train quickly, see below. Why not train a more complex model sequentially the same way - find where it was wrong, and put greater ...


1

Boosting can be seen as gradient descent performed in function space $\mathcal{H}$ of weak learners (see e.g. [1, 2]). From the point of view of empirical risk minimization, at time step $m$ we would like to take a step in the negative gradient direction $-\nabla_{F_{m-1}} L(y, F_{m-1})$, whose coordinate projection on the observed dataset equals to the ...


0

For python, you could use import _pickle as cPickle to save the model to a pickle file and restore the model from the pickle file. The codes for store the model: with open("gbmFit.pkl", "wb") as pickle_file: cPickle.dump(model, pickle_file) For restore the model, one could use following code: with open('gbmFit.pkl', 'rb') as pickle_file: gbmfit =...


2

I'm tending to disagree with user11852's answer. Here's my thinking: With traditional statistical models, such as regression, the human specifies a model structure that he/she believes is a (or the most) reasonable approximation of some underlying "data generating" model. IF that single model structure does not in fact conform well to the data ... i.e., it ...


7

The main rationale is the (wrongly) perceived low return on investment. Lack of time and inappropriate training confound the issue. To a lesser extent, these points are aggravated by laziness and technical difficult respectively. Especially with more complex models it becomes progressively harder to infer why a model made a particular prediction. Yes, ...


Top 50 recent answers are included