New answers tagged

2

If we abstract away all of the details of the algorithms, tree ensembles have simple structure: A function to initialize a tree A function to choose a split A function to determine when to stop building a single tree A function to determine when to stop adding trees to the ensemble ensemble_continue = True for t in range(max_trees): i = 0 ...


-1

Does this comment from a source, possibly capture your state of modeling: The problems involved in obtaining meaningful coefficients of regression by the method of least squares with such intercorrelated data are well known [2, 13, 21] A potential solution, apply Factor Analysis to construct variables as in this work 'Use of factor scores in multiple ...


1

I have a very similar question here When to use mixed effect model? Depending on what you want to do and the amount of data available, you may or may not build a model for each class / each "group of classes". (But 180 classes for 4000 rows, I would suggest the data is not enough if you wan to build too many models.) Building one model for each class (or ...


1

They most likely have a typo in the sign. It should be $\sum_i\frac12 h_i(f_t(x_i) +g_i/h_i)^2$ instead of $\sum_i\frac12 h_i(f_t(x_i) - g_i/h_i)^2$. It is a weighted square loss between $f_t(x_i)$ and labels $-g_i/h_i$. You can see the negative sign in the solution of the optimization problem in (5). In general the minimizer of $w \mapsto \sum_i \alpha_i ...


1

The regularization primarily reduces the magnitude of the leaf weights, yes. When using L1 regularization, that can make a potential split no longer worth it, and the tree may end up being smaller than otherwise (see this colab notebook, originally drafted for my answer to this other question). L2 regularization can do the same, but not nearly as ...


0

The results of the model trained based on the transformed target to the log space are misleading. Briefly, the log transformation shrinks the variance of the outcomes. Hence, the log transformation makes the model appears better even though it does nothing different. I would strongly suggest you to take a look at this post too. Log-Transforming target var ...


1

The criteria here should depend on the goals of the project, including what modelling or other analyses are intended next. Otherwise guidelines might include Outliers are likely to be genuine, and so in general should be included in any analysis, yet not so that results are highly distorted by a small fraction of extreme outliers. Any transformations ...


0

You apparently have a situation that fits the newsvendor model in the inventory control literature: you produce for each period, face an uncertain demand, sell as much as is demanded or as much product as you have (whatever is lower), and throw away any leftovers. Let $p$ denote your selling price and $c$ your purchasing or production cost, and assume your ...


0

As mentioned by @StayLearning, in slide 4, the author defines the logistic loss $L = \sum_{i=1}^n l(y_i,\hat y_i)$ where $$ l(y_i,\hat y_i) = y_i\log(1+\exp(-\hat y_i)) + (1-y_i)\log(1+\exp(\hat y_i)) $$ then grad = \begin{align} \frac{\partial l}{\partial \hat y_i} &= - y_i\times\frac{1}{1+\exp(\hat y_i)} + (1-y_i)\times\frac{\exp(\hat y_i)}{1+\exp(\...


Top 50 recent answers are included