Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [xgboost]

A popular boosting algorithm and software library (stands for "extreme gradient boosting"). Boosting combines weakly predictive models into a strongly predictive model.

0
votes
0answers
13 views

Need help with lag features in regression forecasting

I am trying to build a timeseries prediction model. The problem is that I'm still hesitant whether I should use lag features or not. What makes me wonder is the fact that the training data has these '...
1
vote
1answer
26 views

Regression when target has a wide range

I'm working on a regression model where I have to predict time. These times go from a few seconds to up to 30 min and more. I calculated the sMAPE through 1 minute bins of the target, and noticed ...
0
votes
2answers
32 views

why is number of epochs set as external parameter?

I am confused by the very notion of epochs in neural networks (as well as number of trees in gradient boosting). Gradient descent method (as most optimization algorithms) keep going until the loss ...
0
votes
1answer
28 views

Interplay between early stopping and cross validation

I am a little bit confused by early stopping and in particular by how it can be inserted inside a CV framework. As far as I understand, I can fix the optimal number of epochs (for NN, or number of ...
0
votes
0answers
53 views

xgboost.train : Gradient and Hessian of RMSLE

I'm trying to implement the RMSLE custom function to train XGBoost, but I'm getting a constant training and validation error. Here is my function: ...
3
votes
1answer
52 views

Does XGBoost have a max-depth hyper-parameter?

According to the explanation in Complete Guide to Parameter Tuning in XGBoost, XGBoost doesn't use max_depth argument as Random Forest or GBM does. It expands the ...
-1
votes
0answers
46 views

Unbalanced classes multiclass

Certain ML algorithms have parameters which can be used to deal with the effects of unbalanced dependent variable classes. For example the random forest implementation in Sci-kit learn has the class ...
1
vote
0answers
32 views

How does offset in XGBoost is handled in binary:logistic objective function

I am working on a mortality prediction (binary outcome) problem with “base mortality probability” as my offset in the XGboost problem. I have used gbtree booster and binary:logistic objective function....
0
votes
1answer
52 views

XGboost for Time series - using lag of target variables

I'm trying to make a time series forecast using XGBoost. I have already added many time related variables - day_of_week, month, week_of_month, holiday. I want to add lagged values of target variable ...
0
votes
0answers
31 views

Predicting future price in high inflation economies

I am trying to create a machine learning model in a country which has high inflation. With this model, I am trying to predict the price of a second hand car. As my train data, I have second hand car ...
0
votes
0answers
26 views

Model Performance using Precision as evaluation metric

I am dealing with an imbalanced class with the following distribution : (Total dataset size : 10763 X 20) 0 : 91% 1 : 9% To build model on this dataset having class imbalance, I have compared ...
0
votes
0answers
23 views

Balanced LogLoss with XGBoost

Following the discussion on here I started worrying less about class imbalance. However, I recently started building a predictor, using XGBoost, and I wanted to used LogLoss as my target metric. I ...
0
votes
0answers
43 views

Problem about tuning hyper-parametres

I have tried GridSearchCV and BayesSearchCV for tuning my lightGBM algorithm (for binary classification). I have used 10 iterations and I have indicated scoring ="roc_auc" In the first iteration, I ...
1
vote
0answers
48 views

The question of Taylor expansion of loss function in XGBoost [duplicate]

I am learning XGBoost from documentation, but there are a few questions in the derivation of it. In the part of ...
2
votes
1answer
34 views

A regressor failed to learn extreme values

I am working on a regression problem using xgbclassifier (https://xgboost.readthedocs.io/en/latest/python/python_api.html) The output values range from 0 to 10 (log-normal distribution), but when I ...
1
vote
1answer
41 views

Making a model to predict the error of another model

So basically I have a machine learning model where I want to have a prediction interval, the model is XGBoost so it is tricky to do Quantile Regression and I was looking for an alternative method to ...
0
votes
1answer
26 views

(Feature Selection) Meaning of “importance type” in get_score() function of XGBoost

I'm trying to use a build in function in XGBoost to print the importance of features. My code is like ...
1
vote
1answer
56 views

xgboost tayler expansion detail [duplicate]

This is the objective function for Xgboost. I have no idea where $g_{i}$ and $h_{i}$ came from is some one explain how this two terms came form? or direct me to the related tutorial page then I ...
0
votes
0answers
14 views

Why it is hard for `xgboost` to learn periodic functions?

In this simple example, I try to train a xgboost regressor to learn a periodic function: ...
0
votes
0answers
30 views

Why does `xgboost` find such an unbalanced split in my data?

I'm using xgboost for regression. The data and the python script used for analyzing the data are uploaded here. The ...
0
votes
0answers
52 views

What is intuitive interpretation of the leaf values in dumped XGBoost trees?

I'm learning XGBoost. The following is the code I used and below that is the tree #0 and #1 in the XGBoost model I built. I'm having a hard time understanding the meanings of the leaf values. Some ...
2
votes
1answer
327 views

Classification XGBoost vs Logistic Regression

I have a binary classification problem where the classes are slightly unbalanced 25%-75% distribution. I have a total of around 35 features after some feature engineering and the features I have are ...
0
votes
0answers
30 views

Impact of propensity model

I have built a propensity model, which gives out probabilities of a customer paying given a collection intervention using a xgboost model. The model has an AOC-ROC of 81% with an accuracy of 77% ...
0
votes
0answers
37 views

How is the minimum logarithmic loss calculated when initializing the XGBoost algorithm?

Suppose there are $5$ sample units, $2$ of which carry the feature $y=1$ to be predicted and three of which carry the feature $y=0$. So, $2$ are positive. The XGBoost algorithm initializes with $\...
0
votes
1answer
43 views

What's are the advantages and disadvantages of incremental learning?

Generally speaking, we all know it's to save spaces with incremental learning. According to the ques in stackoverflow , it also said that. But what's the disadvantages? What I know from my ...
1
vote
1answer
25 views

Approaches to reduce dimensions (feature selection/extraction) with high dimensional count data before running tree based model

My dataset has ~100k samples and 3000 dimensions. The data are counts, anywhere between 0-8 and it's pretty sparse. Because of 'curse of high dimension', I want to shrink the number of variables ...
0
votes
0answers
94 views

Feature selection in xgboost vs GBM in H2O

I am working on a big data set( more than 100 variables) and 30 million observations. I tried to build 100 models with a grid search using both XGBoost and GBM in H2O (Sparkling Water). I realized ...
1
vote
0answers
25 views

XGBoost tree dump contains lots of empty trees

After fitting a regression model using XGBoost, I want to inspect the individual trees that were built. In the resulting table, I find a lot of 0-depth trees, i.e. trees with only a leaf node, and ...
0
votes
0answers
74 views

Prediction interval for XGBoost Classifier

I am reading this article https://towardsdatascience.com/regression-prediction-intervals-with-xgboost-428e0a018b where the author presented a method to calculate the prediction interval for XGBoost ...
1
vote
0answers
71 views

Getting prediction interval for xgboost prediction?

I am looking for a solution that can bring prediction interval for xgboost classification (https://xgboost.readthedocs.io/en/latest/), i.e. instead of output the probability that an instance belongs ...
3
votes
1answer
138 views

what does regularization mean in xgboost (tree)

In xgboost (xgbtree), gamma is the tunning parameter to control the regularization. I understand what regularization means in <...
0
votes
0answers
33 views

How to use seasonal features in time series regression with models such as xgboost?

I have a hard time understanding how one can create seasonal indices such as a yearly mean or (x - yearly mean(x)) and use them as predictors for monthly n horizon forecast. For example: I want to ...
0
votes
1answer
44 views

How to deal with overestimation of small values and underestimation of high values in XGBoost? [closed]

I'm running XGBoost to predict prices on a cars dataset, I was wondering what alternatives are there for this kind of problem ...
0
votes
0answers
30 views

Why is XGBoost prediction proba so concentrated within specific range? (unbalanced class)

I am pretty to new to Machine Learning. I am training on some past Kaggle competitions including the Santander Customer Satisfaction Challenge (https://www.kaggle.com/c/santander-customer-satisfaction)...
2
votes
2answers
54 views

Why are all predictions made by XGBoost distinct?

If I understood correctly the XGBoost is a framework that operates on gradient tree boosting. It means that behind the scenes, it uses a decision tree to make a prediction. So, from what I read in the ...
1
vote
0answers
52 views

Xgboost / Boosted decision trees: Representing categorical id numbers as continuous integer variable

I've been reading through some kernels at kaggle.com for a sales forecasting competition, and noticed that a lot of people using Xgboost are feeding it categorial ID variables, represented as ...
0
votes
0answers
13 views

Modeling multiple outputs - one model or several

Recently at work I enter an interesting discussion that I thought could continue here and receive your output. I'm trying to model some data that have as an output a categorical variable (let's say X)...
0
votes
1answer
31 views

How to train XGBoost Classifier with soft output distribution

Please correct me if I am wrong. Is it possible to train XGBoost Classifier on soft output? Usually, the output of the model is (N, 1) in dimension which ...
0
votes
0answers
35 views

How would one debug a machine learning model that has a bias?

I'm predicting values roughly forming a normal distribution with mean 0. However, my machine learning model tends to predict lower than 0 on average. I didn't run any statistical tests, but it's very ...
0
votes
1answer
517 views

Running XGBoost with *highly* imbalanced data returns near 0% true positive rate. Tried SMOTE and it did not improve much. What else can I do?

I'm using XGBoost on a dataset of ~2.8M records of hard drive failures, where less than 200 are tagged as failures. After cleaning, there are 11 features in this dataset. Below is my ...
1
vote
1answer
70 views

(Low cardinality) categorical features handling in gradient boosting libraries

In some popular gradient boosting libraries (lgb, catboost), they all seems like can handle categorical inputs by just specifying the column names of the categorical features, and pass it into a ...
2
votes
1answer
97 views

Xgboost and repeated measures

I am learning xgboost and am planning on running a tree model. My dataset includes repeated measures. In a GLMM I would include the ID to account for repeated measures and I'm curious if I should do ...
0
votes
0answers
29 views

How do I include new features that did not exist before into an existing model?

I have a binary classification model predicting sports result with features covering 10 years worth of matches. However, how would I feed new tracking data that is only limited to the last 3 years. ...
1
vote
0answers
16 views

Regression - many samples have the same target

I have a machine learning problem in which I have a many-to-one relationship from samples to targets. I have ~3k samples but only 11 targets with a shared key YEAR ...
1
vote
1answer
27 views

For a specific dataset do all the features have the same importance across different algorithms?

I wonder if by implementing a feature selection technic using training with a specific algorithm you can select the feature you need to use with other algorithms also. To be more specific after I ...
1
vote
0answers
102 views

terminal values in XGBoost / gradient boosting models

I am writing a follow up question regarding a closed Cross Validated question previosuly. The original question can be found here To give a breif overview, I am using the following code to produce a ...
0
votes
0answers
20 views

Variance of error term is nonconstant between observations

I used XGB algorithm to train a model. The task is to train models to predict human personality based on his/her personal photo. We found some significant features when we extracted them by Pearson ...
0
votes
0answers
35 views

GBDT- randomized repetition feature selection

Consider the following approach for feature selection in the specific case of gradient boosting decision trees: Randomly pick X% of features Run algorithm Record importance of each feature Repeat ...
1
vote
1answer
303 views

XGBoost tree “Value” output: [duplicate]

Using the following R code I obtain a decision tree using the agaricus dataset: ...
0
votes
0answers
47 views

How to prevent GBDT from splitting on uninformative features?

I'm looking into using feature importance scores from GBDT for feature selection. Although GBDT does not need manual feature selection, the number of features is a restriction of the production system ...