Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [xgboost]

A popular boosting algorithm and software library (stands for "extreme gradient boosting"). Boosting combines weakly predictive models into a strongly predictive model.

0
votes
0answers
14 views

In Xgboost, how does Scale_pos_Weight work for regression?

I have noticed I am getting better results if use scale_pos_weight. The training data is imbalanced. I have tried sampling but didnt get good result. I have two questions: 1. How does scale_pos_weight ...
0
votes
1answer
85 views

Prediction of regression coefficients with XGBoost

I am doing survival analysis. There is a dataset of items (id, group_id, observed lifetime, censorship status), each item belongs to a certain group. Each item is ...
0
votes
0answers
29 views

Model Not Performing Well On Validation Data - Customer Attrition Modeling

I am modeling customer churn for the online subscription. I looked back 90 days into customers’ data, using number customer watching behavior etc. I get a pretty strong model based on test data. <...
2
votes
1answer
25 views

Model with target as feature for benchmark

I was reading this article (https://towardsdatascience.com/predicting-the-popularity-of-instagram-posts-deeb7dc27a8f) about predicting the popularity of Instagram post using different techniques and ...
1
vote
0answers
31 views

What derivative to use in Gradient boosting decision tree for a semi-supervised model

I am trying to build a semi-supervised prediction model with a Gradient Boosting decision trees. The learning phase is done using the following input: $X \in \mathbb{R}^{n\times p} $ $O(X) \in \...
1
vote
0answers
58 views

XGBoost and AdaBoostClassifier feature importances

I try to compare XGBoost and AdaBoostClassifier (from sklearn.ensemble) feature importances charts. From this answer: https://stats.stackexchange.com/a/324418/239354 I get know that ...
1
vote
1answer
49 views

Need help with lag features in regression forecasting

I am trying to build a timeseries prediction model. The problem is that I'm still hesitant whether I should use lag features or not. What makes me wonder is the fact that the training data has these '...
1
vote
1answer
43 views

Regression when target has a wide range

I'm working on a regression model where I have to predict time. These times go from a few seconds to up to 30 min and more. I calculated the sMAPE through 1 minute bins of the target, and noticed ...
0
votes
2answers
36 views

why is number of epochs set as external parameter?

I am confused by the very notion of epochs in neural networks (as well as number of trees in gradient boosting). Gradient descent method (as most optimization algorithms) keep going until the loss ...
0
votes
1answer
69 views

Interplay between early stopping and cross validation

I am a little bit confused by early stopping and in particular by how it can be inserted inside a CV framework. As far as I understand, I can fix the optimal number of epochs (for NN, or number of ...
3
votes
1answer
62 views

Does XGBoost have a max-depth hyper-parameter?

According to the explanation in Complete Guide to Parameter Tuning in XGBoost, XGBoost doesn't use max_depth argument as Random Forest or GBM does. It expands the ...
1
vote
0answers
52 views

How does offset in XGBoost is handled in binary:logistic objective function

I am working on a mortality prediction (binary outcome) problem with “base mortality probability” as my offset in the XGboost problem. I have used gbtree booster and binary:logistic objective function....
1
vote
2answers
198 views

XGboost for Time series - using lag of target variables

I'm trying to make a time series forecast using XGBoost. I have already added many time related variables - day_of_week, month, week_of_month, holiday. I want to add lagged values of target variable ...
0
votes
0answers
40 views

Predicting future price in high inflation economies

I am trying to create a machine learning model in a country which has high inflation. With this model, I am trying to predict the price of a second hand car. As my train data, I have second hand car ...
0
votes
0answers
91 views

Balanced LogLoss with XGBoost

Following the discussion on here I started worrying less about class imbalance. However, I recently started building a predictor, using XGBoost, and I wanted to used LogLoss as my target metric. I ...
1
vote
0answers
53 views

The question of Taylor expansion of loss function in XGBoost [duplicate]

I am learning XGBoost from documentation, but there are a few questions in the derivation of it. In the part of ...
2
votes
1answer
39 views

A regressor failed to learn extreme values

I am working on a regression problem using xgbclassifier (https://xgboost.readthedocs.io/en/latest/python/python_api.html) The output values range from 0 to 10 (log-normal distribution), but when I ...
1
vote
1answer
60 views

Making a model to predict the error of another model

So basically I have a machine learning model where I want to have a prediction interval, the model is XGBoost so it is tricky to do Quantile Regression and I was looking for an alternative method to ...
0
votes
1answer
58 views

(Feature Selection) Meaning of “importance type” in get_score() function of XGBoost

I'm trying to use a build in function in XGBoost to print the importance of features. My code is like ...
1
vote
1answer
74 views

xgboost tayler expansion detail [duplicate]

This is the objective function for Xgboost. I have no idea where $g_{i}$ and $h_{i}$ came from is some one explain how this two terms came form? or direct me to the related tutorial page then I ...
0
votes
0answers
16 views

Why it is hard for `xgboost` to learn periodic functions?

In this simple example, I try to train a xgboost regressor to learn a periodic function: ...
0
votes
0answers
59 views

Why does `xgboost` find such an unbalanced split in my data?

I'm using xgboost for regression. The data and the python script used for analyzing the data are uploaded here. The ...
1
vote
1answer
202 views

What is an intuitive interpretation of the leaf values in XGBoost base learners?

I'm learning XGBoost. The following is the code I used and below that is the tree #0 and #1 in the XGBoost model I built. I'm having a hard time understanding the meanings of the leaf values. Some ...
2
votes
1answer
832 views

Classification XGBoost vs Logistic Regression

I have a binary classification problem where the classes are slightly unbalanced 25%-75% distribution. I have a total of around 35 features after some feature engineering and the features I have are ...
0
votes
0answers
42 views

Impact of propensity model

I have built a propensity model, which gives out probabilities of a customer paying given a collection intervention using a xgboost model. The model has an AOC-ROC of 81% with an accuracy of 77% ...
0
votes
0answers
42 views

How is the minimum logarithmic loss calculated when initializing the XGBoost algorithm?

Suppose there are $5$ sample units, $2$ of which carry the feature $y=1$ to be predicted and three of which carry the feature $y=0$. So, $2$ are positive. The XGBoost algorithm initializes with $\...
0
votes
1answer
58 views

What's are the advantages and disadvantages of incremental learning?

Generally speaking, we all know it's to save spaces with incremental learning. According to the ques in stackoverflow , it also said that. But what's the disadvantages? What I know from my ...
1
vote
1answer
32 views

Approaches to reduce dimensions (feature selection/extraction) with high dimensional count data before running tree based model

My dataset has ~100k samples and 3000 dimensions. The data are counts, anywhere between 0-8 and it's pretty sparse. Because of 'curse of high dimension', I want to shrink the number of variables ...
0
votes
0answers
171 views

Feature selection in xgboost vs GBM in H2O

I am working on a big data set( more than 100 variables) and 30 million observations. I tried to build 100 models with a grid search using both XGBoost and GBM in H2O (Sparkling Water). I realized ...
1
vote
0answers
54 views

XGBoost tree dump contains lots of empty trees

After fitting a regression model using XGBoost, I want to inspect the individual trees that were built. In the resulting table, I find a lot of 0-depth trees, i.e. trees with only a leaf node, and ...
0
votes
0answers
133 views

Prediction interval for XGBoost Classifier

I am reading this article https://towardsdatascience.com/regression-prediction-intervals-with-xgboost-428e0a018b where the author presented a method to calculate the prediction interval for XGBoost ...
1
vote
0answers
108 views

Getting prediction interval for xgboost prediction?

I am looking for a solution that can bring prediction interval for xgboost classification (https://xgboost.readthedocs.io/en/latest/), i.e. instead of output the probability that an instance belongs ...
3
votes
1answer
359 views

what does regularization mean in xgboost (tree)

In xgboost (xgbtree), gamma is the tunning parameter to control the regularization. I understand what regularization means in <...
0
votes
0answers
71 views

How to use seasonal features in time series regression with models such as xgboost?

I have a hard time understanding how one can create seasonal indices such as a yearly mean or (x - yearly mean(x)) and use them as predictors for monthly n horizon forecast. For example: I want to ...
0
votes
1answer
71 views

How to deal with overestimation of small values and underestimation of high values in XGBoost? [closed]

I'm running XGBoost to predict prices on a cars dataset, I was wondering what alternatives are there for this kind of problem ...
0
votes
0answers
63 views

Why is XGBoost prediction proba so concentrated within specific range? (unbalanced class)

I am pretty to new to Machine Learning. I am training on some past Kaggle competitions including the Santander Customer Satisfaction Challenge (https://www.kaggle.com/c/santander-customer-satisfaction)...
2
votes
2answers
56 views

Why are all predictions made by XGBoost distinct?

If I understood correctly the XGBoost is a framework that operates on gradient tree boosting. It means that behind the scenes, it uses a decision tree to make a prediction. So, from what I read in the ...
1
vote
0answers
66 views

Xgboost / Boosted decision trees: Representing categorical id numbers as continuous integer variable

I've been reading through some kernels at kaggle.com for a sales forecasting competition, and noticed that a lot of people using Xgboost are feeding it categorial ID variables, represented as ...
0
votes
0answers
17 views

Modeling multiple outputs - one model or several

Recently at work I enter an interesting discussion that I thought could continue here and receive your output. I'm trying to model some data that have as an output a categorical variable (let's say X)...
0
votes
1answer
59 views

How to train XGBoost Classifier with soft output distribution

Please correct me if I am wrong. Is it possible to train XGBoost Classifier on soft output? Usually, the output of the model is (N, 1) in dimension which ...
0
votes
0answers
50 views

How would one debug a machine learning model that has a bias?

I'm predicting values roughly forming a normal distribution with mean 0. However, my machine learning model tends to predict lower than 0 on average. I didn't run any statistical tests, but it's very ...
0
votes
1answer
942 views

Running XGBoost with *highly* imbalanced data returns near 0% true positive rate. Tried SMOTE and it did not improve much. What else can I do?

I'm using XGBoost on a dataset of ~2.8M records of hard drive failures, where less than 200 are tagged as failures. After cleaning, there are 11 features in this dataset. Below is my ...
1
vote
1answer
108 views

(Low cardinality) categorical features handling in gradient boosting libraries

In some popular gradient boosting libraries (lgb, catboost), they all seems like can handle categorical inputs by just specifying the column names of the categorical features, and pass it into a ...
2
votes
1answer
149 views

Xgboost and repeated measures

I am learning xgboost and am planning on running a tree model. My dataset includes repeated measures. In a GLMM I would include the ID to account for repeated measures and I'm curious if I should do ...
0
votes
0answers
29 views

How do I include new features that did not exist before into an existing model?

I have a binary classification model predicting sports result with features covering 10 years worth of matches. However, how would I feed new tracking data that is only limited to the last 3 years. ...
1
vote
0answers
21 views

Regression - many samples have the same target

I have a machine learning problem in which I have a many-to-one relationship from samples to targets. I have ~3k samples but only 11 targets with a shared key YEAR ...
1
vote
1answer
30 views

For a specific dataset do all the features have the same importance across different algorithms?

I wonder if by implementing a feature selection technic using training with a specific algorithm you can select the feature you need to use with other algorithms also. To be more specific after I ...
1
vote
0answers
155 views

terminal values in XGBoost / gradient boosting models

I am writing a follow up question regarding a closed Cross Validated question previosuly. The original question can be found here To give a breif overview, I am using the following code to produce a ...
0
votes
0answers
20 views

Variance of error term is nonconstant between observations

I used XGB algorithm to train a model. The task is to train models to predict human personality based on his/her personal photo. We found some significant features when we extracted them by Pearson ...
0
votes
0answers
56 views

GBDT- randomized repetition feature selection

Consider the following approach for feature selection in the specific case of gradient boosting decision trees: Randomly pick X% of features Run algorithm Record importance of each feature Repeat ...