Questions tagged [xgboost]

A popular boosting algorithm and software library (stands for "extreme gradient boosting"). Boosting combines weakly predictive models into a strongly predictive model.

Filter by
Sorted by
Tagged with
0
votes
0answers
11 views

How can I build a single multi-class GBM model where classes are related and may add noise in one vs all approach

I am trying to build a multi-class GBM model where classes are let's say 0,1,2,3,4. These classes are related in a way that while predicting any non zero class I would want to eliminate classes before ...
1
vote
1answer
34 views

What calculation does XGBoost use for feature importances?

Does anyone know what the actual calculation behind the feature importance (importance type='gain') method in the xgboost library is? I looked through the documentation and also consulted some other ...
0
votes
0answers
19 views

Ratio of data matching / not matching target variable in training dataset in XGBoost

I have inherited an XGBoost model and training script. The current implementation uses a training data set of data split 50 / 50 between inputs matching the target variable, and data not matching the ...
0
votes
0answers
12 views

scale_pos_Weight , weights params impact on loss calculations for xgboost and lgbm for unbalanced classes

I went through all the questions here and discussion available on web, to figure out how the 'scale_pos_weight' (and 'class_Weight' for multi class problems & individual instance weights given by ...
0
votes
0answers
12 views

XGboost vs. LightGBM, node splits

I believe by default, xgboost or lightgbm use all the features in the model for splitting the nodes in each tree (is this correct? can opt to select few by colsample options) and can the features be ...
0
votes
1answer
9 views

Transition for scikit learn to xgboost: Where can I find a comprehensive documentation for xgboost? (Python)

As the internet seems to be conviced that xgboost is well worth a shot when working with decision trees anyways, I set out to try it. I deal with a binary classification problem. Up to now, I was ...
0
votes
1answer
29 views

Get individual features importance with XGBoost [closed]

I have trained an XGBoost binary classifier and I would like to extract features importance for each observation I give to the model (I already have global features importance). More specifically, I ...
1
vote
1answer
48 views

Panel Regression vs. XGBoost Time Series Features

Panel regression is a technique to merge longitudinal and cross sectional data together in a linear model. Linear model doesnt work well since by bringing time series features into the model, it can ...
2
votes
2answers
72 views

How to Reduce Number of Variables Before Running Random Forrest or XGBoost

I've simplified the problem I'm working on for this post, so that the focus is on the issue I'm having. I'm trying to predict if a patient will be diagnosed with arthritis in 2019, based on the ICD-...
0
votes
2answers
24 views

How to perform SHAP explainer on a system of models

I have developed a model with Autoencoder + XGBoost. Autoencoder is used to reduce dimensionality and then passed on to XGBoost for prediction. I would like to understand the feature importance of the ...
3
votes
2answers
64 views

gamma parameter in xgboost

I came across one comment in an xgboost tutorial. It says "Remember that gamma brings improvement when you want to use shallow (low max_depth) trees". My understanding is that higher gamma higher ...
1
vote
0answers
28 views

Are boosted machine learning methods robust against low probable feature combinations when predicting?

I would like to use machine learning methods in the potential outcome framework, that is, simulating outcome for all observations under different values of a specific predictor, while keeping all ...
1
vote
3answers
101 views

Binary Classification in Imbalanced Data; Oversampling and Imputation

Together with two friends I participate in a university course about data mining in R and we chose the topic of bankruptcy prediction. We started with some "clean" data found on an "In class" kaggle ...
1
vote
1answer
24 views

How can I make XGBoost have an exponentially distributed output?

The input distribution is exponential, but XGBoost's predictions' distribution is always unimodal but not exponential. Is there a way to make it exponential?
1
vote
1answer
18 views

What could cause ML predictions to be multimodal when the inputs are unimodal? [closed]

This appears to be a systematic issue that occurs with the whole range of data points: In general, what are some possible causes for discrepancies like this? I know it's impossible to pinpoint the ...
1
vote
1answer
41 views

Using instance weights in XGBoost

I want to understand whether giving weights to instances across a dataset in XGBoost using the below method makes sense. I switched to this method after trying out a few approaches that didn't fare ...
2
votes
2answers
40 views

Important features for the XGboost algorithm are also the most important for the training of DNN?

I know that both a deep neural network (DDN) and the gradient boosting decision tree algorithm Xgboost can be used for the task of classification. I'm using a DNN first and it works fine. With ...
1
vote
1answer
76 views

Calibration curve of XGBoost for binary classification

I'm working on a binary classification problem, with imbalanced classes (10:1). Since for binary classification, the objective function of XGBoost is ...
0
votes
0answers
29 views

Way to stop model from overfitting in automated training pipeline?

I'm currently training a gradient boosting model for which I want to create an automated training pipeline containing hyperparameter optimization with hyperopt and also cross-validation. While trying ...
0
votes
0answers
21 views

Small sample size with very skewed right response [duplicate]

I have a dataset of 300 observations with 7 predictor variables with 1 continuous response variable. The response is strongly skewed to the right and there are no significant correlations among any ...
0
votes
0answers
16 views

Why Would 2 Tree Based Methods Have Different Variables Of Importance

I am using the data set Boston to predict the dependent variable medv. I used two tree based methods, xgboost and random forest. Both models give me a very similar R^2 on the test data set. ...
1
vote
1answer
18 views

Testing set accuracy by using cross validation using xgboost with caret

I am working on an xgboost model using caret. I'm using cross validation, but don't know if I'm understanding it correctly. As I understand, it creates multiple training and test sets. Does this mean ...
0
votes
0answers
20 views

Not splitting test set and just using competition public score

With relatively small (~700) training set, would it be better if I train model with XGboost+k-fold validation and just use public score to evaluate the model's performance on unseen data? I wonder if ...
0
votes
1answer
30 views

Cross validation best practice for competition purpose

I'm fairly new to DS scene and I have been learning about theories and doing practices on kaggle/participate in private competition. For real world problems, my understanding is that you split out ...
0
votes
2answers
61 views

Using bagging and random forests together

I was looking at a kernel implementation (for text classification) and the following piece of code got me a little bit confused (I removed part of the features - in order to keep it light - as most of ...
0
votes
0answers
42 views

In Xgboost, how does Scale_pos_Weight work for regression?

I have noticed I am getting better results if use scale_pos_weight. The training data is imbalanced. I have tried sampling but didnt get good result. I have two questions: 1. How does scale_pos_weight ...
0
votes
1answer
144 views

Prediction of regression coefficients with XGBoost

I am doing survival analysis. There is a dataset of items (id, group_id, observed lifetime, censorship status), each item belongs to a certain group. Each item is ...
0
votes
0answers
36 views

Model Not Performing Well On Validation Data - Customer Attrition Modeling

I am modeling customer churn for the online subscription. I looked back 90 days into customers’ data, using number customer watching behavior etc. I get a pretty strong model based on test data. <...
2
votes
1answer
28 views

Model with target as feature for benchmark

I was reading this article (https://towardsdatascience.com/predicting-the-popularity-of-instagram-posts-deeb7dc27a8f) about predicting the popularity of Instagram post using different techniques and ...
1
vote
0answers
39 views

What derivative to use in Gradient boosting decision tree for a semi-supervised model

I am trying to build a semi-supervised prediction model with a Gradient Boosting decision trees. The learning phase is done using the following input: $X \in \mathbb{R}^{n\times p} $ $O(X) \in \...
1
vote
0answers
86 views

XGBoost and AdaBoostClassifier feature importances

I try to compare XGBoost and AdaBoostClassifier (from sklearn.ensemble) feature importances charts. From this answer: https://stats.stackexchange.com/a/324418/239354 I get know that ...
1
vote
1answer
117 views

Need help with lag features in regression forecasting

I am trying to build a timeseries prediction model. The problem is that I'm still hesitant whether I should use lag features or not. What makes me wonder is the fact that the training data has these '...
1
vote
1answer
62 views

Regression when target has a wide range

I'm working on a regression model where I have to predict time. These times go from a few seconds to up to 30 min and more. I calculated the sMAPE through 1 minute bins of the target, and noticed ...
0
votes
2answers
37 views

why is number of epochs set as external parameter?

I am confused by the very notion of epochs in neural networks (as well as number of trees in gradient boosting). Gradient descent method (as most optimization algorithms) keep going until the loss ...
0
votes
1answer
126 views

Interplay between early stopping and cross validation

I am a little bit confused by early stopping and in particular by how it can be inserted inside a CV framework. As far as I understand, I can fix the optimal number of epochs (for NN, or number of ...
3
votes
1answer
70 views

Does XGBoost have a max-depth hyper-parameter?

According to the explanation in Complete Guide to Parameter Tuning in XGBoost, XGBoost doesn't use max_depth argument as Random Forest or GBM does. It expands the ...
1
vote
0answers
68 views

How does offset in XGBoost is handled in binary:logistic objective function

I am working on a mortality prediction (binary outcome) problem with “base mortality probability” as my offset in the XGboost problem. I have used gbtree booster and binary:logistic objective function....
1
vote
3answers
352 views

XGboost for Time series - using lag of target variables

I'm trying to make a time series forecast using XGBoost. I have already added many time related variables - day_of_week, month, week_of_month, holiday. I want to add lagged values of target variable ...
0
votes
0answers
48 views

Predicting future price in high inflation economies

I am trying to create a machine learning model in a country which has high inflation. With this model, I am trying to predict the price of a second hand car. As my train data, I have second hand car ...
0
votes
0answers
135 views

Balanced LogLoss with XGBoost

Following the discussion on here I started worrying less about class imbalance. However, I recently started building a predictor, using XGBoost, and I wanted to used LogLoss as my target metric. I ...
1
vote
0answers
56 views

The question of Taylor expansion of loss function in XGBoost [duplicate]

I am learning XGBoost from documentation, but there are a few questions in the derivation of it. In the part of ...
2
votes
1answer
53 views

A regressor failed to learn extreme values

I am working on a regression problem using xgbclassifier (https://xgboost.readthedocs.io/en/latest/python/python_api.html) The output values range from 0 to 10 (log-normal distribution), but when I ...
1
vote
1answer
84 views

Making a model to predict the error of another model

So basically I have a machine learning model where I want to have a prediction interval, the model is XGBoost so it is tricky to do Quantile Regression and I was looking for an alternative method to ...
0
votes
1answer
108 views

(Feature Selection) Meaning of “importance type” in get_score() function of XGBoost

I'm trying to use a build in function in XGBoost to print the importance of features. My code is like ...
1
vote
1answer
97 views

xgboost tayler expansion detail [duplicate]

This is the objective function for Xgboost. I have no idea where $g_{i}$ and $h_{i}$ came from is some one explain how this two terms came form? or direct me to the related tutorial page then I ...
0
votes
0answers
17 views

Why it is hard for `xgboost` to learn periodic functions?

In this simple example, I try to train a xgboost regressor to learn a periodic function: ...
0
votes
0answers
86 views

Why does `xgboost` find such an unbalanced split in my data?

I'm using xgboost for regression. The data and the python script used for analyzing the data are uploaded here. The ...
1
vote
1answer
411 views

What is an intuitive interpretation of the leaf values in XGBoost base learners?

I'm learning XGBoost. The following is the code I used and below that is the tree #0 and #1 in the XGBoost model I built. I'm having a hard time understanding the meanings of the leaf values. Some ...
2
votes
1answer
1k views

Classification XGBoost vs Logistic Regression

I have a binary classification problem where the classes are slightly unbalanced 25%-75% distribution. I have a total of around 35 features after some feature engineering and the features I have are ...
0
votes
0answers
66 views

Impact of propensity model

I have built a propensity model, which gives out probabilities of a customer paying given a collection intervention using a xgboost model. The model has an AOC-ROC of 81% with an accuracy of 77% ...