Questions tagged [xgboost]

A popular boosting algorithm and software library (stands for "extreme gradient boosting"). Boosting combines weakly predictive models into a strongly predictive model.

Filter by
Sorted by
Tagged with
0
votes
0answers
19 views

XGBoost classification of panel/longitudinal data observations

I have a dataset of several firm quarters for a 10 year period (around 400 firms and around 30 observations per firm). For each quarter, there are a number of annualized financial ratios, which are ...
0
votes
1answer
30 views

What are the disadvantages of models with many parameters to tune?

I modeled with 3 different methods but all showed bad accuracy. So im trying to reason why! one of the issues might be the many parameters that i tuned. So i want to know if there are disadvantages in ...
2
votes
1answer
18 views

Is it better to build N models for each category of data?

I'm new to data science and I'm working on a challenge with some friends, I have a data set of 80 feature and around 4000 rows. The data is split into 180 category (A,B,C,D...etc), at first I tried ...
2
votes
1answer
36 views

XGBoost question on weighted quantile sketch described in paper

The right-hand-side of Equation (3) in the XGBoost paper is $$\sum_{i=1}^n [g_i f_t(\mathbf{x}_i)+\frac{1}{2}h_if_t^2(\mathbf{x}_i)]+\Omega(f_t) \tag{3}$$ In the Section 3.3 "Weighted Quantile ...
0
votes
0answers
16 views

Do I need to select features for XGBoost [duplicate]

I have a regression task and I have around 79 numeric features that predict a numeric target value between 0 and 1, I used gradient boosting trees as there are several relationships both linear and ...
1
vote
1answer
64 views

Where can i find pseudocodes for tree ensemble Algorithms?

I would like to know if there is any such resource where we can get the pseudocodes for Gradient Boosting algorithms ? I am looking for pseudocodes for XGBoost, Random Forest and LGBM. I have read ...
1
vote
0answers
14 views

Forecasting for future periods with machine learning models - how to treat input variables

I have a dataset of X1,X2,X3,etc. to predict the number of units, and one or some of my X variables are lagged versions of the units (my Y variable) I am trying to predict in addition to other ...
0
votes
0answers
17 views

Latest XGboost and Sklearn giving error [migrated]

xgb=XGBClassifier(objective="binary:logistic", n_estimators=100, random_state=42, eval_metric=["auc"]) xgb.fit(X_train, y_train) KeyError ...
0
votes
1answer
35 views

Yeo-Johnson does not increase normality

I have used Box-Cox Yeo-Johnson transformation to make my skewed data columns less skewed and more normal so that I can remove outliers. e.g. originally most of my columns have a 'skewness' of 400! ...
1
vote
0answers
11 views

Scale_Pos_Weight Before or After Train Test Split?

I know XGBoost has a scale_pos_weight we can add in where we take (positives/negatives) and we get our coefficient. However, what is the sample we are drawing from? Is it the whole X, Y set? Or is it ...
0
votes
0answers
19 views

Time series forecasting how to structure train test sets for a year of predict

I have three years worth of data (2017-2019) and the goal is to create a model that can predict out the next year. I am having trouble understanding how to structure my train test splits given this ...
0
votes
0answers
47 views

removing outliers in skewed data for xgboost

i have a couple of columns in my data which are postively skewed. they are non-normal from the hist plots. plotting a qq plot further cinfirms this. i should remove outliers from my data for xgboost. ...
0
votes
0answers
17 views

Interpreting Variable Importance of XGBoost based decision tree models

While building an XGBoost based decision tree model, working with a set of say 100 variables gives a particular variable importance. Now if I want to improve performance through an additional set of ...
0
votes
0answers
24 views

restricting xgboost predictions value within a range

I am trying to predict the users rating on movies. These ratings are continuous ranging from 1 to 5. I have been using xgboost with objective function ...
1
vote
1answer
26 views

XGboost Regression Log Transform Target

I'm training a XGBoost Regressor to predict price which has a highly right skewed distribution. (all prices are positive) I took log transformation on the target thinking it would help to 1) stable ...
1
vote
0answers
25 views

How to judge whether model is overfitting or underfitting

When validation or test data cannot be predicted properly and the model is suitable only for train data, the model is overfitting or underfitting. When I looking for a lot of reference , it seems to ...
1
vote
1answer
18 views

Identify suitable scoring metric for food prediction

I am using GridSearchCV to find the best parameter that help me tune XGBoost for a food prediction algorithm. I am struggling to identify the best scoring metric that would result in the best profit (...
1
vote
0answers
28 views

High cross validation score but low model performance on test set

I'm doing a machine learning project and need to predict a user's credit default probability. I tried some simple automated feature engineering and got a good AUC score on training set using ...
1
vote
0answers
28 views

what is the impact of using large number of features compared to a small data?

I have time series data with 25 features and 181 observations, and the number of classes are 7. I used 3 models svm,random forest and xgboost to classify but the performance was really bad for each. i ...
1
vote
0answers
24 views

eval_set in xgboost and validation data

In my understanding, if I am picking from a set of models, each with a different set of hyper-parameters, the proper way to approach it is like this: First, split the data to ...
0
votes
1answer
34 views

Accuracy of the xgboost classifier is less than random forest?

In general the xgboost classifier is built by the idea of reducing the total error. Im using both xgboost and random forest to classify using small dataset (181 observations) and i noticed that the ...
1
vote
1answer
23 views

Why does my cross validation score get better after each iteration

I am building a regression task using XGboost in R. I noticed my cv score gets better as I increase my cv score, even as high as 100k. What could be resposible for this? I will like my model to ...
3
votes
1answer
54 views

Why do Random forest and XGBoost gives different importance weight on the same set of features?

I am using both random forest and xgboost to examine the feature importance. but i noticed that they give different weights for features as shown in both figures below, for example HFmean-Wav had the ...
1
vote
1answer
26 views

Difference between GBTree and GBDart

From my understanding, GBDart drops trees in order to solve over-fitting. However, when tuning, using xgboost package, rate_drop, by default is 0. I understand this is a parameter to tune, however, ...
0
votes
0answers
8 views

I want to replace XGBRegressor with a simple model to make feature selection

I will make some for loop on to select the best features by my Data frame is big 10M row and about 50 columns so if i replaced xgb with a single Decision tree would it be the same results for the best ...
0
votes
1answer
18 views

Small dataset and optimal parameters for XGboost

I am in the process of tuning the features for my xgboost such as ordinal (label) encoding and one-hot encoding. For example, run the model with column A one-hot ...
2
votes
1answer
70 views

Why are boosted trees difficult to interpret?

I might not fully understand the topic but: Bootstrapped/bagged trees are difficult to interpret because the decisions are made from averaging the prediction of possibly hundreds of trees (ensemble). ...
0
votes
1answer
23 views

How exactly should repeated measures situation be treated in machine learning models?

I was reading this SE about repeated measures and I was a little bit confused. I have a data set consisting of claim information for million locations for up to 5 years and 99.9% of the observations ...
0
votes
0answers
14 views

Is it reasonable to use one vs all(one vs rest) this way?

I am making a multiclass classification model(XGBOOST). But it's not separated well. Some category are dependent on each other. Suppose categories are A B C D E, and C D are dependent on each other ...
0
votes
0answers
17 views

Are the predictions of gradient boosted trees made sequentially? If so, how do these algorithms not have slow prediction speed?

I understand the general process of how gradient boosted trees are trained. A sequence of learners is trained, with the initial learner being fit to the data, and each subsequent learner being fit to ...
2
votes
1answer
19 views

How does regularization part of w help in XGBoost

In the regularization part of XGBoost objective function, it contains gammaT and also lambdasquare(W). I understand gamma is the minimum node split criteria and T is number of leaves and ...
1
vote
0answers
11 views

What is use of XGboost objective function/what's best objective function/best w as in the docs

I am studying how XGboost works but I got completely confused and lost at the end. I am aware how the GBM works and boosting works. I even completely understand the xgboost docs (atleast I thought) to ...
0
votes
0answers
15 views

The difference between train loss and test loss increases as the number of repetitions increases

I am developing machine learning model for classification of documents. For this, I parsed the document into morphemes(NNP,NNG) with tf-idf, and fit them with xgboost. Train losses and validation ...
0
votes
0answers
41 views

Custom objective function for classification in XGBoost using score values rather than probabilities

Is it possible to let customized objective functions in python-based XGBoost (classification setup) accept leaf scores as arguments (i.e. predicted values), rather than class probabilities? More ...
4
votes
2answers
124 views

Why my model overfits despite selecting best hyperparameters value in each tuning step?

I am fitting xgboost classification model to my data with highly inbalanced classes in response variable (99% vs 1%). I use cross-validation with k=5 to tune my ...
2
votes
1answer
48 views

Multiclass gradient boosting: how to derive the initial guess, how to predict a probability

I have some questions regarding multiclass boosted-tree-algorithmus. Currently, I apply xgBoost as implemented in R to solve a multi-classification problem. According to StatQuest, for a simple two-...
0
votes
1answer
19 views

Machine Learning models and “rare-event” independent variables

I have a data set which tries to predict a continuous variable, say house prices $Y$. My independent variables consits of things such as, square meters, number of bedrooms, bathrooms etc. However, I ...
0
votes
0answers
10 views

Literature on applying XGBoost to Time Series Data

I'm currently working on doing a time-series model with very limited data. However, most of the independent variables I have are not time-dependent, cross-sectional data. As such I want to apply some ...
2
votes
1answer
125 views

Sample weights in XGBClassifier

I am using Scikit-Learn XGBClassifier API with sample weights. If I multiply sample weights by 2, I get totally different results with exact same parameters and random_state, I am expecting that If we ...
0
votes
2answers
75 views

XGBoost one-step ahead forecast

I have trained and cross-validated an xgboost classification algorithm in R using the following code: ...
0
votes
0answers
26 views

Why gradient boosting use first-order Taylor expansion approximation?

The target of boosting at step $m$ is (see Wikipedia): $$F_{m}(x)=F_{m-1}(x)+\underset{h_{m} \in \mathcal{H}}{\arg \min }\left[\sum_{i=1}^{n} L\left(y_{i}, F_{m-1}\left(x_{i}\right)+h_{m}\left(x_{i}\...
1
vote
1answer
11 views

After training an XGBoost classifier on a set of features, (how) can I use it to make new predictions based on one of those features?

Forgive me if this is a somewhat naïve question. I have trained an XGBoost classifier that uses COVID-19 patients' age, sex, location, etc. to predict their mortality risk (here is the dataset). The ...
0
votes
0answers
8 views

sample weights for time series classification

I am using time series of macro data (i.e. FX rates, commodity indices, sentiment indices, bond yields/spreads, and equity indices and change in their P/E estimates, etc) for predict market pullback. ...
0
votes
0answers
10 views

Doc2Vec score keep getting worse

I'm using Doc2Vec on kaggle with XGB and MLPClassifier but i noticed that for five times in a row the roc scorse got worse without me changing the code (from 90 to 87). I set a fixed random state for ...
3
votes
0answers
63 views

What is exactly query group “qid” in XGBoost

In XGBoost documentation it's said that for ranking applications we can specify query group ID's qid in the training dataset as in the following snippet: ...
1
vote
0answers
12 views

Different/Unexpected test results using xgboost and gridsearchcv with mean square error loss

I'm noticing very different test results for these two sets of codes: ...
1
vote
1answer
57 views

XGBoost obtain n_estimators obtimal parameter with early stopping

If i use early stopping with an evaluation set for training, when i have to train the model for the final evaluation what is the best approach? Generally I'd train the model with the full dataset but ...
0
votes
0answers
20 views

Classification Model showing different accuracy for SAME data?

This is my first post here, so kindly pardon any commonplace errors. So, i have been training an XGBoost multi-class model on Google Colab. I am using a balanced dataset, with 31000 rows, where each ...
2
votes
1answer
34 views

Performance drops when adding a feature using XGBoost

I did some feature engineering with my data set. When I added on of the new features, the performance significantly dropped. How is this possible? I thought XGBoost is robust to irrelevant variables.
1
vote
1answer
46 views

Does XGBoost has anything to do with logistic regression?

If the objetive is binary:logistic, then it actually running a logistic regression and the weights of the records are changing in every step? Or the XGBoost still creates decision trees sequently? How ...

1
2 3 4 5
8