Questions tagged [catboost]

CatBoost is an open-source gradient boosting on decision trees library with categorical features support out of the box for Python & R

Filter by
Sorted by
Tagged with
0
votes
0answers
11 views

Why is Recall increasing as number of iterations in CatBoost is decreasing?

I have a highly imbalanced data. I'm using CatBoost to build a predictive model as all the variables are categorical. The default number of iteration is 1000. In order to make it faster I gave 100 as ...
0
votes
0answers
26 views

Summing of Catboost's scikit learn 'permutations importance' values for groups of features

I have built a catboost classifier trained on approximately 600 features, and I want to calculate the permutation importance of groups of features. My question is, can I use the permutation importance ...
1
vote
0answers
56 views

Role of weighted quantile sketch in XGBoost, LightGBM and CatBoost

Have i understood it correctly if i say that "weighted quantile sketch" allows XGBoost to use histogram search on feature values for split finding? Also, do LightGBM or CatBoost use ...
0
votes
0answers
15 views

What is the problem of biased point-wise gradient estimates?

I was going through the paper on catboost. It was mentioned that: all classical boosting algorithms suffer from overfitting caused by the problem of biased pointwise gradient estimates. (Page 3, 1st ...
2
votes
0answers
81 views

Looking for information regarding tree-based gradient boosting algorithms comparative performance on data sets with different underlying properties

I have a difficult time finding any theoretical or empirical comparative research regarding the tree-based gradient boosting algorithms on data sets with different underlying properties. Is there any ...
1
vote
1answer
820 views

Why does LightGBM Classifier gives some folks a probability of 1 of belonging in a class with log-loss?

I'm trying to use the LightGBM package in python for a multi-class classification problem and I'm baffled by its results. For a minority of the population, LightGBM predicts a probability of 1 (...
2
votes
1answer
729 views

catboost does not overfit - how is that possible?

I'm fitting and evaluating a CatBoostRegressor and a XGBRegressor to the same regression problem. I tried matching their ...
2
votes
0answers
187 views

Gradient boosting (GB) splitting methods (categorical features)

Regarding categorical features - ordinary trees treat categorical features in two main ways, CART - considers only binary splitting, those computing the mean response value (y_mean_i per each category ...
1
vote
1answer
2k views

L2 Regularization in CatBoost

I am studying the CatBoost paper https://arxiv.org/pdf/1706.09516.pdf (particularly Function BuildTree in page 16), and noticed that it did not mention regularization. In particular, split selection ...
1
vote
1answer
521 views

negative value of prediction from a training set that only contains positives

a catboost model recently gave me a % of predictions that are negative while none of the training set contains negative values for the label. How is this possible especially in cases where the % of ...
1
vote
1answer
458 views

Model Size of Random Forest VS CatBoost

I trained models based on the same dataset, using random forest (sklearn) and CatBoost. I use n_estimators=1000 for random forest, and n_estimators(iterations)=1000 for CatBoost. The random forest ...
0
votes
0answers
20 views

Tweak Model or Process to Focus More on Category Values for Regression

I have built a machine learning model using Catboost and I noticed something as I examine the results. When I am looking at the results there is a big fluctuation in the errors between my test set and ...
2
votes
1answer
549 views

Negative Feature Importance Value in CatBoost LossFunctionChange

I am using CatBoost for ranking task. I am using QueryRMSE as my loss function. I notice for some features, the feature importance values are negative and I don't know how to interpret them. It says ...
1
vote
2answers
312 views

AUPRC vs AUROC and updating training set in quasi-classification problem

I have an unbalanced classification problem (95% "0", 5% "1") regarding quality control."0" means "no problem" and "1" means "problem". I'm not predicting real cases one by one, this is, my client ...
3
votes
1answer
901 views

Feature Interaction Strength in Catboost

I was wondering if anyone knew how the feature interaction strength is calculated in the catboost package. The documentation https://catboost.ai/docs/concepts/output-data_feature-analysis_feature-...
3
votes
1answer
1k views

(Low cardinality) categorical features handling in gradient boosting libraries

In some popular gradient boosting libraries (lgb, catboost), they all seems like can handle categorical inputs by just specifying the column names of the categorical features, and pass it into a ...
0
votes
1answer
374 views

Catboost Regression. Function Extrapolation

I'm new at ML and have a problem with catboost. So, I want to predict function value (For example cos | sin etc.). I went over everything but my prediction is always straight line Is it possible and ...
1
vote
1answer
68 views

catboost ignoring my holiday category

I'm experimenting with catboost, predicting electricity demand from temperature, timeofday, dayofweek and if a day is a public holiday or not (and a few other continuous and categorical columns but ...
4
votes
1answer
183 views

Please correct my assumption on how regression trees work

I'm trying to understand how regression trees work, I've been experimenting with catboost and xgboost in python, and I'm getting results which I don't expect, can someone please clarify (and apologies ...
-1
votes
1answer
1k views

What are the key hyperparameters to tune in CatBoost?

I've used XGBoost for a long time but I'm new to CatBoost. If I wanted to run a sklearn RandomizedSearchCV, what are CatBoost's hyperparameters worthwhile including for a binary classification problem?...
4
votes
1answer
1k views

IncToDec Catboost Explained

I am struggling to understand how the overfitting detector with catboost works: https://tech.yandex.com/catboost/doc/dg/concepts/overfitting-detector-docpage/#overfitting-detector I am finding ...