Skip to main content

Questions tagged [catboost]

CatBoost is an open-source gradient boosting on decision trees library with categorical features support out of the box for Python & R

Filter by
Sorted by
Tagged with
0 votes
0 answers
30 views

Can a Catboost oblivious tree split on the same feature more than once

When training an oblivious decision tree in Catboost, can it use the same feature more than once for splitting? Let's say there is a feature age. If the first split ...
NotProbable's user avatar
1 vote
0 answers
89 views

Calibrating CatBoostClassifier produces worse results

I'm performing multiclass probability prediction using CatBoostClassifier on a dataset with ~4000 rows, 13 features, 4 target classes. Dataset has outliers, but it is balanced. For this task I'm using ...
primadonna's user avatar
0 votes
1 answer
69 views

Is my model overfitting or is my training process wrong?

I'm predicting multiclass probabilities using CatBoost Classifier. I have a balanced dataset with roughly 4000 rows, 13 features, 4 target class labels. Dataset has some outliers which I decided not ...
primadonna's user avatar
2 votes
1 answer
61 views

Tuning the learning rate parameter in GBDT models

I've always been taught that decreasing the learning rate parameter in gbdt models such as XGBoost, LightGBM and Catboost will improve the out-of-sample performance, assuming the number of iterations ...
Casper's user avatar
  • 21
1 vote
0 answers
68 views

Is there a way to set an exposure for a Poisson loss in Catboost?

I'd like to use Catboost for actuarial models (eg claims frequency). Although I see that Poisson loss is an option, I don't see that exposures are directly supported. How do people deal with this?
thecity2's user avatar
  • 1,955
0 votes
0 answers
50 views

Custom loss function optimized for correlation for gradient boosted trees algorithms like XGBRegressor or LGBMRegressor

I have a rather classical tabular data prediction problem and more or less successfully using XGBRegressor or LGBMRegressor both using MSE as their loss function. A slight deviation from the standard ...
Josef Švenda's user avatar
3 votes
0 answers
123 views

Can I use multiple quantile regression to estimate the probability a dependant variable is above / below a certain value?

Let's say I have a dataset of characteristics of newly launched products in a retail environment, and the dependant variable Y is total $ sales in the first year of ...
SCool's user avatar
  • 277
1 vote
0 answers
49 views

Is there a better way to self-train on tabular data?

Context: I'm training a classifier on some fraud data. Only a chunk of data is labeled (~2000) so I'm trying a self-training approach, what I'm doing for now is: Iteratively training a model then ...
Oussama Bastamy's user avatar
2 votes
1 answer
402 views

Why doesn't CatBoost Encoding cause target leakage?

I'm currently working on a fraud detection problem with a dataset of 300,000 rows and 500 columns, 70 of which are categorical with over 10 categories each. I'm facing memory constraints and exploring ...
Connor's user avatar
  • 655
0 votes
0 answers
542 views

Improving an underfit model with Catboost for regression problem

I'm currently using CatboostRegressor(iterations=500, random_seed=123, cat_features=['month_number', 'day_of_week', 'year']) for developing a 1-year predictive ...
user177196's user avatar
1 vote
0 answers
697 views

Why are my catboost predictions pushed towards 0 and 1 and how to smooth it? [closed]

I am using catboost in a classification problem with extreme unbalanced classes (positive class consists of ~ 0.25% of all observations). I fitted a model using catboost in R and got a very good ...
Carol's user avatar
  • 11
1 vote
0 answers
213 views

Total number of learned weights/parameters in CatBoost model?

I wasn't really able to find this anywhere in the documentation, or after searching through the model object itself (though it must be there somewhere, of course. The ...
Matt Phillips's user avatar
1 vote
1 answer
345 views

Generalization of model performance (AUC) and tuning of a catboost classifier

I was wondering if it is good practice to overfit on the training data while tuning a catboost classifier for a binary outcome. Wouldn't it be better to reguralize until validation error equals ...
infinite789's user avatar
3 votes
2 answers
897 views

How do Ordered Target Statistics work for CatBoost?

This question follows closely this paper . I'm trying to fully understand how Ordered Target Statistics (TS) (for CatBoost) works. E.g. the CatBoost algorithm uses this method to group categorical ...
mugdi's user avatar
  • 90
8 votes
1 answer
3k views

Any reasons to prefer neural networks over boosting methods in tabular data?

Based on Kaggle winners data, it seems that ensemble boosting methods like XGBOOST, LIGHTGBM, CATBOOST are the top choices when dealing with structured or tabular data for maximizing the prediction ...
mhsnk's user avatar
  • 307
0 votes
0 answers
266 views

High precision and low recall but with a balanced dataset

When i evaluated my model (CatBoost classifier), I noticed that my model has high precision and low recall (Recall: 0.59, Precision: 0.89) but the classes are perfectly balanced (1: 45.5, 2: 54.5) and ...
Federico Mele's user avatar
1 vote
0 answers
119 views

Analytical expression of a CatBoost regression model in R

When adjusting a multiparametric regression model, an analytical expression that characterizes the fitted model (e.g., in a linear multiparametric regression, the equation is $\hat\beta= \hat\beta_o+\...
Daaviid's user avatar
  • 97
1 vote
1 answer
108 views

If evaluation set is the same as training set, why would the evaluation error be different from training error?

I understand the use of evaluation set for parameter tuning and over-fitting in general. The examples in the evaluation set should be unseen and different from training set. However, in the following ...
yowtzu.lim's user avatar
1 vote
0 answers
284 views

Role of weighted quantile sketch in XGBoost, LightGBM and CatBoost

Have i understood it correctly if i say that "weighted quantile sketch" allows XGBoost to use histogram search on feature values for split finding? Also, do LightGBM or CatBoost use ...
Polarni1's user avatar
2 votes
0 answers
82 views

Looking for information regarding tree-based gradient boosting algorithms comparative performance on data sets with different underlying properties

I have a difficult time finding any theoretical or empirical comparative research regarding the tree-based gradient boosting algorithms on data sets with different underlying properties. Is there any ...
Polarni1's user avatar
2 votes
1 answer
3k views

Why does LightGBM Classifier gives some folks a probability of 1 of belonging in a class with log-loss?

I'm trying to use the LightGBM package in python for a multi-class classification problem and I'm baffled by its results. For a minority of the population, LightGBM predicts a probability of 1 (...
Guillaume F.'s user avatar
2 votes
1 answer
3k views

catboost does not overfit - how is that possible?

I'm fitting and evaluating a CatBoostRegressor and a XGBRegressor to the same regression problem. I tried matching their ...
ihadanny's user avatar
  • 3,350
2 votes
0 answers
326 views

Gradient boosting (GB) splitting methods (categorical features)

Regarding categorical features - ordinary trees treat categorical features in two main ways, CART - considers only binary splitting, those computing the mean response value (y_mean_i per each category ...
afek's user avatar
  • 21
2 votes
1 answer
7k views

L2 Regularization in CatBoost

I am studying the CatBoost paper https://arxiv.org/pdf/1706.09516.pdf (particularly Function BuildTree in page 16), and noticed that it did not mention regularization. In particular, split selection ...
Poland Spring's user avatar
1 vote
1 answer
2k views

negative value of prediction from a training set that only contains positives

a catboost model recently gave me a % of predictions that are negative while none of the training set contains negative values for the label. How is this possible especially in cases where the % of ...
jim andr's user avatar
1 vote
1 answer
721 views

Model Size of Random Forest VS CatBoost

I trained models based on the same dataset, using random forest (sklearn) and CatBoost. I use n_estimators=1000 for random forest, and n_estimators(iterations)=1000 for CatBoost. The random forest ...
I Wonder's user avatar
0 votes
0 answers
29 views

Tweak Model or Process to Focus More on Category Values for Regression

I have built a machine learning model using Catboost and I noticed something as I examine the results. When I am looking at the results there is a big fluctuation in the errors between my test set and ...
DS_Misc's user avatar
  • 11
3 votes
1 answer
987 views

Negative Feature Importance Value in CatBoost LossFunctionChange

I am using CatBoost for ranking task. I am using QueryRMSE as my loss function. I notice for some features, the feature importance values are negative and I don't know how to interpret them. It says ...
Kemeng Zhang's user avatar
1 vote
2 answers
798 views

AUPRC vs AUROC and updating training set in quasi-classification problem

I have an unbalanced classification problem (95% "0", 5% "1") regarding quality control."0" means "no problem" and "1" means "problem". I'm not predicting real cases one by one, this is, my client ...
PeterTschuschke's user avatar
3 votes
1 answer
2k views

Feature Interaction Strength in Catboost

I was wondering if anyone knew how the feature interaction strength is calculated in the catboost package. The documentation https://catboost.ai/docs/concepts/output-data_feature-analysis_feature-...
Niamh Belton's user avatar
3 votes
1 answer
3k views

(Low cardinality) categorical features handling in gradient boosting libraries

In some popular gradient boosting libraries (lgb, catboost), they all seems like can handle categorical inputs by just specifying the column names of the categorical features, and pass it into a ...
Sam's user avatar
  • 377
0 votes
1 answer
702 views

Catboost Regression. Function Extrapolation

I'm new at ML and have a problem with catboost. So, I want to predict function value (For example cos | sin etc.). I went over everything but my prediction is always straight line Is it possible and ...
Яков Гущин's user avatar
1 vote
1 answer
129 views

catboost ignoring my holiday category

I'm experimenting with catboost, predicting electricity demand from temperature, timeofday, dayofweek and if a day is a public holiday or not (and a few other continuous and categorical columns but ...
David Waterworth's user avatar
4 votes
1 answer
372 views

Please correct my assumption on how regression trees work

I'm trying to understand how regression trees work, I've been experimenting with catboost and xgboost in python, and I'm getting results which I don't expect, can someone please clarify (and apologies ...
David Waterworth's user avatar
-1 votes
1 answer
2k views

What are the key hyperparameters to tune in CatBoost?

I've used XGBoost for a long time but I'm new to CatBoost. If I wanted to run a sklearn RandomizedSearchCV, what are CatBoost's hyperparameters worthwhile including for a binary classification problem?...
ADJ's user avatar
  • 435
4 votes
1 answer
2k views

IncToDec Catboost Explained

I am struggling to understand how the overfitting detector with catboost works: https://tech.yandex.com/catboost/doc/dg/concepts/overfitting-detector-docpage/#overfitting-detector I am finding ...
B_Miner's user avatar
  • 8,810