Skip to main content

Questions tagged [lightgbm]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
2 votes
1 answer
25 views

Efficient prediction using Lightgbm/XGBoost when varying single feature keeping the remaining constant

Assume we have a pre-trained Lightgbm/XGBoost model $f$ dependent on the feature matrix: $$X=\left[z, C\right]$$ Here $z$ is a single feature column and $C$ is the remaining feature columns. I need to ...
BLaursen's user avatar
  • 293
0 votes
0 answers
7 views

Inverse Problem: Using LightGBM model to recommend X (feature) ranges to achieve a specific y (target) range

I am trying to build a LightGBM regression model, where in I have aroud 15-20 Input features and my target variable within a range of 20-40. I have used the SHAP beeswarm plot to kind of understand ...
Debadri Dutta's user avatar
1 vote
0 answers
29 views

Significant performance drop between train and validation set

I am trying both Lgbm and RandomForest for a classification, and I observe the same problem. I am using various metaparams to prevent overfitting, such as max_depth, num_trees (keeping it small for ...
Baron Yugovich's user avatar
0 votes
0 answers
12 views

Hyperparameters tuning and Backward feature selection : which one first?

If i have a lot of features and i want to train a light gbm, on the side i want to do the hyperparameter optimisation and on the other side i want to do backward feature selection to reduce the number ...
Lula's user avatar
  • 1
3 votes
1 answer
115 views

Warning when using sparse categorical values with LightGBM

When training a LightGBM model with lgbm.train, I get the following warning: [LightGBM] [Warning] Met categorical feature which contains sparse values. Consider ...
DustByte's user avatar
  • 131
2 votes
1 answer
61 views

Tuning the learning rate parameter in GBDT models

I've always been taught that decreasing the learning rate parameter in gbdt models such as XGBoost, LightGBM and Catboost will improve the out-of-sample performance, assuming the number of iterations ...
Casper's user avatar
  • 21
1 vote
0 answers
80 views

LightGBM Regressor miscalibratred/underestimating on high fitted values and overestimating on low fitted values

I'm training a pretty standard LightGBM regressor and noticing a strange pattern with the residuals (see images below--I'm bunching the predicted values and taking the observed average for the group). ...
dfried's user avatar
  • 201
0 votes
0 answers
36 views

Can someone explain how the lambdas in lambdarank work?

I am reading From RankNet to LambdaRank to LambdaMART: An Overview. Section 4.1 describes LabdaRank. The last paragraph on page 8 and first paragraph of page 9 describe how the score ($s_i$) of each ...
Abhay Gupta's user avatar
2 votes
1 answer
821 views

XGBoost - Linear Tree

I’ve been reading about linear tree models particularly the linear-tree package and the option to use linear trees in LightGBM if one sets the parameter linear_tree ...
harrynak's user avatar
1 vote
0 answers
16 views

Avoiding over-fitting in Gradient Boosted tree models when multiple sequential observations share the same label [closed]

I am trying to train a Multi-class classification model where every K minutes I receive a set of features x and use the set to ...
Dean Grosbard's user avatar
0 votes
0 answers
91 views

Prediction when Target's lag values are part of Predictors

I'm using LGBM for regression, where the Target column's lagged values (7 columns for each lag day) are also used as predictors when training the model. Absence of the 7Day lag values severely ...
fast_crawler's user avatar
0 votes
1 answer
489 views

Preventing Data Leakage in Time Series Forecasting with Feature Engineering

In a previous question (linked here), I sought guidance on forecasting thousands of time series. Based on the suggestion to treat it as a regression problem, I used the LightGBM model with extensive ...
Tirth's user avatar
  • 113
1 vote
0 answers
123 views

Is perfect isotonic probability calibration realistic?

I work with a labelled tabular dataset of about 1 million observations, with the target being binary. The dataset is heavily imbalanced - about 0.5% positive class. I have trained a gradient boosting ...
StrLdn's user avatar
  • 11
2 votes
0 answers
346 views

Low coverage of prediction intervals from quantile regression using LightGBM on heldout data

I fit three models using LightGBM with quantile objective (which uses pinball loss) using alpha values 0.10, 0.50, and 0.90. The following code is used to wrap the three models into a single class. ...
Julia Maddalena's user avatar
2 votes
1 answer
6k views

How to use categorical features in lightGBM? [closed]

I am working on an attrition dataset which has a large number of categorical parameters. Each categorical parameter has a high cardinality, so one-hot encoding them is out of question. I was looking ...
Ashish Samant's user avatar
0 votes
0 answers
233 views

LightGBM accuracy not increasing with iterations on Validation Set?

I am training a model with LightGBM, and I am getting an output like this: ...
the man's user avatar
  • 285
2 votes
1 answer
579 views

Boruta followed by LightGBM for feature selection

Assume that we have a high-dimensional data with a few samples. We want to select a minimum set of best features from this dataset using LightGBM feature importance. This is because of an external ...
ML Guy's user avatar
  • 21
1 vote
1 answer
182 views

LGBM fails to overfit

I have this data: ...
Hossein's user avatar
  • 3,494
2 votes
1 answer
1k views

Can missing data imputations outperform default handling for LightGBM?

Here is my understanding: LightGBM by default handles missing values by putting all the values corresponding to a missing value of a feature on one side of a split, either left or right depending on ...
Akavall's user avatar
  • 2,671
1 vote
0 answers
19 views

Approaching multiple records for one observation; radiomics of 2D slices of a 3D object

Background I am trying to create a model that can predict Type 2 diabetes in a patient based on MRI scans of their thigh muscle. Previous literature has shown that fat deposition in the muscle of ...
Saminy Creed's user avatar
2 votes
1 answer
1k views

Overfitting using lightGBM?

I have a small dataset composed of 800 data points where I need to perform a regression task. I randomly chose 10% of the dataset to be used as validation. The problem is that I am not sure if I am ...
Rods2292's user avatar
  • 371
3 votes
1 answer
7k views

Sample weights in LightGBM - where to specify?

I want to introduce samples weights to my lgbm classifier. From what I see the weights can be added both in the lgb.Dataset and in the ...
user377065's user avatar
3 votes
1 answer
2k views

LightGBM interpretation of monotonic constraints in multiclass classification

When using LightGBM in classification problems it is possible to use monotonic constraints. In binary classification problems the interpretation is straightforward: "The probability of class (say)...
BLaursen's user avatar
  • 293
2 votes
1 answer
2k views

How do we make predictions for future data when you have lagged dependent features used in training?

I am executing a lightGBM model to forecast my units sold (qty) over a period of time. Objective is to run a model for each product group and be able to capture the trends, price elasticity, etc and ...
Siddhartha Srivastava's user avatar
0 votes
1 answer
47 views

Score of LGBM Classifier ranging only between a short interval

I am working on a fraud problem and I am trying to predict either some market/stores has done fraudulent transactions or not. I've trained a boosting model (lgbm algorithm) on a unbalanced dataset. I'...
Gabriel Monteiro's user avatar
2 votes
1 answer
1k views

Gradient boosting on a loss/objective function without second derivatives

In principle, it should be possible to build a gradient boosted tree model on a loss function that only has (nonzero) first derivatives. I've found in practice xgboost and lightgbm make heavy use of ...
Alex Eftimiades's user avatar
0 votes
0 answers
22 views

t-statistics in gradient boosted machine/forest such LightGBM

Is there a t-statistics in the gradient boosted forest regression model such as that in LightGBM? If so, how is it defined, extracted and used?
Hans's user avatar
  • 1,015
5 votes
2 answers
8k views

how does `subsample` parameter work in boosting algorithms like xgboost and lightgbm?

From what I know, both of them are sequential learners and only the 1st tree in the sequence gets built on the data and all the following trees that get built are to correct the mistakes of previous ...
Naveen Reddy Marthala's user avatar
0 votes
1 answer
390 views

Lightgbm, time-series and spikes repeated on a yearly basis

I have a data set (time-series) with the shape {$2190$x$63$}. There are 63 variables, 2 products ($A$ and $B$) worth of 3 years of daily data, thus I have $1095$ observations per product and total of $...
User's user avatar
  • 87
0 votes
0 answers
495 views

When we use k-means clustering with Light GBM, comparing with Random Forest

I am developping the prediction model with many parameters. As I was not satisfied by the performance of Random Forest Regression, I tried to use k-means clustering to regroup the similar variable and ...
stat_man's user avatar
0 votes
0 answers
40 views

How to fix the tree structure for a tree-based algorithm?

Background Some of our BI analysts and most of our managers are interested in making explainable predictions. One of our colleagues proposed an approach based on individual tree leaves from a tree-...
mirekphd's user avatar
  • 165
1 vote
0 answers
720 views

How to get the best num_boost_round on the full training data?

I have a huge training data of size 5.5 GBs with over 55m rows. Because iterating over the whole dataset again and again was too slow, I used a 1% sample of this whole data to select the best ...
rohit kumar's user avatar
8 votes
1 answer
3k views

Any reasons to prefer neural networks over boosting methods in tabular data?

Based on Kaggle winners data, it seems that ensemble boosting methods like XGBOOST, LIGHTGBM, CATBOOST are the top choices when dealing with structured or tabular data for maximizing the prediction ...
mhsnk's user avatar
  • 307
0 votes
0 answers
61 views

Predictor With Lower Mean Absolute Error Ends Up Worse

I have been recently working on a problem to estimate the ETAs of vehicles using ensemble techniques such as LightGBM. As expected, the distance taken by the vehicle's route to its destination is a ...
James Balajan's user avatar
1 vote
0 answers
459 views

How to tune LightGBM parameters to overcome underfitting? [closed]

I'm using LightGBM for a regression task. My training data's shape is (2000000, 1600), which means the number of training data is 2 million +, and each sample has 1600 features. The figure below is ...
Yuhua Wei's user avatar
3 votes
1 answer
868 views

Why can EFB(Exclusive Feature Bundling) works in lightGBM?

As I know, EFB can help you to decrease features which are sparse. They put two features together and add offset every feature in feature bundles. They combine features into same histogram. After ...
ChrisChu's user avatar