Questions tagged [lightgbm]
The lightgbm tag has no usage guidance.
31
questions
1
vote
1
answer
54
views
XGBoost - Linear Tree
I’ve been reading about linear tree models particularly the linear-tree package and the option to use linear trees in LightGBM if one sets the parameter linear_tree ...
- 11
1
vote
0
answers
9
views
Avoiding over-fitting in Gradient Boosted tree models when multiple sequential observations share the same label [closed]
I am trying to train a Multi-class classification model where every K minutes I receive a set of features x and use the set to ...
0
votes
0
answers
23
views
Prediction when Target's lag values are part of Predictors
I'm using LGBM for regression, where the Target column's lagged values (7 columns for each lag day) are also used as predictors when training the model. Absence of the 7Day lag values severely ...
- 25
0
votes
1
answer
85
views
Preventing Data Leakage in Time Series Forecasting with Feature Engineering
In a previous question (linked here), I sought guidance on forecasting thousands of time series. Based on the suggestion to treat it as a regression problem, I used the LightGBM model with extensive ...
- 13
1
vote
0
answers
95
views
Is perfect isotonic probability calibration realistic?
I work with a labelled tabular dataset of about 1 million observations, with the target being binary. The dataset is heavily imbalanced - about 0.5% positive class.
I have trained a gradient boosting ...
- 11
2
votes
0
answers
208
views
Low coverage of prediction intervals from quantile regression using LightGBM on heldout data
I fit three models using LightGBM with quantile objective (which uses pinball loss) using alpha values 0.10, 0.50, and 0.90. The following code is used to wrap the three models into a single class. ...
- 330
0
votes
0
answers
46
views
LightGBM precision optimization
In LightGBM model from scikitLearn Python library, how precision (True Positives / (True Positives + False Positives)) can be optimized?
2
votes
1
answer
3k
views
How to use categorical features in lightGBM? [closed]
I am working on an attrition dataset which has a large number of categorical parameters. Each categorical parameter has a high cardinality, so one-hot encoding them is out of question. I was looking ...
0
votes
0
answers
147
views
LightGBM accuracy not increasing with iterations on Validation Set?
I am training a model with LightGBM, and I am getting an output like this:
...
- 235
2
votes
1
answer
330
views
Boruta followed by LightGBM for feature selection
Assume that we have a high-dimensional data with a few samples. We want to select a minimum set of best features from this dataset using LightGBM feature importance. This is because of an external ...
- 21
1
vote
1
answer
122
views
2
votes
1
answer
584
views
Can missing data imputations outperform default handling for LightGBM?
Here is my understanding:
LightGBM by default handles missing values by putting all the values corresponding to a missing value of a feature on one side of a split, either left or right depending on ...
- 2,661
1
vote
0
answers
19
views
Approaching multiple records for one observation; radiomics of 2D slices of a 3D object
Background
I am trying to create a model that can predict Type 2 diabetes in a patient based on MRI scans of their thigh muscle. Previous literature has shown that fat deposition in the muscle of ...
- 11
2
votes
1
answer
811
views
Overfitting using lightGBM?
I have a small dataset composed of 800 data points where I need to perform a regression task. I randomly chose 10% of the dataset to be used as validation.
The problem is that I am not sure if I am ...
- 371
0
votes
0
answers
35
views
Why does different bagging fractions lead to the same result?
I am using R to subset a few data samples with a random seed beforehand. However, by trying different values of bagging fraction, the results somehow are the same. The lgbm model (lgbm.mod) is ...
- 41
2
votes
1
answer
4k
views
Sample weights in LightGBM - where to specify?
I want to introduce samples weights to my lgbm classifier. From what I see the weights can be added both in the lgb.Dataset and in the ...
- 21
0
votes
0
answers
37
views
What is reduce scatter? how it savea training time in lightGBM?
The lightGBM document mentions:
"Data Parallel in LightGBM:
We reduce communication cost of data parallel in LightGBM:
Instead of “Merge global histograms from all local histograms”, LightGBM ...
- 13
3
votes
1
answer
1k
views
LightGBM interpretation of monotonic constraints in multiclass classification
When using LightGBM in classification problems it is possible to use monotonic constraints. In binary classification problems the interpretation is straightforward: "The probability of class (say)...
- 261
2
votes
1
answer
671
views
How do we make predictions for future data when you have lagged dependent features used in training?
I am executing a lightGBM model to forecast my units sold (qty) over a period of time.
Objective is to run a model for each product group and be able to capture the trends, price elasticity, etc and ...
0
votes
1
answer
41
views
Score of LGBM Classifier ranging only between a short interval
I am working on a fraud problem and I am trying to predict either some market/stores has done fraudulent transactions or not.
I've trained a boosting model (lgbm algorithm) on a unbalanced dataset. I'...
2
votes
1
answer
726
views
Gradient boosting on a loss/objective function without second derivatives
In principle, it should be possible to build a gradient boosted tree model on a loss function that only has (nonzero) first derivatives. I've found in practice xgboost and lightgbm make heavy use of ...
- 413
0
votes
0
answers
21
views
t-statistics in gradient boosted machine/forest such LightGBM
Is there a t-statistics in the gradient boosted forest regression model such as that in LightGBM? If so, how is it defined, extracted and used?
- 945
5
votes
2
answers
6k
views
how does `subsample` parameter work in boosting algorithms like xgboost and lightgbm?
From what I know, both of them are sequential learners and only the 1st tree in the sequence gets built on the data and all the following trees that get built are to correct the mistakes of previous ...
0
votes
1
answer
269
views
Lightgbm, time-series and spikes repeated on a yearly basis
I have a data set (time-series) with the shape {$2190$x$63$}. There are 63 variables, 2 products ($A$ and $B$) worth of 3 years of daily data, thus I have $1095$ observations per product and total of $...
- 87
0
votes
0
answers
403
views
When we use k-means clustering with Light GBM, comparing with Random Forest
I am developping the prediction model with many parameters.
As I was not satisfied by the performance of Random Forest Regression, I tried to use
k-means clustering to regroup the similar variable and ...
- 1
0
votes
0
answers
39
views
How to fix the tree structure for a tree-based algorithm?
Background
Some of our BI analysts and most of our managers are interested in making explainable predictions. One of our colleagues proposed an approach based on individual tree leaves from a tree-...
- 165
1
vote
0
answers
607
views
How to get the best num_boost_round on the full training data?
I have a huge training data of size 5.5 GBs with over 55m rows. Because iterating over the whole dataset again and again was too slow, I used a 1% sample of this whole data to select the best ...
- 31
7
votes
1
answer
2k
views
Any reasons to prefer neural networks over boosting methods in tabular data?
Based on Kaggle winners data, it seems that ensemble boosting methods like XGBOOST, LIGHTGBM, CATBOOST are the top choices when dealing with structured or tabular data for maximizing the prediction ...
- 178
0
votes
0
answers
61
views
Predictor With Lower Mean Absolute Error Ends Up Worse
I have been recently working on a problem to estimate the ETAs of vehicles using ensemble techniques such as LightGBM. As expected, the distance taken by the vehicle's route to its destination is a ...
1
vote
0
answers
400
views
How to tune LightGBM parameters to overcome underfitting? [closed]
I'm using LightGBM for a regression task. My training data's shape is (2000000, 1600), which means the number of training data is 2 million +, and each sample has 1600 features. The figure below is ...
- 11
3
votes
1
answer
700
views
Why can EFB(Exclusive Feature Bundling) works in lightGBM?
As I know, EFB can help you to decrease features which are sparse. They put two features together and add offset every feature in feature bundles. They combine features into same histogram.
After ...
- 31