3
$\begingroup$

When training a LightGBM model with lgbm.train, I get the following warning:

[LightGBM] [Warning] Met categorical feature which contains sparse values. Consider renumbering to consecutive integers started from zero

The reason is that I have a column, specified as categorical, that contains the following integers:

[1015, 1033, 1128, 1398, 1541, 1673, 1677]

In the documentation at lightgbm.train.html it says:

"All values in categorical features will be cast to int32 and thus should be less than int32 max value (2147483647). Large values could be memory consuming. Consider using consecutive integers starting from zero."

My values are not particularly large. The "considering using consecutive integers starting from zero" seems to be a suggestion. What happens if they do not? How does the sparseness affect the performance of LightGBM? Another categorical column of the dataset has the values

[1, 3, 4]

and it does not cause the same warning.

Improve this question
$\endgroup$
2
  • $\begingroup$ Questions solely about how software works are off-topic here, but you may have a real statistical question buried here. You may want to edit your question to clarify the underlying statistical issue. You may find that when you understand the statistical concepts involved, the software-specific elements are self-evident or at least easy to get from the documentation. $\endgroup$
    – kjetil b halvorsen ♦
    Commented Mar 25 at 17:24
  • 1
    $\begingroup$ @kjetilbhalvorsen Right, I was contemplating if Cross Validated was the right forum for the question as it more focused on the he LightGBM implementation. Perhaps I should re-iterate the question in another forum. It does however also touch on the underlying algorithm for handling categorical values, but again, main focus was on LightGBM's warning. $\endgroup$
    – DustByte
    Commented Mar 26 at 8:44

1 Answer 1

Reset to default
1
$\begingroup$

We are safe to ignore this warning, given we don't have any memory issues. In practical terms, the histogram binning algorithm will have to do a bit more work when it gets initialised. It should have no other practical implications.

Ultimately, when doing histogram binning it is not that values' magnitude that is the main issue, but rather the (implied) difference between successive values. Large gaps between distinct values, often suggest a non-uniform distribution and obviously a histogram binning procedure isn't thrilled with that prospect, ergo the warning. For this particular case, where there are 7 distinct values, the check in binning complains as it would expect actual numerical values no greater than 700 for a categorical variable with 7 levels. (See bin.cpp#L431 for the exact code and the SPARSE_RATIO used being 100.)

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.