How to Handle Infinite Values in Feature Engineering for Machine Learning Models

Question

I'm currently working on a machine learning project where I am creating new features related to the ratio of bytes sent and received in a communications network. However, I'm facing a challenge: when the number of bytes received is zero, the ratio calculation leads to infinite values. This is problematic for my machine learning models.

I'm seeking advice on best practices for handling such infinite values in feature engineering. Specifically, my questions are:

• What are the most effective ways to handle infinite values in features, especially in the context of ratio calculations like bytes sent/received? Should I replace these infinite values with a specific number, or is there a more nuanced approach that would yield better results for machine learning modeling? Are there any standard practices in the industry for dealing with this kind of issue, particularly in network data analysis?

Any insights, references to research papers, or examples from personal experience would be greatly appreciated.

Thank you in advance for your help!

Answer 1

You might be better off modelling the process in a more native form. You send certain number of bytes $N$, and, with some probability $p$, you receive $0\le k \le N$ out of them. The distribution, for $k$, in this setting, would be binomial:

$$ k\,|\,N,p\sim Binomial\left(N,\,p\right) $$

I get the feeling you are actually more interested in the probability of receiving bytes, which is more or less, your ratio. The probability distribution for that, also has a simple form, it is a Beta distribution:

$$ p \,|\,N,k\sim Beta\left(k+1,\,N-k+1\right) $$

For example, if you have sent 5 bytes and received 0, then the 90% confidence interval for $p$ is 1%...39%. Using Python:

import scipy.stats as sp_st
sp_st.beta(0+1, 5-0+1).ppf([0.05, 0.95])

You get: array([0.00851244, 0.39303777]).

If you want a single value, you could take the mean or median of the implied $p$.