I have the following dataset:
userid days1 days2 days3 avg_days 1 Nan 0 3 1.5 2 6 8 1 5 3 0 0 0 0 4 1 5 Nan 3 5 NaN NaN NaN NaN 6 0 0 0 0 7 1 Nan Nan 1 8 NaN NaN NaN NaN
In this dataset, I have the information regarding how early user makes a request about items 1, 2 and 3. In some cases, users may not have requested the item, which are indicated as
NaN. We try to calculate the avg_days (the average days until the request was made after the release of the item). We want to correlate this variable with some other behaviours of the users.
To explain the data better, for example, user #1, didn't request the item 1 (i.e.,
NaN, and requested the item 2 on the same day (i.e., 0, when the item was released), and requested the item 3, after 3 days of its release.
However, for users with no requests (e.g., user 5 and user 8), it is impossible to calculate the avg_days. As a solution to this issue we have considered replacing
NaN values (in
avg_days column) with the maximum
avg_day value calculated (which is 5 in this case):
userid days1 days2 days3 avg_days 1 Nan 0 3 1.5 2 6 8 1 5 3 0 0 0 0 4 1 5 Nan 3 5 NaN NaN NaN 5 * replaced 6 0 0 0 0 7 1 Nan Nan 1 8 NaN NaN NaN 5 * replaced
Our logic was that as the higher
avg_days values would indicate negativity (delay of requesting the item), it would be reasonable to place the most negative value for the users who have not request anything. This way, we were able to include them in the analysis, and we obtained some meaningful correlations. Without this (i.e., when the missing data was discarded), there were no significant correlations.
However, we are not sure about the validity of this approach. I wonder if someone has opinion on this and if such approaches have been used in the literature? Thanks for help!