2
$\begingroup$

I have website page load data. Every row represents one visitor, and the attributes are pages they have visited. However, they often only visit 2-3 pages, so most values are null. The values are how long it took for the page to load in seconds. The target_flag is 0/1 indicating conversion. I want to know whether longer page load times lead to lower conversion and whether it's certain pages that may affect the conversion rate. 

visitor_cookie  Page A    Page B   Page C   Page D   Page E    Target

10ed4da0e0      .4        NA       .6       NA       NA         0

18f7746         .3        .4       NA       NA       NA         1

1ffdc2f6        0.527     NA       NA       NA       4.05733    0

226b9dc52f      NA        .3       NA       NA       3.077      0

241a6095a8      NA        .7       .4       .8       NA         1

I want to do some sort of logistic/GLM model, but I'm not sure how to handle all the NAs. I can't just remove them because every record has nulls and I can't just impute 0s, because every column has too many nulls. I thought about changing the dataframe from wide to long, so that every record represents one page load time (instead of every record being one customer visit). However, it seems like I can't do that, because then I'm removing an important factor that certain records are related.

How can I account for these varying null values? Thank you.

Improve this question
$\endgroup$
5
  • $\begingroup$ Rather than think about imputing the page-load times of pages that a visitor didn't visit - which you can safely assume will have no effect at all on their probability of conversion - , think about encoding the fact that they didn't visit those pages as a predictor in your model. $\endgroup$
    – Scortchi ♦
    Commented May 10, 2018 at 19:44
  • $\begingroup$ Thank you for the suggestion. I'm not sure how to do that though. I suppose I could add a column for every page that is 0/1 for whether they visited that page or not. But then I'd still have NAs in the original columns. $\endgroup$
    – pythonnoob
    Commented May 10, 2018 at 20:01
  • $\begingroup$ Consider what, after including such a set of indicator variables as predictors, will be the effect of setting a page-load time of 'NA' to 0, or to 42 , or to any other value, on your model. $\endgroup$
    – Scortchi ♦
    Commented May 10, 2018 at 20:42
  • $\begingroup$ hmmm, so if I set a "PageAIndicator" variable as 0 when "Page A" is NA (or 0 or 42). Then would my model just not take into account the weight of the "Page A" value? But wouldn't it still affect the final result when I don't want it too? Maybe I could account for that by adding them as an interaction effect in the model? Like PageAIndicator*PageA? Thank you for helping me think through this. $\endgroup$
    – pythonnoob
    Commented May 10, 2018 at 23:52
  • $\begingroup$ See e.g. stats.stackexchange.com/a/105258/17230. And you're welcome. $\endgroup$
    – Scortchi ♦
    Commented May 11, 2018 at 8:46

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.