# Very low $R^2$ linear model, trying to improve it

I have data from a content website where the primary goal is to have users spend more time reading content.

Here is a link to the data set (too large to paste) which can be downloaded and read into R as a csv.

eg_data <- read.csv("eg_data.csv", header = T, stringsAsFactors = F)
> glimpse(eg_data)
Observations: 35,326
Variables: 7
$ID <int> 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, ...$ pageType        <chr> "cats", "dogs", "bunnies", "cats", "cats", "cats", "cats", "cats", "cats", "dog...
$word_count <int> 1267, 1892, 345, 1565, 1941, 1267, 1271, 1565, 1941, 1892, 1271, 1941, 1892, 20...$ deviceCategory <chr> "desktop", "desktop", "mobile", "mobile", "mobile", "mobile", "mobile", "mobile...
$userType <chr> "Returning Visitor", "Returning Visitor", "Returning Visitor", "New Visitor", "...$ Medium         <chr> "carrots", "carrots", "oranges", "apples", "oranges", "oranges", "oranges", "or...
$timeOnPage <int> 3, 87, 16, 150, 318, 109, 29, 48, 35, 0, 149, 264, 98, 165, 69, 25, 0, 5, 132, ...  Target variable is timeOnPage. Having explored the data I have found considerable variation within the variables. For example, device can be one of either of Tablet, Mobile or Desktop: library(tidyverse) # device category eg_data %>% group_by(deviceCategory) %>% summarise(AvgTime = round(mean(timeOnPage), 0), countn = n()) %>% arrange(-AvgTime) # A tibble: 3 x 3 deviceCategory AvgTime countn <chr> <dbl> <int> 1 desktop 192 3052 2 tablet 182 4791 3 mobile 155 27483  Clearly some variation, particularly between smaller screens (mobile phones) and either tablet or desktop. Similar across source of traffic (Medium): # medium eg_data %>% group_by(Medium) %>% summarise(AvgTime = round(mean(timeOnPage), 0), countn = n()) %>% arrange(-AvgTime) # A tibble: 8 x 3 Medium AvgTime countn <chr> <dbl> <int> 1 pinapple 280 2 2 apples 178 308 3 carrots 168 15 4 oranges 164 817 5 coconuts 163 32512 6 pears 150 1311 7 cabbages 147 360 8 peas 44 1  Here's the breakdown by pageType > # pageType > eg_data %>% + group_by(pageType) %>% + summarise(AvgTime = round(mean(timeOnPage), 0), + countn = n()) %>% + arrange(-AvgTime) # A tibble: 6 x 3 pageType AvgTime countn <chr> <dbl> <int> 1 cats 175 19261 2 dogs 167 1623 3 birds 157 3435 4 horses 152 2601 5 bunnies 141 7738 6 fish 73 668  Because there is some variation among the predictors, I believe that my explanatory variables should at least be reasonably good predictors of target variable time spent. Here's my attempt at building a simple linear model: # tuning & parameters set.seed(123) train_control <- trainControl( method = "cv", number = 5, verboseIter = TRUE ) # train a linear model lm_mod <- train( timeOnPage ~ word_count + pageType + deviceCategory + userType + Medium, data = eg_data, method = "lm", trControl = train_control )  After running this if I type lm_mod into the console I see the following statistics:  RMSE Rsquared 223.6664 0.01054297  Both of these numbers are very poor given the data. Given mean(eg_data$timeOnPage) = 162.059 and RMSE is over 200.

I have experimented by stripping some variables out but resulting changes to the evaluation metric are minimal.

Anyone familiar with web analytics will know that both device category and medium variables are typically important when analysing performance, be it conversion to transaction, sign up or in this case time reading.

I tried a decision tree and found similarly poor RMSE. I prefer not to use e.g. Random Forrest or any other method because I like the ability of lm to explain relationships using the coefficients (So hoping to stay within linear model framework).

Having reached a point of desperation I'm hoping that by posting a disguised copy of the data someone can verify that my data are indeed poor predictors of timeOnPage, or, hopefully, point out a way of improving this simple linear model wrt RMSE and $R^2$.

• Any particular reason why the response and e.g. count are left untransformed? – Michael M Nov 5 '17 at 11:00
• @MichaelM why would they be? Please expand if I have missed something or you have a suggestion? – Doug Fir Nov 5 '17 at 16:02

There are perhaps some small things that can be done to improve your model. First of all the relation with word_count is probably nonlinear. So adding it like this doesn't help, it only hurts the model. There might be a sweet spot of the number of words for an article, and you should use some transformation that allows the model to find this sweet spot. Second, the smallest categories are probably overfit, so for example the group apples has a high average time, but it's not a big group, so this could be just coincidence. In your model, it has its own coefficient, which probably overestimates the actual average time spent on those pages. For regularization, I would look at a mixed model with lme4, or something like LASSO from the glmnet package.
Conversely, as in, there your model might not be flexible enough, there are no interactions in it. But short articles may be more popular on mobile, while long articles are more popular on desktop. There is no freedom for the model to find such a pattern at the moment. I don't think adding this interaction (wordcount * channel) will immediately improve things, but combined with some nonlinearity in wordcount, and regularization, it could help a bit.
All this might lift your rsquared a bit, but I wouldn't count on getting up to 10%. Whether or not someone stays longer at the page has a lot to do with whether it's actually what he or she is looking for, whether nothing IRL happens that stops him or her from reading on, perhaps if it's a fast or a slow reader, probably patience has a role in it. None of that can be 'predicted' from your data.