7
$\begingroup$

This question is often asked during interviews, e.g. what would the distribution of time spent per day using YouTube or Facebook look like?

If the data do not contain imperfections (e.g., people being AFK on the site or people opening and quickly closing the site), I would expect the distribution to be normal. Is that wrong?

$\endgroup$
5
  • 1
    $\begingroup$ I would not expect a normal distribution (if for no other reason: time cannot be negative). I would expect a clump around very short times (a few minutes) and then a long tail. Maybe other clumps? $\endgroup$ Commented May 19, 2017 at 9:44
  • $\begingroup$ @kjetilbhalvorsen I don't see a sequitur between the fact time cannot be negative and the normal distribution. Weight cannot be negative either, but people's weight does distribute normally. Why would you expect short times to be more likely than long times? $\endgroup$ Commented May 19, 2017 at 10:19
  • $\begingroup$ Well, I sort of agree with your first comment. The second: Why short times dominate? I have never seen such data, so I cannot know, but many people do not really have much time to such things as youtube, but will nevertheless visit shortly from time to time. Then for the long tail, it could be expected to be more spread that the clump close to zero. $\endgroup$ Commented May 19, 2017 at 10:21
  • $\begingroup$ Of course the distribution cannot be exactly normal because times are non-negative. The real question is how well the normal approximates the data. In that regard kjetil gives some suggestions why the normal might not fit well. But to really know requires looking at the data. $\endgroup$ Commented May 19, 2017 at 13:47
  • 1
    $\begingroup$ Beta - you can’t spend less than 0 or more than 24 hours $\endgroup$
    – Aksakal
    Commented Sep 28, 2021 at 21:52

4 Answers 4

2
+25
$\begingroup$

For youtube I think the distribution depends on the distribution of video length. Most videos are about 8 to 10 minutes (this is the average span of the people attention). One might watch 0, 1, 2, or any other number of videos. A Poison distribution might be a good candidate to model the number of visits to a website or the number of videos watched per day. If the distribution of the time of a single video would be $P_V(t)$ then the distribution of total time would be something like $$P_{Total}(t)=\sum_{i}^{} P_p(N=i)P_{Vi}(t)$$ where $i$ is the number of visits, $P_p(N=i)$ is the Poisson distribution and $P_{Vi}$ is distribution of total time of $i$ videos which can be obtained by performing convolution on $P_v$. If $P_v(t)$ is approximately Gaussian then the distribution would be a mixture of Gaussians. Something that looks like the following picture. (I only guessed the distribution, not to scale)

enter image description here

For facebook there might be something similar to a video length. For example the time required to look at one page.

$\endgroup$
2
$\begingroup$

Having the vocabulary to describe a distribution is an important skill as a data scientist when it comes to communicating ideas to your peers. There are 4 important concepts, with supporting vocabulary, that you can use to structure your answer to a question like this. These are:

Center (mean, median, mode) Spread (standard deviation, inter quartile range, range) Shape (skewness, kurtosis, uni or bimodal) Outliers (Do they exist?) In terms of the distribution of time spent per day on Facebook (FB), one can imagine there may be two groups of people on Facebook:

People who scroll quickly through their feed and don’t spend too much time on FB. People who spend a large amount of their social media time on FB. From this point of view, we can make the following claims about the distribution of time spent on FB, with the caveat that this needs to be validated with real world data.

Center: Since we expect the distribution to be bimodal (see Shape), we could describe the distribution using mode and median instead of mean. These summary statistics are good for investigating distributions that deviate from the classical normal distribution. Spread: Since we expect the distribution to be bimodal (see Shape), the spread and range will be fairly large. This means there will be a large inter quartile range that will be needed to accurately describe this distribution. Further, refrain from using standard deviation to describe the spread of this distribution. Shape: From our description, the distribution would be bimodal. One large group of people would be clustered around the lower end of the distribution, and another large group would be centered around the higher end. There could also be some skewness to the right for those people who may spend a bit too much time on FB. Outliers: You can run outlier detection tests like Grubb’s test, z-score, or the IQR methods to quantitatively tell which users are not like the rest.

$\endgroup$
1
$\begingroup$

The answer is wrong in the sense that you do not explain how you arrived at this answer. The reason that they ask a question like this during in an interview is to see how you think.

One way to answer such a question would be to say that you don't know but that you can make an educated guess. Let's assume that if a person is visiting a site there is a probability $p$ after one unit of time $t$ has passed that she will leave the website. With a probability of $p$ her visit will be limited to $1$ unit of time. With a probability of $(1-p)p$ (i.e. the probability she hasn't left times the probability she will) her visit will be limited to $2$ units of time. With a probability of $(1-p)^2p$ her visit will be limited to $3$ units of time. Etc. The probability mass function of this distribution is therefore $(1-p)^t p$. This the geometric distribution.

Note: I'm not saying that this is correct but this is a correct answer for the interview. You might also complicate it a bit by then saying that perhaps $p$ is a function of $t$ etc.

$\endgroup$
5
  • 3
    $\begingroup$ Asks the interviewer:-"How many units of time are there in a day?" $\endgroup$
    – Scortchi
    Commented May 22, 2017 at 9:27
  • $\begingroup$ @Scortchi, the right answer to that is that it depends. We can treat the length of a unit of time as a another free parameter to make the distribution fit the data. We can also say that the unit of time is the optimal width of a histogram bin as given by for example the Freedman - Diaconis rule. I'm sure that there are other possible answers but these two seem the most sensible to me right now. I have a slight preference for the latter because this means that the histogram approximates the CDF. And it is more likely that the geometric distribution can be wrong. Meaning we learn something. $\endgroup$
    – user31790
    Commented May 22, 2017 at 10:01
  • 1
    $\begingroup$ Treating the length of a time unit as a parameter to be estimated is essentially ignoring any aspects of the data that don't fit in with your model! What I was hinting at was that (1) there isn't really any need to pick a more or less arbitrary time unit - make it small enough & you end up with an exponential distribution, the continuous analogue of the geometric; & (2) there are, notoriously, only so many hours in a day, & perhaps a model ought to take that into account. $\endgroup$
    – Scortchi
    Commented May 22, 2017 at 10:28
  • $\begingroup$ I'm not that sure it does. The probability for every unit of time is still the same. This places a restriction on the distribution. In addition, if you really believe the model then this is the way to go. Ignoring any aspects of the data that don't fit the model isn't a vice if the model is true. The lack of fit might be due to e.g. measurement error: i.e. lack of fit doesn't logically imply that it is the model that must be wrong. $\endgroup$
    – user31790
    Commented May 22, 2017 at 11:23
  • $\begingroup$ Re (1) Yes, that's true. What I don't like about picking an arbitrary small time interval though is that you can then hide behind the fact that is an approximation when there is a discrepancy between the model and the data. I'm not sure if this is worse than picking the time interval to fit the data. It probably isn't. Re (2) I don't follow? You mean that the support of the distribution should take into account that there is a maximum? $\endgroup$
    – user31790
    Commented May 22, 2017 at 11:26
1
$\begingroup$

This type of question raises a lot of issues that would be worth talking about. I can only speak for myself, but I would want to make the following points.

  • The most basic properties you would expect for this distribution is that it would be continuous and have non-negative support. You could quibble with the first of these properties if you were to measure time in discrete increments (e.g., measurement goes down to seconds), but even in that case, it would be best to think of the actual time spent on the site as continuous, but then it is discretised by the measuring instrument.

  • Given that you have a large number of users operating separately from each other, with immense variation in the length of activities on the sites (e.g., different video lengths on YouTube), it would be reasonable to think that the distribution would have a smooth density and would be quasi-concave (i.e., unimodal). Since we are looking at the amount of time spent on the site per day the distribution would also be truncated at one day (i.e., this is the upper bound on the outcome).

  • The time spent on the site can be framed as a "truncated survival process" and so it would be reasonable to consider standard distributions in survival analysis as starting points for speculation. One could reasonably speculate on the distribution by thinking about the likely shape of the "hazard function" for the process. Most likely, the hazard function would start off low and then get larger over time (i.e., the user is more likely to leave the longer they have been on the site).

  • It is generally bad practice to assume away nasty empirical realities, such as a person being away-from-keyboard or opening and then quickly closing the site. There is no particular reason that either of these realities would cause problems in speculating on the distribution. I see no reason to remove them from consideration.

Now, if one agrees with the above ideas, some reasonable speculations for the shape of the distribution would be the truncated gamma distribution or the truncated Weibull distribution. The latter has a imple hazard function that can be set to be exponentially increasing. Finally, it is important to note that these are just speculations. Ultimately, if we want to know the distribution of the time on these sites per day, we need to obtain data on that outcome and let the data "speak for itself". It may be that standard parametric distributions do not fit this data particularly well, in which case we might fall back on non-parametric analysis.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.