Is there a way to estimate the number of unique monthly visitors to a site based on a limited sample of one week of data? I have information about when a given user visited the site. This isn't as simple as just multiplying the number of unique visitors the first week by 4, due to the hotel problem. If 10 people visit your site the first week and the same people are the only visitors to your site the second, third, and fourth week, the total number of monthly unique visitors to your site is only 10.

I shared this problem with some of my friends, and the best solution we were able to come up with was plot the total number of unique visitors over time and run a function to fit the curve.

  • $\begingroup$ Do you have historical data from this site? Or from a similar website? It's hard to do this in general without knowing more about the visitor patterns (e.g. Maybe people check back every two weeks), but easy to fit a regression model, month ~ week $\endgroup$ – Eoin Aug 2 at 8:35
  • $\begingroup$ No prior data from the site, but I do have data from similar sites. $\endgroup$ – malisper Aug 2 at 8:50

It's hard to do this in general without knowing more about the visitor patterns (e.g. Maybe people check back every two weeks). Fortunately, if you have historical data from a similar website (per the comments), you can use this to predict unique monthly visitors. There's a lot you could do here, so I'll go through the simple options, with pointers to more complicated approaches in bold.

1. Inspect Historical Data

Let's assume you have the historical containing weekly and monthly unique visitor counts for other similar sites. You can start by plotting monthly ~ weekly (weekly on x-axis, monthly on y) and seeing what the relationship is. Hopefully, it will be straight line, e.g. Monthly visitors = 3.5 * Weekly visitors.

2. Fit a simple model

If the relationship is linear, you can fit a linear regression model.

$$\text{Month} = \alpha + \beta \text{Week} + \epsilon$$

This should ideally be a poisson regression, and if the relationship is not linear you can use polynomial (linear or poisson) regression, but linear regression is good enough for now.

You can then plug in the number of weekly unique visitors to your new site, and get a predicted number of monthly visitors, given the trends on the other sites. This approach assumes that the trends are the same across all sites. If this isn't the case, your simple model will be overconfident, in that it underestimates how uncertain you should be about the predictions it makes. More on this below.

3. Fit a more complicated model

You probably have other information about your sites, and weekly/monthly predictors (e.g. the number of non-unique visitors). Go ahead and add them to your model and see if the predictions about monthly visitors become more accurate. Now you're doing machine learning!

4. Fit a multilevel model

Since you have data from multiple websites, this is a good place to use a multilevel regression model, where the relationship between predictors (weekly visits) and outcomes (monthly visits) is allowed to vary between websites:

$$ \text{Month} = \alpha_{\text{site}} + \beta_{\text{site}} \text{Week} + \epsilon \\ \alpha_{\text{site}} \sim N(\mu_{\alpha}, \sigma_{\alpha}) \\ \beta_{\text{site}} \sim N(\mu_{\beta}, \sigma_{\beta}) $$

or, using lme4,

# Linear regression
lmer(month ~ 1 + week + (1 + week|site), data = data)
# Poisson regression
glmer(month ~ 1 + week + (1 + week|site), data = data, family = poisson(link = "log"))

However, this is a fairly advanced topic, and may not be worth the effort for your problem!

| cite | improve this answer | |

After digging into this for a bit, I came across this paper which provides a solution. The paper gives an approach to estimating the number of new species that will be observed given an initial sampling period. It models observing a given species as a Poisson distribution. It gives the following estimator for the number of new species that will be discovered:

$$ \hat{\Psi}(t)=\sum_{k=1}^{k_{m a x}} N_{k} e^{-k}-\sum_{k=1}^{k_{m a x}} N_{k} e^{-k(1+t)} $$

where $N_{k}$ is the number of species that were observed $k$ times and $t$ is the length of the second sample relative to the initial sample.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.