I am looking to build a regression model to understand the impact of signing up on a website with sales. I took a random 200k sample of active customers (out of the 30 million or so).. it turns out none of them signed up on the website. So then, I took the population of people who signed up on the website in a given month (~30k).

So I have a dataset that consists of a random sample of 200k customers who did not sign up and 30k customers who did sign up.... I want to build my regression off of this.

Is there anything I need to take into consideration here? Do I need to apply sample weights or something like that, or is my test design ok?

Thank you for the help.


The way you've constructed this, the test design sounds quite simply as sales=A*signup, where sales is your sales indicator, A is your parameter, and signup is your flag variable for signing up.

As far as weighting, think about reasons why you might weight your customers differently. Is a certain customer more important to your question? (Maybe people in a certain country, people who purchased in the month of interest, etc). If you can clearly identify that weighting factor and have the data for it, I'd recommend adding that factor in as an additional variable in your model rather than directly adjusting each unit by an arbitrary weight.

One thing jumped out at me immediately. The sampling method you used isn't the same between the 200k and the 30k. You found the 30k from a given month. To eliminate error due to unequal sampling and potentially time dependencies, I would recommend you sample another 30k non-signups from the same month. Run the model using these two groups of 30k and set the 200k aside.

  • $\begingroup$ When do people use survery weights or sampling weights then? $\endgroup$ – blast00 May 10 '14 at 17:21
  • $\begingroup$ It usually makes a lot more sense when compiling descriptive statistics as opposed to running a linear model like you are. Sampling methods may have certain biases that would mess up your descriptive statistics, thus weights can be used to adjust. Let's say you were trying to profile your customer sales by country, but you knew your database has half as many European customers as you actually have (maybe half were deleted). So your random sample would underrepresent the Europeans by a factor of 2x. Thus you would upweigh each European customer by 2x. $\endgroup$ – Gary Chung May 11 '14 at 1:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.