2
$\begingroup$

Self learning and not quite good at probablility and statistics, my question is regarding solution to exercise 1.2.1 in chapter 1 of Mining of Massive Datasets book.

The text of the exercise reads:

Using Information from Section same 1.2.3, what would be the number of suspected pairs if following changes were made to the data (and all other numbers remained as they were in that section)?

(a) The number of days of observation was raised to 2000.
(b) The number of people observed was raised to 2 billion (and there were therefore 200,000 hotels).
(c) We only reported a pair as suspected if they were at the same hotel at the same time on 3 different days.

Information from Section 1.2.3:

Suppose there are believed to be some “evil-doers” out there, and we want to detect them. Suppose further that we have reason to believe that periodically, evil-doers gather at a hotel to plot their evil. Let us make the following assumptions about the size of the problem:

  1. There are one billion people who might be evil-doers.
  2. Everyone goes to a hotel one day in 100.
  3. A hotel holds 100 people. Hence, there are 100,000 hotels – enough to hold the 1% of a billion people who visit a hotel on any given day.
  4. We shall examine hotel records for 1000 days.

An approximation used in calculations: $\binom{n}{2} = n^2/2$

My Solution:

Number of pairs of people = $\binom{2 * 10^9}{2}$ approx = $\frac{(2 * 10^9)^2}{2}$ = $2 * 10^{18}$

Number of pairs of days = $\binom{2000}{2}$ approx = $\frac{(2000)^2}{2}$ = $2 * 10^{6}$

Probability of two people deciding to visit a hotel on any given day = $10^{-4}$ (same as in section 1.2.3)

Number of hotels = $2 * 10^5$

Chance of a pair visiting the same hotel = $\frac{10^{-4}}{2 * 10^5} = \frac{1}{2} * 10^{-9}$

Chance that they visit the same hotel on 3 different days = $(\frac{1}{2} * 10^{-9})^3 = \frac{1}{8} * 10^{-27}$

Expected number of events that look like evil-doings = $2 * 10^{18} * 2 * 10^{6} * \frac{1}{8} * 10^{-27} = \frac{1}{2} * 10^{-3} = 0.0005$

I am not quite sure about my solution, and would appreciate help of the community to better understand the problem as well as the solution.

$\endgroup$
  • $\begingroup$ If Bonferroni is just a name for this exercise than it has nothing to do with Bonferroni adjustment (as in your tags) and you should rename it to avoid confusion. $\endgroup$ – user2974951 Jan 14 '19 at 8:37
  • 1
    $\begingroup$ The number of pairs of days should be (2000 3), right? $\endgroup$ – Aarth Tandel Jan 14 '19 at 22:06
0
$\begingroup$

There are 2 billion people, everyone goes to a hotel (out of 200,000 hotels and average 100 people capacity/hotel) 1 day in 100, examine hotel records for 2000 days: look for people who, on 3 different days, were both at the same hotel, the number of pairs of people is

C(2x109, 2) ≈ 2 X 1018, the number of two people meet of 3 same days is C(2000, 3) ≈ 1.3 X 109

so the number of pairs of people who look like evil-doers is 2 X 1018 X 1.3 X 109 X (0.012 / 200,000)2 ≈ 0.325

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.