# How to find if people's preference is significant

I am trying to find out the wheither people prefer ads on Instagram or Facebook. As in are they more likely to choose x or y? Each person is given if you had to choose from and they pick x or y. Those are added together per person and then for the whole sample size. How should I best evaluate this data? Just by looking at mean and standard deviation? Or should I use a paired or unpaired t test I can't figure out which one fits. Or a chi square goodness of fit.

• Please can you clarify what you mean by 'Those are added together by person' - what is it that you are adding together? The number of times that each person chooses Instagram and the number of times they choose Facebook? – Izy May 2 '19 at 11:09

As you give no exact details about your data, I will give an example that you can adjust to however your data looks. I'm basing this on an example from the book 'Statistics' by Freedman et al., chapter 26 'Zero-one boxes'.

## Example of applying the z-test here

Let's investigate the number of people that preferred Facebook in your example and formulate the given box model: the choice is between 'Prefers Instagram' , which we'll represent as a piece of paper with 0 written on it, while the other option 'Prefers Facebook' is represented by a similar paper with 1 on it.
We can consider your data collection like drawing from this box with replacement. Our null hypothesis will be "There is no user preference, both options are equally likely and any variations are merely by chance" .

Let us, for the sake of argument, assume this is true and see whether it holds water.
If the null hypothesis is true and both options are equally likely, then we would expect to see roughly half of the draws to be 0s and the other half to be 1s.
Assume we ask 10000 people about their preference in this regard, of which 4200 report preferring Instagram and 5800 report preferring Facebook.

We can use the z-test to evaluate the null hypothesis by computing the test statistic $$z$$ as

$$z=\frac{\textrm{observed} - \textrm{expected}}{\textrm{standard error}}$$

We observed 5800 1s in our sample, while we expected (assuming the null hypothesis is valid) 5000 1s. Is this difference purely by chance?

Considering the null hypothesis, we compute the standard error as follows

$$\textrm{standard error}=\sqrt{\textrm{number of draws from box}} \cdot \textrm{standard deviation of box}$$ with $$\textrm{standard deviation of box} = \sqrt{P(1)\cdot P(0)} = 0.5$$ which gives us $$\textrm{standard error}=\sqrt{10000} \cdot 0.5=5000.$$

Using this we can determine $$z$$ as

$$z=\frac{5800 - 5000}{5000}=0.16$$

This tells us, that the value we observed in our sample is 0.16 standard errors above the expected value. This gives us P $$\approx$$ 0.436441. This leads us to accpet the null hypothesis on basis of all common significance levels. We conclude that the observed difference in our sample is likely due to chance.

## Note

As previously mentioned, I made some assumptions to have a concrete enough example. The most problematic assumption of these is the sample size of 10000. One has to consider that from what I've read the z-test is unreliable with small sample sizes. In such cases, a t-test might be better suited.

I'm still somewhat new to hypothesis testing, so if I made any errors in this answer I'd be glad to see them pointed out.