# How to select subjects in my experiment?

I had two visualization methods (Vis A and Vis B). In order to answer following questions:

1) Which one of these visualization perform better? 2) How different genders (male, female) perform with each of these visualizations? 3) How different people with different academic background (science and non science) perform different tasks using each of these visualizations?

I recruited 40 participants. During the recruitment process I made sure that 20 subjects are from science college (lets say group A) and another 20 are from Art and music college (group B). I then made sure that each group I have 10 male and 10 female. I have no information about background of my participants. What I know is their major and gender.

Now my advisor believes that this process of selecting subjects was not randomly. Is that correct? Can I claim that I picked my subjects randomly or not?

If I understand your experiment correctly, it sounds like you are interested in finding the effect of two different visualisation methods on some visualisation outcome, across four groups defined by different gender and university major. If that is correct then your variables are the following:

\begin{aligned} \text{Response variable} & & = & \quad \text{Visualisation outcome,} \\[6pt] \text{Treatment variable} & & = & \quad \text{Visualisation method (A/B),}\\[6pt] \text{Covariates} & & = & \quad \text{Gender (F/M) + Major (STEM/Art)}. \\[6pt] \end{aligned}

Since your goal is to find the effect of the your treatment variable, this is the one you need to randomise. In this case you can easily use block randomisation over the four blocks defined by your covariates. In each of these blocks you have five people. Within each block, randomly assign a specified number of people into the treatment groups (e.g., visualisation method A) and keep the remaining people in the control group (e.g., visualisation method B). Since this variable is randomly assigned, it is statistically independent from the covariates, and any possible lurking variables, so you should be able to make a causal inference about its effect on the response variable. By using block randomisation you also ensure that your randomisation will tend to yield a treatment vector that is closer to orthogonal to the covariates, thereby reducing variance in the subsequent regression analysis. You also don't have to worry about colliders in this analysis, since gender and university major are not causally affected by the treatment or response.