How to select subjects in my experiment? I had two visualization methods (Vis A and Vis B). In order to answer following questions:
1) Which one of these visualization perform better? 
2) How different genders (male, female) perform with each of these visualizations?
3) How different people with different academic background (science and non science) perform different tasks using each of these visualizations?
I recruited 40 participants. During the recruitment process I made sure that 20 subjects are  from science college (lets say group A) and another 20 are from Art and music college (group B). I then made sure that each group I have 10 male and 10 female. I have no information about background of my participants. What I know is their major and gender. 
Now my advisor believes that this process of selecting subjects was not randomly. Is that correct? Can I claim that I picked my subjects randomly or not?
I appreciate answers with reference. 
 A: If I understand your experiment correctly, it sounds like you are interested in finding the effect of two different visualisation methods on some visualisation outcome, across four groups defined by different gender and university major.  If that is correct then your variables are the following:
$$\begin{equation} \begin{aligned}
\text{Response variable} & & = & \quad \text{Visualisation outcome,} \\[6pt]
\text{Treatment variable} & & = & \quad \text{Visualisation method (A/B),}\\[6pt]
\text{Covariates} & & = & \quad \text{Gender (F/M) + Major (STEM/Art)}. \\[6pt]
\end{aligned} \end{equation}$$
Since your goal is to find the effect of the your treatment variable, this is the one you need to randomise.  In this case you can easily use block randomisation over the four blocks defined by your covariates.  In each of these blocks you have five people.  Within each block, randomly assign a specified number of people into the treatment groups (e.g., visualisation method A) and keep the remaining people in the control group (e.g., visualisation method B).  Since this variable is randomly assigned, it is statistically independent from the covariates, and any possible lurking variables, so you should be able to make a causal inference about its effect on the response variable.  By using block randomisation you also ensure that your randomisation will tend to yield a treatment vector that is closer to orthogonal to the covariates, thereby reducing variance in the subsequent regression analysis.  You also don't have to worry about colliders in this analysis, since gender and university major are not causally affected by the treatment or response.
There are two issues here that need to be de-coupled.  The first is the requirement of randomisation for causal inference in an experiment.  The second is the requirement of random sampling for inference about a broader population.  It is worth noting that if your treatment variable is the visualisation method then randomisation of this variable allows causal inference, but your inference is still confined to your sampling frame.  In other words, so long as you randomise your treatment variable, you can legitimately infer the effect of this variable for the population of people corresponding to your non-random sampling method.  Discuss this further with your advisor to see what it is he wants you to make an inference about.  If he wants you to make an inference about the entire body of college students then it would be appropriate to use a random sampling method to sample from this population.
A: You haven't described any mechanism by which individuals were randomly assigned to groups. As written this is not a random allocation. Also, you don't mention how recruitment took place. Paying attention balance in design can  make analysis easier, but if your goal is to make unbiased inferences about the population(s) from which your subjects were drawn you also need to a) sample randomly from the target population(s) and b) randomly allocate subjects from the sample to groups (possibly stratified by academic background and sex).
There is a thread here that offers suggestions of books that may help.
