I have binomial data representing the frequency of a trait (genetic marker) from 98 sites in Sweden. Thisdata for an allele is associated with populations living in the mountainous NW part of the country. These mountains run north to south along the Norwegian border. Sites in the mountainswhere sites are nearly fixed for this allele and lowland sites in the lowlands and coast to the east that lack the allele. At the south end of the mountains is a hybrid zone and a region of low abundance for the species. Despite the mountains continuing south through the hybrid zone, the allele frequency shifts from nearly fixed for the allele to becoming nearly absent. This suggests that despite the assumed strong selectionbenefit for this trait in the mountains, that there is a barrier for it in the hybrid zone preventing its spread south alongin the mountains.
I have used a simple glm approach in R to identify that yes, altitude is an extremelya significant predictor of the allele frequency north of (and excluding) the hybrid zone. If I include the hybrid zone sites in the model the altitude remains significant, butaltitude is less sosignificant. If I addedadd a hybrid zone dummy variable to the model for hybrid zone/no hybrid zone. Running the modeland it is also very significant along with altitude.
I would like to find a statistical approach to identify these "outlier" mountain sites (most likely in the hybrid zone) that has a lower than "predicted" allele frequency for this trait. This would support my hypothesis that a barrier to expansion of this presumably beneficial trait occurs in the hybrid zone (where we find low abundance).
I am happy to rephrase my question, provide sample data or just take suggestions for approaching this problem of identifying these "outlier" sample sitesboth statistically and graphically.