I am wondering if there is a simple way of detecting outliers.

For one of my projects, which was basically a correlation between the number of times respondents  participate in physical activity in a week and the number of times they eat outside the home (fast food) in a week, I drew a scatterplot and literally removed the data points that were extreme. (The scatterplot showed a negative correlation.)

This was based on value judgement (based on the scatterplot where these data points were clearly extreme). I did not do any statistical tests.

I am just wondering if this is a sound way of dealing with outliers. 

I have data from 350 people so loss of (say) 20 data points is not a worry to me.