I recently received this email from a graduate student, and I get similar questions often enough, that I thought I'd post it here:
I'm using factor analysis, multiple regression, and SEM and currently checking statistical assumptions. I have found numerous univariate and multivariate outliers. If I deleted them all, it would mean a large chunk out of my sample size ($N \approx 350$). I also have problems with non-normality, non-linearity, heteroscedasticity (Multiple regression), and large standardised residual covariances (SEM).
I have tried reducing the influence of the outliers (allocating them a value one unit larger/smaller than the next most extreme non-outlier value), and transformations (mostly the variables remained skewed and some outliers remain). When I compare original results with altered data, there is little effect. Given this, I am wondering whether it would be acceptable to leave the data as it is? I'm inclined to, particularly because this data is from a non-clinical population and I have used clinical measures.