I am working with a distribution that has outliers beyond
1.5*3rd Qu.. I'm using Shankar, et al. Recommendations for the validation of immunoassays used for detection of host antibodies against biotechnology products (2008), as a guide and it suggests (if outliers cannot be removed):
Data transformation is often needed in order to satisfy the distributional assumptions of the statistical analysis (e.g., ANOVA). Low and high outliers arising from analytical or biological abnormalities should preferably be excluded, or appropriately down-weighted (e.g., by use of Median and Median Absolute Deviation or Tukey’s biweight function) in the determination of a screening cut point.
The distribution can be created with:
set.seed(12) tmp <- rnorm(100, mean = 100, sd = 1) set.seed(12) outliers <- rnorm(10, mean = (1.5 * summary(tmp)[["3rd Qu."]]), sd = 1) ## Bind the columns together dat <- data.frame(cbind(c(tmp, outliers), c(rep("Expected", each = length(tmp)), rep("Outlier", each = length(outliers))))) ## Add Column Names colnames(dat) <- c("Units", "Category") ## Convert ECL values to numeric dat$Units <- as.numeric(as.character(dat$Units))
I tried to normalize through a log transformation, as well as a median-based transformation, but I wasn't able to generate a normal distribution.
How could I appropriately weight the distribution to normalize it?