4
$\begingroup$

I have a data feature that follows closely a bimodal distribution (mixture of two separate normal distributions with different mean, standard deviation and weights).

Is it meaningful to transform that feature in the 2 following features ?

  1. A factor corresponding to one of the two the normal distributions
  2. the distance to the mean of the selected normal distribution

Here is an example :

set.seed(122)

# fake bimodal data 
bimodal <- as.data.frame(
  c(rnorm(10000, mean=10, sd=5),
    rnorm(2000, mean=22, sd=2)))
names(bimodal) <- c("x")

ggplot(bimodal, aes(x=x)) + geom_histogram(binwidth = 0.5)

bimodal histogram

Estimating the mean and the boudary between the two normal distributions :

densityCurve <- ggplot(bimodal, aes(x=x)) + geom_density()
densityCurveData <- ggplot_build(densityCurve)

localMax <- which(ggpmisc:::find_peaks(densityCurveData$data[[1]]$density) == TRUE)
localMax <- densityCurveData$data[[1]]$x[localMax]
    
localMins <- which(ggpmisc:::find_peaks(-densityCurveData$data[[1]]$density) == TRUE)
localMins <- densityCurveData$data[[1]]$x[localMins]
localMins <- c(-Inf, localMins, +Inf)

ggplot(bimodal, aes(x=x)) + geom_histogram(aes(y = ..density..), binwidth = 0.5) + geom_vline(xintercept = localMax, color="red", linetype="dashed") + geom_vline(xintercept = localMins, color="blue", linetype = "dashed") + geom_density(lwd=1, col="green", linetype = "dotted")

Means and boundary

Data Transformation :

bimodal$Mode <- cut(bimodal$x,
            breaks=localMins,
            right=TRUE)

returnMode <- function(modes, x) {
    distances <- sapply(modes, function(i) (x - i))
    distances[which(abs(distances) == min(abs(distances)))][1]
}

bimodal %<>% mutate(distToMode = sapply(x, returnMode, localMax))

ggplot(bimodal, aes(x=x, y=Mode)) + geom_jitter()
ggplot(bimodal, aes(x=distToMode)) + geom_density()
ggplot(bimodal, aes(x=distToMode)) + geom_histogram()

Visualization of the two new features created

Test the normality of distToMode :

shapiro.test(bimodal$distToMode[sample(5000)])
Shapiro-Wilk normality test

data:  bimodal$distToMode[sample(5000)]
W = 0.9801, p-value < 2.2e-16

Do you believe it's an efficient way to do? Thanks,

StayHome

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.