Perhaps you are trying to unbury some data that doesn't exist in raw form. In any case, I don't think its a good idea to standardize the data and then run mean/rank difference tests.
Doing any form of t-test here wouldn't work on standardized data for the simple reason that the mean will by definition equal zero for both groups, and so it follows there will be no statistically significant difference. Converting the z-scores to percentiles doesn't fix this issue because you are now just testing location differences in the same way. A ranked test doesn't fix this issue either for the reasons you stated already. As proof of that, here are those tests on simulated data in R.
#### Sim Data ####
library(tidyverse)
set.seed(123)
x <- rnorm(100,mean=50,sd=10)
y <- rnorm(100,mean=60,sd=10)
df <- data.frame(x,y)
data <- df %>%
gather() %>%
as_tibble()
data
#### Test ####
t.test(data$value ~ data$key,
paired=T)
#### Standardize ####
zx <- scale(x)
zy <- scale(y)
zdf <- data.frame(zx,zy)
zdata <- zdf %>%
gather() %>%
as_tibble()
zdata
#### Test Again ####
t.test(zdata$value ~ zdata$key,
paired=T)
#### Percentiles ####
px <- pnorm(zx)
py <- pnorm(zy)
pdf <- data.frame(px,py)
pdata <- pdf %>%
gather() %>%
as_tibble()
pdata
#### Test Once More ####
t.test(pdata$value ~ pdata$key,
paired=T)
The first t-test on raw data shows what we expect, a statistically significant difference:
Paired t-test
data: data$value by data$key
t = -5.8876, df = 99, p-value = 5.379e-08
alternative hypothesis: true mean difference is not equal to 0
95 percent confidence interval:
-10.723482 -5.317464
sample estimates:
mean difference
-8.020473
The second test of z-scores is fairly nonsensical, as now the mean difference is zero (or very very close to it with floating points shown by the scientific notation), and thus $p = 1$:
Paired t-test
data: zdata$value by zdata$key
t = 7.6905e-16, df = 99, p-value = 1
alternative hypothesis: true mean difference is not equal to 0
95 percent confidence interval:
-0.2874763 0.2874763
sample estimates:
mean difference
1.114216e-16
Percentile differences have similar issues:
Paired t-test
data: pdata$value by pdata$key
t = 0.2507, df = 99, p-value = 0.8026
alternative hypothesis: true mean difference is not equal to 0
95 percent confidence interval:
-0.07050474 0.09089738
sample estimates:
mean difference
0.01019632
If you mean that you want to run a t-test on the difference scores of percentiles alone, this isn't possible because you are only using one vector of scores (there is no comparison to another group). To get a sense of what this looks like visually, here is the code to plot their densities:
#### Plot Them ####
p1 <- data %>%
ggplot(aes(x=value,
fill=key))+
geom_density(alpha=.4)+
geom_vline(xintercept = mean(x))+
geom_vline(xintercept = mean(y))
p2 <- zdata %>%
ggplot(aes(x=value,
fill=key))+
geom_density(alpha=.4)+
geom_vline(xintercept = mean(zx))+
geom_vline(xintercept = mean(zy))
p3 <- pdata %>%
ggplot(aes(x=value,
fill=key))+
geom_density(alpha=.4)+
geom_vline(xintercept = mean(px))+
geom_vline(xintercept = mean(py))
ggpubr::ggarrange(p1,p2,p3,ncol = 1)
Whereas the first plot shows the actual mean differences (the black line running down the middle), the other two have nonsensical mean differences now:
