Here are fictitious data, sampled using R, somewhat similar to your data.
set.seed(1234)
x1 = c(rnorm(50,100, 2), rnorm(140, 125, 5), rnorm(40,155,3))
x2 = c(rnorm(50,105, 2), rnorm(150, 125, 5), rnorm(40,156,2))
summary(x1); length(x1); sd(x1)
Min. 1st Qu. Median Mean 3rd Qu. Max.
95.31 118.30 124.70 125.01 131.94 163.76
[1] 230
[1] 18.12978
summary(x2); length(x2); sd(x2)
Min. 1st Qu. Median Mean 3rd Qu. Max.
98.53 117.40 124.34 125.83 130.64 160.12
[1] 240
[1] 16.01616
par(mfrow=c(1,2))
hist(x1, br=5, col="skyblue2")
hist(x2, br=5, col="skyblue2")
par(mfrow=c(1,1))

x = c(x1,x2)
g = c(rep(1,230),rep(2,240))
boxplot(x~g, notch=T, col="skyblue2", horizontal=T)
Notches in the sides of boxplots are nonparametric CIs roughly
calibrated so that overlapping CIs suggest no significant difference
in location.

Shapes of the samples are similar, so a nonparametric 2-sample
Wilcoxon rank sum test should test for a significant difference
in location. The P-value $0.75 > 0.05 = 5\%$ indicates no significant
difference.
wilcox.test(x~g)
Wilcoxon rank sum test
with continuity correction
data: x by g
W = 27134, p-value = 0.7518
alternative hypothesis:
true location shift is not equal to 0
If these were the actual data, I would stop here. Of course,
your data may show different results.
For the reasons you mention, I would not trust a 2-sample t test
to give reliable results, but for such large sample sizes
its P-value may be roughly correct. Again, no indication of
a significant difference.
t.test(x~g, var.eq=T) # pooled t test
Two Sample t-test
data: x by g
t = -0.51847, df = 468, p-value = 0.6044
alternative hypothesis:
true difference in means is not equal to 0
95 percent confidence interval:
-3.914830 2.280273
sample estimates:
mean in group 1 mean in group 2
125.0149 125.8322
Even though I would not trust a pooled t statistic to have
Student's t distribution in these circumstances, the t statistic
does seem a reasonable way to measure a difference between means
of the two samples.
A permutation test using the pooled t statistic as metric,
does not find a significant difference, its P-value is
about the same as for the pooled t test. No significant difference.
set.seed(2021)
t.obs = t.test(x~g, var.eq=T)$stat
t.prm = replicate(10^5, t.test(x~sample(g), var.eq=T)$stat)
mean(abs(t.prm) > abs(t.obs))
[1] 0.6094 # P-value of simulated permutation test
hdr = "Simulated Permutation Dist'n"
hist(t.prm, prob=T, col="skyblue2", main=hdr)
abline(v = c(t.obs,-t.obs), col="red", lwd=2)

wilcox.test
in R, so we do not have to address issues like this. $\endgroup$