Suppose you have $n = 10$ observations in vector
x in R from a normal distribution
and wish to test $H_0: \mu = 0$ against $H_a: \mu \ne 0.$ at the 5% level.
 7.04 1.94 -2.42 3.85 3.58 -5.70 -4.86 -3.14 4.50 4.04
Min. 1st Qu. Median Mean 3rd Qu. Max.
-5.700 -2.960 2.760 0.883 3.993 7.040
The sample mean is a little above $0;$ the stripchart shows six observations
above $0$ and four below. But in view of the relatively large spread of the
of the data, there seems to be inadequate evidence that $\mu \ne 0.$
A t test in R, gives P-value 0.5496, which is not below 5%. That is pretty
much the end of the story. Your ten observations don't contain information
for you to believe $\mu$ differs from $0.$
If you're P-hacking, you might try a Wilcoxon
signed rank test, a median test, or some kind of permutation test. But the truth is that your ten observations
are not enough to detect a population mean $\mu$ different from $0,$ a t test was
an appropriate one, and you should accept its verdict.
One Sample t-test
t = 0.62168, df = 9, p-value = 0.5496
alternative hypothesis: true mean is not equal to 0
95 percent confidence interval:
mean of x
The 95% confidence interval $(-2.33, 4.10)$ includes $0.$
A confidence interval might be considered an interval of potential
values of $\mu$ that could not be rejected in view of the data
at hand. So it will do nothing to change the verdict that $H_0$
cannot be rejected.
If you have reason--beyond the data at hand--to believe that $\mu \ne 0,$
and sufficient budget to continue investigation, then a reasonable "additional step"
might be to perform an experiment with more observations.
If you have a specific
amount $\Delta$ in mind by which you believe $\mu$ differs from $0,$ you might
use a power and sample size computation to find a sample size $n$ that would
give a good chance of rejecting $H_0.$ if such a difference is really present. Part of that computation
requires you to guess at the population standard deviation $\sigma.$ The sample
standard deviation $S = 4.49$ of your current data is an estimate of $\sigma.$
However, if $\mu = 0$ is really true, then more investigation is not going to be useful.