I'm looking at data from experiments that have been run to look for a difference between proportions, 1-tailed.
In many of these it has been found that there is a significant difference between the two proportions (e.g., effect size of 3%, p<0.05)
However, I have run a post-hoc power analysis, inputting the effect size (as it was observed), desired power (0.8), significance (0.05), and it suggests that a sample size notably larger (for example 3x larger) than that which was used is in fact needed to run the test at this power / significance level.
The problem I have is interpreting this. If the power analysis had been done before and we had just happened to input the actual resulting effect size - the required sample size would have shown as this larger N, so by finding significance at a lower N, how should I interpret the result? How much confidence (if any) is lost?
As an aside: I know it is advised against to do post-hoc power analysis with the observed effect size, but I'm just trying to validate the quality of the test and how confident I can be in the significance that was found... as described above, hypothetically the exact same power analysis could've been done identically beforehand and would have suggested this higher N that was ultimately not reached.
I just don't know what implications this has for the effect size and p<0.05 that's then found in the data when the sample size < required N.