P-value under 5% but power under 80% Im trying to understand how to accurately report results on AB tests
If my type 1 error is 5% and type 2 is 20% , can we have instances where the p-value is < 0.05 and power is < 0.8?
If so, how ? What does it mean ?
Is something statistical significant when p-value is under 0.05 or power over 80% (assuming alpha and beta are defined to be 5% and 20%)?
EDIT: Im performing my power analysis during an experiment. Im trying to give updates on how a change in a button colour is performing. And in that update I want to specify the power it has.
 A: Power depends on the $\alpha$-level, which you set, not on the p-value (which you calculate).
If you do a power calculation and determine that you have $80\%$ power to detect the difference of interest at the $\alpha$-level you've selected (often, but not necessarily, $\alpha = 0.05$), then that's the power. You don't even need to calculate a p-value for that to be the power. In fact, it might be common in certain fields not to have any data when the power calculation is performed, but to perform it in order to get funding for an experiment that will collect the data and result in subsequent calculations (such as p-values).
A: Possible situations abound. Here a few:
Suppose you are testing $H_0: \mu = 50,$ against $H_a: \mu \ne 50$ at significance level $\alpha = 0.01 = 1\%,$ using $n = 10$ observations from a normal population
with $\sigma \approx 4.$
Then a power and sample size procedure (here from a recent release of Minitab)
gives power values about 70%, 75%, 80% for alternative values
$\mu_a = 54.82, 55.06,$ and $55.33,$ respectively.
Power and Sample Size 

1-Sample t Test

Testing mean = null (versus ≠ null)
Calculating power for mean = null + difference
α = 0.01  Assumed standard deviation = 4

Sample
  Size  Power  Difference
    10   0.70     4.81897
    10   0.75     5.05952
    10   0.80     5.32813

Corresponding points on the relevant power curve are shown below:


Note: A trivial example is to set the Rejection region
to be so far out into the tails of the null distribution (say, $|T|\ge 30)$ that $\alpha \approx 0.$ Then the power
(probability of rejection for the specified alternative)
could also be very small--in particular less than 80%.
A: If you observe a p-value equal to the alpha level, then you have roughly $50\%$ power for a true effect that equals the observed effect. Observations that are closer to the null hypothesis will not be significant and observations further from the null hypothesis will be significant. You have roughly 50:50 probability that these will happen (often the distribution is more or less symmetrical).
Here is an example for a normal distribution. From this question Report power if result is statistically significant
The alpha level is here $5\%$. We observe a p-value below $0.05$ and the power is below $0.8$.

