Why does power reduce when increasing the number of post-hoc comparisons? I'm performing a Dunn's post test comparison of my data after a Kruskal-Wallis test (4 groups, treated and control each one has drug and vehicle injection).
The comparisons I've been doing are 
Treated Vehicle vs Control Vehicle (I want to see the vehicle doesn't affect the treatment)
Treated Drug    vs Control Drug (I want to see the drug affects the treatment)
Control Vehicle vs Control Drug (I want to see that both control are the same)

I was wondering if the other comparisons were also possibly significant but when I increased the number of comparisons I lost significance in the ones I had before and everything was "ns" (not significant).
I looked for the "test help" (I use GraphPad Prism) and read that if you increase the number of comparisons it will be harder to resolve differences because it divides the alpha you choose between the n number of comparisons but I don't understand why is this happening.
When I say "why is this happening", I mean I want to understand intuitively why it is necessary to divide the given alpha.
My intuition tells me that if two things were significant before they must continue to be significant. Otherwise if we suppose that the "better" comparison is everybody against each other, then we should only trust that comparison because the other comparisons have some kind of "masking effect".
 A: The problem is probably best known as the Multiple Comparisons Problem.
When you choose a significance level for a given comparison you are allowing for a certain Type I error rate.  If I perform multiple comparisons (among multiple groups as you wrote above) I am necessarily increasing the likelihood that I make a Type I error.  At the extreme end if I have two identical groups but $100$ metrics I'm testing in one sample from each group then $\alpha$ fraction of those will come back erroneously significant.  But at $100$ metrics I'm almost guaranteed to make an error every time.
One possible correction for this is to decrease $\alpha$ such that as I increase the number of comparisons I maintain the desired false positive rate.  From the previous example, if instead of an $\alpha$ significance level I modify it to an $\alpha / 100$ level with my $100$ metrics comparisons I can recover my desired false positive rate.  Intuitively if $\alpha$ gives me a $5\%$ false positive rate then $\alpha / 100$ should give me a $0.05\%$ false positive rate so when I perform my $100$ tests I will recover the overall significance level I desired.
This correction, that GraphPad is doing and outlined above, is known as the Bonferroni Correction.  It is derived by adjusting the significance level of a multiple testing scenario with the requirement that the familywise error rate (the notion of type I error) is kept at the ceiling of the single comparison error rate.
The intuition in changing $\alpha$ is bound up in the notion that the per-test and familywise error rates are different in multiple testing scenarios and what we desire is a constraint on the familywise.
