If your control variable was measured prior to the predictor of interest, what you have observed usually occurs when the control variable has an effect in the same direction on both the predictor and the outcome. E.g., if prior academic achievement positively affects both admission into a scholarship program and future achievement, then the effect of the scholarship program on future achievement will be smaller conditional on prior achievement. In addition, if the control variable is highly associated with the predictor and less associated with the outcome, your increase in p-value may be due to colinearity or conditioning on a near-instrument, both of which don't change the effect of the predictor but increase it's variance.
If you control variable was measured after the predictor of interest, you may be blocking the effect of the predictor on the outcome. In the same scenario, imagine you instead control for an intermediate outcome, like depressed feelings. Part of the effect of the scholarship on future achievement may occur through the effect of the scholarship on lowering depressed feelings, which improve achievement. Controlling for depressed feelings will block part of this effect, decreasing the predictor's effect.