Assume we carry out a hypothesis test at the 5% significance level. We have an observed test statistics $t$ with calculated p-value $0.03$. Does that imply that the observation has to lie in the critical region? I mean $3\%$ of the distribution is at least as extreme and the critical region is the most extreme 5% of the distribution, therefore $t$ must be contained in the critical region?
If p-value<α does the observed test statistics always belongs to critical region?
Yes, that's right.
(It doesn't depend on whether the t-test is appropriate as suggested in comments -- the appropriateness of the assumptions doesn't come into this at all; this is a question of the decision you make when presented with a p-value. The appropriateness of the assumptions would matter when interpreting the p-value and it would matter in relation to the decision-process yielding the properties you desire, but none of that is at issue.)
I mean 3% of the distribution is at least as extreme and the critical region is the most extreme 5% of the distribution
This is correct. Anything up to (and including) 5% is at least as extreme as 5%.