Testing Hypothesis with different alternatives I want to test whether $\mu=\mu_0$ where $\mu_0$ is some fixed number.
Consider the following two different testings.
Hypothesis Testing 1: $H_0:\mu=\mu_0,H_1:\mu<\mu_0$
Hypothesis Testing 2: $H_0:\mu=\mu_0,H_1:\mu>\mu_0$
It is possible that testing 1 fails to reject $H_0$ and testing 2 rejects $H_0$. In this situation, I can't compare two testings as they are pear and apple.
Let's suppose you are not allowed to use $H_1:\mu\neq\mu_0$ here.
If I am in this situation, what should I do? Should I declare I can't reject $H_0$? Should I declare $H_0$ false by some probability? Which test should I trust?
 A: Imagine testing $\mu=0$. You do your calculations and find that $\bar{x}=99$ and your z-statistic (or t-stat) is 123.
I would have serious doubts about hypothesis 1 and very much believe hypothesis 2.
A: Doing it right the first time is best.
First, in practice, this should be an unlikely situation.

*

*Maybe you have re-engineered a pharmaceutical process hoping
that the new process has a higher yield than the current one
with $\mu_0 = 100,$ so you'd take data from runs of the new process,
average them and test $H_0: \mu= 100$ vs. $H_a: \mu > 100.$

*Maybe your town has changed the widths of lanes and the
sequencing of traffic lights on its main road hoping that the
average late afternoon travel average travel time on the main
stretch is reduced from the former $\mu_0 = 20$ min. Then
you'd get travel times under the new configuration to test
$H_0: \mu = 20$ vs $H_a: \mu < 20.$

*Maybe your old supplier whose product had 200mg of active
ingredient per bottle has gone out of business and you are
checking to see the amount of active ingredient in a former competitive
supplier is the same as for the old supplier. Then you'd
test $H_0: \mu = 200$ vs $H_a: \mu \ne 200,$ based on the average
of $n$ randomly selected bottles from the prospective new supplier.

So ordinarily you would test one of the three kinds of tests
and act according to the results of the test. One hopes you would have done
a 'power and sample size' computation beforehand so you'd take
a large enough sample $n$ in order to have a good chance (say 90%)
of rejecting if there is a meaningful difference from $\mu_0.$
Then you would likely take the result of the one test as sufficiently
good evidence to act upon.
But best-laid plans don't always work out. However, as a direct answer to your question, let's suppose you
have taken data to test $H_0: \mu = 100$ vs. $H_1: \mu < 100$
at the 5% level, and cannot reject.  Here are data simulated in R
that would give such a result.
set.seed(806)
x = rnorm(10, 98, 15)
t.test(x, mu=100, alt="less")

        One Sample t-test

data:  x
t = -0.69053, df = 9, p-value = 0.2536
alternative hypothesis: true mean is less than 100
95 percent confidence interval:
     -Inf 104.6308
sample estimates:
mean of x 
 97.20135 

The mean of my $n=10$ observations is $\bar X = 97.2,$ which is
below the hypothetical mean $\mu = 100,$ but not enough smaller
to be considered statistically significant. Maybe we put the wrong
assumptions into our power computation so we didn't use a large
enough $n.$  In this case, there's no use testing $H_0: \mu = 100$ vs. $H_1: \mu > 100$ because $\bar X < 100$ could never lead to rejection.
But what do we do if we guessed completely wrong and got data such as those
in the simulation below?
set.seed(806)
x = rnorm(10, 110, 15)
t.test(x, mu=100, alt="less")\

    One Sample t-test

data:  x
t = 2.2703, df = 9, p-value = 0.9753
alternative hypothesis: true mean is less than 100
95 percent confidence interval:
     -Inf 116.6308
sample estimates:
mean of x 
 109.2014 

Of course, we can't reject in favor of $H_a: \mu < 100$ based
on a sample mean $\bar X = 109.2.$ Then we might be tempted to
try testing $H_0: \mu = 100$ vs. $H_1: \mu > 100.$ [In R, the notation
p.val gives just the P-value of the test, not the full printout.]
t.test(x, mu=100, alt="gr")$p.val
[1] 0.02466914

So we could have rejected a test of $H_0: \mu = 100$ vs. $H_1: \mu > 100$
at the 5% level because the P-value $0.025 < 0.05 = 5\%.$ Doing multiple
tests on the same data is always dangerous. If we try enough different
things, we might accidentally get a rejection on one of our tries--just by chance. (The result would be a 'false discovery'.)
Rejecting at the 2% level isn't a really strong result, but if is really important to resolve the true value of $\mu,$ then we might consider getting fresh data and doing the right test
the second time. Or maybe making a two-sided 95% confidence interval
to get a good guess at the actual value of $\mu$ in order to plan our
course of action.
A: The research question is king. The job of the hypothesis test is to answer the research question. The job of the data (and statistics) is to help you perform the hypothesis test.
I think you are somewhat confused about how to set up your hypotheses. Your hypotheses should be formed from your research question and before you even look at your data! In particular, let's consider your statement
"Let's suppose you are not allowed to use $H_1: \mu \neq \mu_0$ here."
There is no such circumstance! You are always allowed to specify any valid alternative hypothesis you like, and this $H_1$ is a valid alternative hypothesis except in the pathological case where $\mu$ can only possibly take the single value $\mu_0$.
If you are struggling to work out which alternative hypothesis to use, it is easiest to write out what you are trying to test in words and then try to put it into algebra. Dave gives some examples of this in his answer, but the question will fall into one of three categories:
(A) You want to see if the mean is above some threshold or not. If the mean is $\leq$ the threshold you don't really care whether it is equal or lower. (Maybe you are seeing if a new, expensive drug is more effective than an existing cheap drug. If it isn't more effective then you don't care whether it is equally or less effective, you won't pursue it further because it is expensive.)
(B) You want to see if the mean is below some threshold or not. If the mean is $\geq$ the threshold you don't really care whether it is equal or higher. This is just the reverse of (A).
(C) You want to see if the mean is different from some value or not. (was Don Bradman a "100-runs-per-innings" batsman?)
All of these are legitimate research questions. A and B translate to one-sided hypothesis tests, C translates to a two sided test. However, we could have formulated research questions which were asking whether the two drugs had the same efficacy as each other (this is important for regulatory reasons in some cases), or whether Don Bradman's batting skill was over 100 runs per over. Those would have led to different alternative hypothesis.
A: Yes, it is possible that test 1 fails to reject $H_0$ and testing 2 rejects $H_0$. (Consider for example a t-test with level 1% on $n = 9$ datapoints where $s^2 = 1$, $\overline{X} = \mu_0 + 3$ ).
In such a case, you have to... wonder what is the relevant alternative, and this depends on what you want to test.
Keep in mind that one only rejects the null hypothesis in favor of the alternative. And one can only have a positive conclusion in case of a reject of $H_0$. Not rejecting $H_0$ does not allow you to accpet it.
So in the case you described, you cannot conclude that $\mu < \mu_0$ (since testing 1 didn't reject $H_0$) but you can conclude that $\mu > \mu_0$ (since testing 2 did reject $H_0$). If you just want to know if $\mu = \mu_0$ then use the alternative $\mu \neq \mu_0$.
