Using the distance of p-value from alpha I have two statistical tests which are inverse of each other, meaning that the null hypothesis are reversed. I want to use both the tests to take a decision. For this purpose, I am planning to do the following:


*

*If both the tests point to same result (by, say, rejecting the null hypothesis in test A and not rejecting the null hypothesis in test B, or vice versa), then I go ahead and take that decision

*On the other hand, if the results of the two tests are conflicting, I measure the difference of the p-values from alpha (say, 0.05) for each test, and go with the one having the largest deviation (I have some tie-breaking rules, but let's leave that here)


It sounds reasonable to me, but is there some statistical ground in interpreting p-values like this? I certainly haven't seen similar application before (in my limited exposure).
Edit: Let me clarify the question some more with the actual context and tests. I am testing for unit roots in time series, in order to determine if the series needs to be differenced in order to render it mean-stationary. The particular tests are KPSS test, with null being no unit roots, and ADF test with null being an unit root exists. Although the null hypotheses are related (and inverse of each other), but the test regression and the statistics are quite different in my opinion.
 A: I have a question based on what you asked and assuming I understood you correctly.

Why do you need two null hypotheses to decide about one decision? 

Are you doing this just because we null hypotheses can only be rejected and never accepted? 
Now the answer to your question about using p-value as a measure of "truth of the null" (similar to measuring the distance of p-value from $\alpha$) is not justified from a statistical perspective. This can lead to possible wrong decisions. Here is a great reference explaining the dangers of using p-values in the sense you are trying to use.
A: I think you are confusing the concepts of stationarity vs unit roots. Unit root implies non-stationarity but the converse need not hold. Thus, KPSS and ADF tests are not both a test for unit roots. KPSS tests for stationarity whereas ADF tests for one particular form of non-stationarity (i.e., existence of unit roots). 
I would suggest going through these notes and in particular see the figure 1  excerpted from the notes below for example of non-stationary series that are not because of unit roots:

A: I don't know anything about unit roots, but your problem may be analogous to an equivalence test where, say, a new drug is tested for inferiority, equivalence or superiority compared to another drug. That is a reasonably common issue in clinical testing and so there is quite a literature on it.
You might like to start here: http://www.graphpad.com/library/BiostatsSpecial/article_182.htm
