You may be looking for a form of power calculation (perhaps based on raw effects rather than standardized ones).
[This shows a typical situation in calculating power. The top plot is the circumstance under the null, from which you have a rejection rule; the other plot is the situation under the alternative, using the same rejection rule -- you calculate the probability of rejection for a specific situation under the alternative.]
Indeed it sounds like you're describing a post-hoc power analysis, but as you say, this will be a function of the p-value. In addition some aspects of post hoc power analysis can be counterintuitive. They're somewhat controversial and should be used with caution.
However, an a priori calculation of power can be highly informative at any point -- not just in choosing sample size (which is what it's often used for), but in understanding, for example, how to interpret a rejection (if you have very low power but reject, you might wonder whether there's really anything going on there at all)
If you can specify the size of the difference you're looking to pick up you can compute the probability of rejecting the null at a given sample size.
Indeed you can calculate power curves which show the probability of rejection at every size of difference from the null (for a given sample size), or the probability of rejection at each sample size for a given effect.
(This power curve is generated by calling a function in R that gives power for the ordinary two-sample t-test]
Many questions on site deal with calculation of power.
For a variety of more complicated tests explicit expressions for small sample power may not be easy. In large samples you can often get asymptotic expressions that will work well. In the Welch t-test if you specify the ratio of variances, the two sample sizes and the how many of one of the standard deviations the two means are apart, you can calculate an asymptotic expression for the power.
In cases where algebraic calculation is difficult, simulation can give rejection rates at any specific alternative of interest, or across a sequence of alternatives. There's an example in R here (under Power of the t-test).
Here's an example power curve from a bit of research I was working on recently:
The points are simulated power values. This is for a widely used test.The situation being considered would be fairly representative in the particular application under consideration. The test has a problem in this particular circumstance, because even though it's at a big sample size, the test has low power -- particularly on the left side. (It also doesn't get very close to the required significance level and is biased -- the rejection rate is lower than $\alpha$ for a range of values where the null is false. This sort of thing would not be a substantial issue for your circumstances.)
[In this case, the fact that the power isn't quite high when the parameter (delta) is greater that 0.1 or less than -0.1 would be alarming; it will likely be a shock to the people who use the test in practice, because a difference of that size would be financially very important. The fact that they have little hope of detecting that large a difference with a widely-used test matters.]
