5
$\begingroup$

So I know that t-tests and ANOVAs are used to determine if the means of 2 normally distributed random variables are significantly different, and the p-values give a statistical confidence level of that result.

However, for the problem I'm working on, I am wondering if there is some extension of this idea to determine if the mean of a pair of random variables is "significantly" located inside of some region on the cartesian plane.

EDIT: Thanks to a helpful comment, I now know that this is called a "composite hypothesis". To get help in setting this up, I want to provide details about my specific problem.

Consider the cartesian plane x-y, where the domain of x stretches from [0, 45] and the domain of y stretches from [-30,30]. Let's then also say we have random variables $rx$ & $ry$, both of which are normally distributed random variables with a mean and std of 20 and 5.5 for $rx$ and -2.5 and 2 for $ry$ respectively. $rx$ & $ry$ are paired samples, and thus when plotted in the plane x-y, form a cloud of points.

Furthermore, consider a region in the x-y plane that is formed by the lines defined by equations 1 & 2 respectively: $y = 0$ and $y = -1.85*(x_2 - 15)$, where $x_2$ stretches from [15,45]. Separately, consider the line defined in equation 3: $y = 0.5 * x$, where here $x$ encompasses the entire domain being considered: [0, 45].

I have 2 goals that I hope composite hypothesis testing will solve:

1) I want to determine, with a reasonable level of statistical confidence, that the center of the cloud of points formed by the pair $(rx, ry)$ are "significantly" inside the region formed by equations 1 and 2.

2) I similarly want to determine that the center of the cloud of points formed by the pair $(rx, ry)$ is "significantly" far away from the locus of points formed by equation 3.

Edit 2: I am adding a figure that might help with visualization, as well as the code I wrote to generate it:

clear all;
close all;
home;

%define the domain we will display the data over
x_domain = [0:1:45];
y_domain = [-30:1:30];

%define number of samples for rx and ry
num_samples = 26;

%define variables rx and ry
rx = 20 + 5.5 .* randn(num_samples,1);
ry = -2.5 + 2 .* randn(num_samples,1);

%deine equations 1-3
eq1 = x_domain * 0;
eq2 = -1.85 .* [(15:x_domain(end)) - 15];
eq3 = x_domain ./ 2;

%start plotting
figure; hold on;
Lw1 = 1.5;
Lw2 = 2.5;
marker_size = 8;

%plot the pair rx, ry
plot(rx, ry, '.', 'markersize' ,marker_size,'color','k', 'displayname', 'points of rx & ry');

%plot the grand mean of the data with a star
plot(mean(rx), mean(ry),'p','markersize',12,'linewidth',Lw1,'markerfacecolor', 'r', 'markeredgecolor', 'k', ...
    'displayname', 'grand mean');

legend('show');

%display the lines that form the region (eq1 and 2)
plot(x_domain, eq1, 'color', 'k', 'linewidth', Lw2, 'displayname', 'equation 1');
plot([15:x_domain(end)], eq2, 'color', 0.5 * [1,1,1], 'linewidth', Lw2, 'displayname', 'equation 2');

%plot the line corresponding to eq3
plot(x_domain,eq3,'--','color','k', 'linewidth', Lw2, 'displayname', 'equation 3');

h1 = gca;
h1.YTick = [-30,-15,0,15,30];
h1.YTickLabel = {'-30','-15','0', '15', '30'};
h1.XTick = [0, 15, 30, 45];
h1.XTickLabel = {'0', '15', '30','45'};

xlabel('X');
ylabel('Y');
set(gca,'Fontsize',12);

xlim(x_domain(end)*[0, 1]); 
ylim(y_domain(end)*[-1,1]); 

axis square;

Result: data and equations visualized

$\endgroup$
  • 2
    $\begingroup$ Yes; this is called a "composite hypothesis." But how you go about testing it depends on the hypothesis and the assumptions. Thus, if you have a particular question to ask, it would be best to describe your actual situation rather than offering just an example. $\endgroup$ – whuber Jun 1 '20 at 21:10
  • $\begingroup$ OK thanks whuber, I'm going to go ahead and edit the question for a bite more detail $\endgroup$ – John Alperto Jun 1 '20 at 21:14
2
+50
$\begingroup$

Frequentist method

You can do this with some 2-dimensional equivalent of confidence intervals based on a t-test / t-distribution.

Constructing a 1D confidence interval based on a t-test

With a t-test you draw confidence intervals based on the ratio of the mean and the standard deviation. The confidence interval will contain those points/means for which a t-test will pass.

Say in the below image the unknown mean of the distribution is $X_{n+1}$ and we sample a point out of that distribution (in the image highlighted in red). The we can make a lower and upper interval boundaries based on the t-distribution (the dotted red lines highlight where those boundaries will be). These boundaries/lines are such constructed that for $\alpha\%$ (95% in the image) of the observations you will get a correctly constructed boundary (correct in the sense that the true population mean is inside the boundary)

geometric example

In 2D

This cone of $\alpha\%$ observations can also be made in 2D.

2D cone

So based on that you can make a 2D interval and decide whether the interval is inside your region of interest or not.

example

Note1: the confidence region constructed in this way is not optimal and only guarantees a limit of the fail rate (in the 1D case it is exact). It might be that you are too conservative. This is because of two problems.

  • The shape of the border of the confidence region is not unique, you can use a square but any other shape can be used as well. Without any additional information there is no option that is better then any other. (a similar case is here where a confidence region/interval based on two values can be made in multiple ways)

    Edit: For a moment I thought I had made a mistake here because this would allow to solve the Behrens-Fisher problem. But, I believe that I did not make a mistake after all (it doesn't solve the problem). The shape of the border for the confidence region... if you have it fixed and not dependent on the location of the mean $\bar{x},\bar{y}$ then it is a correct confidence region. You can not make it adapt according to the location (in order to compare with the line $y=x$ which would relate to the question whether the means are equal) in order to solve the Behrens-Fisher problem. So your problem is related to that Behrens-Fisher problem.

  • Because you are comparing a region, a composite null hypothesis, you will be selecting the worst case points in the region. This is the same in the 1D situation, where you only get a unique dominant/optimal test if you are dealing with simple hypotheses, and for composite null hypothesis it is neither exact.

    You condition on the case that is the worst point from the region that defines the null hypothesis. For that worst point you will get $100-\alpha \%$ of the time that the $\alpha \%$ confidence interval fails. If another weaker point in the region of the null hypothesis is the 'true value', then this failure rate will be less.

Note2: The equation 3 is a weaker condition than the equations 1 and 2. If the mean is significantly in the region 1 and 2 then it will also be significantly far away from equation 3.

Bayesian method

In the frequentist method you determine a confidence region which conditions on the true mean. For a given true population mean you will not be wrong in $\alpha\%$ of the time.

With a Bayesian interval, e.g. a credible interval you will condition on the observation. For a given observation you will not be wrong in $\alpha \%$ of the time. (and also conditioned on your prior distribution being correct, which may sometimes be a contradiction if a prior is referring to believe about parameters instead of some distribution of population parameters)

The mathematics for the Bayesian case will be much simpler. Say if we use an uninformed prior $\pi(\mu,\sigma) \sim \frac{1}{\sigma}$ then the posterior marginal distribution for the mean will be a generalized student-t distribution.

$$ f(\mu_x) \sim \bar{x} + \frac{\hat\sigma_x}{\sqrt{n}} t_{\nu = n-1}$$

and

$$ f(\mu_y) \sim \bar{x} + \frac{\hat\sigma_x}{\sqrt{n}} t_{\nu = n-1}$$

So actually, the result is much the same as the frequentist computation (also a t-distribution). However, now we can do the "test" by computing the (posterior) probability that the mean is outside the region. To do this you can compute the double integral for the t-distribution or you computationally simulate (draw from) the distribution.

The interpretation of the result is now a bit more straightforward and does not have the two problems from the ambiguity/non-uniqueness with the confidence interval. With the frequentist method you condition on the true mean (which is ambiguous since it is a region of true means) and with the Bayesian method you condition on the observation (of which there is only one).

$\endgroup$
  • 1
    $\begingroup$ I believe I must be making a small mistake in my idea of allowing other shapes than a square 2D-cone/pyramid since it would solve the Behrens-Fisher problem if it was that simple. $\endgroup$ – Sextus Empiricus Jun 12 '20 at 13:31
  • $\begingroup$ Thanks for your answer. I think both approaches are compelling, especially the Bayesian one, but to be honest I don't think I quite have the background to put them into practice. I'm experimenting right now and running into problems, but will probably update my question once I have something. I may need to post another question in the worst case scenario. $\endgroup$ – John Alperto Jun 19 '20 at 18:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.