1
$\begingroup$

Context: There are two groups of scalar observations, A and B. The values in both groups increase (in a different manner) as a function of the value S. This scenario is shown below in a scatter plot of A and B values, sorted according to S. I wish to estimate the value of S at which it begins to influence $A$.

A scatter plot of A and B values, sorted according to S

Tried so far: I estimated the required value as the smallest $k$ such that for any $i \geq k$, the groups:

$\{ a \in A | i \leq S(a) \leq i+d \}$ and,

$\{ b \in B | i \leq S(b) \leq i+d \}$

where $d$ is set to $30$, are significantly different in the statistical sense. To measure the difference between the two groups I applied the two-sample t-test.

Problem: This approach assumes A and B "behave" similarly for low values of $S$, and at some level of $S$ show different behavior. However, this is not always true. For example, in the following scenario, the opposite occurs: A and B are different from the start and at some point seem similar.

enter image description here

what would be an appropriate approach to estimate the required value of S?

Thank you for your time.

$\endgroup$
4
  • 1
    $\begingroup$ What do you mean by It is required to estimate the value of S for which the observations in A begin to be affected by S? Affected in what way? Are you trying to figure out in which sections A and B are significantly different, based on S? $\endgroup$ Jul 17, 2019 at 9:54
  • $\begingroup$ Thanks for your comment. Yes, exactly, I wish to use S to figure out when A and B are significantly different. $\endgroup$
    – edelburg
    Jul 23, 2019 at 9:02
  • $\begingroup$ I don't know how you would do that with a continuous variable. An alternative would be to split up the S variable into a number of bins (categorize), and then test for each bin (category). $\endgroup$ Jul 24, 2019 at 12:01
  • $\begingroup$ I intend to do so based on the sampled data. The smooth graphs are only there for visualization. I am trying to find an appropriate way to determine at which point S seems to begin influence the number of A observations, given all the data I described. $\endgroup$
    – edelburg
    Jul 27, 2019 at 20:28

1 Answer 1

1
$\begingroup$

It looks like the values for both Group A and Group B increase more or less linearly with $S$ (whose values I take to be plotted along your horizontal axes). The Groups differ in both intercepts (values when $S = 0$) and slopes, the rates of change as $S$ changes.

So consider approaching this problem from a different angle: use linear regression to find values of $S$ above or below which the estimated mean values for Groups A and B are statistically distinguishable.

A linear multiple regression model that takes into account different intercepts and slopes for Groups A and B would be:

$$ y = \beta_0 + \beta_1 G + \beta_2 S + \beta_3 G S + \epsilon $$

where $y$ is your amount, $G$ takes the value 0 for Group A and 1 for Group B, and $\epsilon$ represents the error around the mean multiple regression. Then $\beta_0$ is the intercept (at $S=0$) for Group A, $\beta_1$ is the difference of the intercept for Group B from $\beta_0$, $\beta_2$ is the slope with respect to $S$ for Group A, and $\beta_3$ is the difference of the slope for Group B from Group A.

Standard statistical software can estimate all 4 of the $\beta$ coefficients. Then the difference between the mean $y$ values predicted for Groups B and A at any value of $S$ will be:

$$ \hat y_b(S) - \hat y_a(S) = \beta_1 + \beta_3 S .$$

The variance of this difference, applying the rules for variance of a weighted sum is:

$$\operatorname{Var}(\beta_1) + S^2\operatorname{Var}(\beta_3) + 2S\operatorname{Cov}(\beta_1,\beta_3) .$$

Standard statistical software provides variance estimates for each of the coefficients and for the covariances between them. The individual variances for $\beta_1$ and $\beta_3$ are the squares of their reported standard errors. You might have to dig into the software manual pages to see how to extract the covariance; look for the coefficient covariance matrix, sometimes called the variance-covariance matrix.

The square root of this variance is the standard error of the difference between the mean predicted values for Groups A and B at any value of $S$. Assuming a normal distribution for the coefficient estimates, you could take a significant difference (at $p$ < 0.05) to be anything greater than 1.96 standard errors. Solve for values of $S$ beyond which

$$|\beta_1 + \beta_3 S| > 1.96 \sqrt{\operatorname{Var}(\beta_1) + S^2\operatorname{Var}(\beta_3) + 2S\operatorname{Cov}(\beta_1,\beta_3)}. $$

There will be 2 regions of $S$ for which this will hold, one below some low value of $S$ and one above some high value of $S$ (with "low" and "high" meaning below or above the value of $S$ at which the 2 groups have equal mean values of $y$). Note that in your top plot the two curves linearly extrapolated would diverge significantly again (in the opposite direction) at some value of $S$ below 0, and in your bottom plot they would similarly diverge at some value of $S$ above 1000. If such extreme values of $S$ aren't relevant to your study those second regions can be ignored.

This approach has the significant advantage of using information from all of your data points, unlike your present sampling sets of 30 values starting at various values of $S$.

Note that the above is based on the variance of the difference in the mean responses predicted by linear multiple regression for the two Groups at a value of $S$. If you want to work instead with differences in individual predictions including the error term $\epsilon$ then you need to proceed as outlined on this Wikipedia page describing mean and predicted responses. If linear regression is not appropriate for your data then a similar approach could be used with non-linear modeling.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.