1
$\begingroup$

I need to test if y1 (in blue) and y2 (in green) are significantly different from an estimated null NULL.y (orange) in this simple example here?

If the intercept is different, one way I know to test against the null is mod1 <- lm(y1 ~ x + 1, data = df) if say you are testing if it's significantly different from 1.

Here, I am testing if values above the line NULL.y is significantly different. My guess is that y2 will be but perhaps not y1.

Question: How do I compare the data with the null?

set.seed(111)
library(truncnorm)
x <- rtruncnorm(n = 288, a = 0, b = 10, mean = 5, sd = 2)
y1 <- -0.02 * x + 1
y2 <- -7 * x + 28
NULL.y <- -7.82 * x + 25
df <- data.frame(x, y1, y2, NULL.y)

mod1 <- lm(y1 ~ x, data =  df)
summary(mod1)

mod1 <- lm(y2 ~ x, data =  df)
summary(mod1)

df <- gather(df, key ="y.key", value = "y.vals", y1, y2, NULL.y)
df$y.key <- as.factor(df$y.key)

ggplot(aes(x=x, y=y.vals, color = y.key), data = df) + 
  geom_point() + 
  geom_smooth(method="lm", formula = y ~ x)

enter image description here

EDIT: For those interested in what the actual plots are--

y1 and y2 are the index of mean crowding of the patch (J) = (Variance/(mean)^2) - (1/mean)

where variance is the variance in the abundance of #trees in a patch and mean is the mean abundance of the trees.

The x-axis x is the mean number of trees.

Since Y is calculated from X, I calculated the null by shuffling X from the original sample, calculate Y, the slope between them. Repeat 1000 times. Find the average slope. Use the equation of the slope to calculate NULL.y from the original x value.

The NULL.y turns out to be a negative correlation. Now, I want to test if the slope foryi are different from the null.

In my study I actually have two nulls one corresponding to y1 and another to y2. For simplicity here, I just represented it as one null.

$\endgroup$
2
  • $\begingroup$ What are $y_1$ and $y_2$? What is the $y_{null}?$ $\endgroup$
    – Dave
    Jun 18, 2022 at 17:25
  • $\begingroup$ @Dave have edited the answer for clarity on what is yi and ynull. TLDR; Yi is a function of X, hence related. I shuffled X and recalculated Y to estimate the null. $\endgroup$
    – slicer
    Jun 18, 2022 at 21:48

1 Answer 1

1
$\begingroup$

Since Y is calculated from X, I calculated the null by shuffling X from the original sample, calculate Y, the slope between them. Repeat 1000 times. Find the average slope. Use the equation of the slope to calculate NULL.y from the original x value.

The NULL.y turns out to be a negative correlation. Now, I want to test if the slope foryi are different from the null.

So you attempted to run a permutation test. What you did incorrectly however was averaging the slopes. Instead, to run a permutation test you would shuffle the $X$ values, calculate the slope for the $(y, X_\text{shuffled})$ and calculate the test statistic between the slope calculated on the raw data vs shuffled data, for example $\mathbf{1}\big(\hat{\beta}_0^\text{raw} > \hat{\beta}_0^\text{shuffled}\big)$ (this will differ, depending on your hypothesis), and repeat it many times. The fraction of the cases where the condition is met would be your $p$-value.

$\endgroup$
5
  • $\begingroup$ Thanks! I attempted some such analysis but with correlation and not slope. I guess I am not 100% certain how to compare the raw with the shuffled. I was wondering if you have any ideas?stats.stackexchange.com/questions/579545/… $\endgroup$
    – slicer
    Jun 21, 2022 at 21:22
  • $\begingroup$ @Biotechgeek the link doesn’t work. The comparison can be either > or <, or abs() > where you basically count how often it is true. $\endgroup$
    – Tim
    Jun 21, 2022 at 21:58
  • $\begingroup$ stats.stackexchange.com/questions/579545/… $\endgroup$
    – slicer
    Jun 21, 2022 at 22:16
  • $\begingroup$ I have linked it in the comments. $\endgroup$
    – slicer
    Jun 21, 2022 at 22:17
  • $\begingroup$ @Biotechgeek it seems like you've already got an answer. I agree with the answer: its not clear what are you trying to do with the code, but it is not a $p$-value. $\endgroup$
    – Tim
    Jun 22, 2022 at 3:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.