I designed an experiment,in which there is one two-level categorical independent variable, and a continuous variable as a covariate. The interaction between the two is not significant, however, when I look at the correlation between covariate and dependent variable at each level of the categorical independent variable. The correlation is significant at one level, but not the other. This pattern shows up repeatedly when I use multiple experimental stimuli, and is consistent with my theory. So can I still interpret the result, even if the interaction effect itself is not significant?

To make it clearer, I know that a significant categorical by continuous interaction means that the slope of the continuous variable is different for one or more levels of the categorical variable.However, I am wondering whether the test for interaction effect takes the significance of the slope into account or not or it just test whether there's a significant difference between the two slope coefficients.