One of us is confused. If I understand correctly, your model (ignoring residuals) is
$Y = \alpha + \beta_1 \cdot M + \beta_2 \cdot X + \beta_3 \cdot M \cdot X$
The intercept is $\alpha$. It does not visually represent the interaction; the slope (the coefficient $\beta_3$) on $M \cdot X$ is the interaction. If $\beta_3$ is not significantly different from zero, it's sound to omit the interaction term from the regression, and to re-fit the model without it, which means that now your model looks like
$Y = \alpha + \beta_1 \cdot M + \beta_2 \cdot X$
If the remaining coefficients are all significant, the visual representation would have two lines.
One line for the observations where the factor is present (e.g., male):
$Y = \alpha + \beta_1 \cdot 1 + \beta_2 \cdot X = (\alpha + \beta_1) + \beta_2 \cdot X $
And one line for the observations where the factor is not present (e.g., female):
$Y = \alpha + \beta_2 \cdot X$
So that's why it doesn't make sense to me to ask about omitting the intercept $\alpha$ based on the value of $\beta_3$.