I'm creating a linear model for a data set with a fairly small amount of observations (roughly 40). I've found that one observation has a significantly larger value than the others for the response variable in my model. The data point is not an mistake, it is a correct value but it just happens to be an extreme value. When I include the value, it heavily influences the inference, in a way which supports the linear relationship. However, when I take it away the linear relationship in my model is much weaker, to the point where most of the inference from the dataset including the extreme point is invalid.
Is there a good way of dealing with this? I don't want to fully exclude it as it's still a valid observation, but this single observation seems to have a very substantial effect on the model.