I performed a linear regression and I got a p-value < 0.05 but my R^2 is low (< 0.04). How can I interpret the results? The model is significant but there's not a good fit, what can I do to solve this? (without affecting too much of my p-value) Thank you
The p value is an estimate of (deep breath) the probability that the relationship you see would occur in a randomly drawn sample of size n from the population if the relationship you see did not actually exist in the population. If your p value happens to equal or be under the arbitrarily chosen threshold (usually 0.05), then it is called "significant", which means that, for the p = 0.05 threshold, you accept a 1 in 20 chance that the result you see actually is just a random burp in the population and not reflective of any real relationship in the population. So, "significant" might mean "likely to exist".
R^2, on the other hand (coefficient of determination) is an estimate of the strength of that relationship. That is, to what extent does the model "explain" the response? The conventional interpretation of the R^2 is that it represents the fraction of the "activity" in the response explained by the model. Thus, an R^2 of 0.2 means that 20% of the "activity" in the of the response is "captured" by the model. The other 80% is due to "other stuff". Note that I say "activity". This could correspond to "upward trend", "downward trend", "trend that matches a sinusoidal curve", "trend that matches a parabola", "trend that matches the different category distributions", or whatever the underlying model assumption is.
Thus, a low p/high R^2 model would mean that it is unlikely (4% chance) that the results you got do not reflect the population at large, but that "other stuff" not in your model "explains" the outcome far better than your current model does. If you have the funding, it's a great excuse to do more experiments that take into account other possible predictor variables for a multiple regression.