2
$\begingroup$

I have three data points. (110,1.95), (120, 3.03), and (130, 4.75) Y-values are observed deltas of means and X-values represent temperature settings. After plotting and running a linear regression I get an R$^2$ value 0.983 (Adjusted R$^2$ = 0.966). An F-statistic of 57.42 with a p-value of 0.08.

Would the observation that as temperature increases so do the predicted Y-value, be valid? Am I confusing myself by looking at the F-test? With the p-value > 0.05?

I have checked Whats the relationship between $R^2$ and F-Test?

and If the f-test is insignificant but coefficients are significant, can I use it?

so if your comment is to refer me to these answers please add something to help me see the connection.

In case it matters—For linear regression and OLS regression results I used Python scipy.stats.linregress(x,y) and statsmodels.formula.api

Improve this question
$\endgroup$
5
  • $\begingroup$ I would welcome suggestions on how to make a better Title in relation to the question. $\endgroup$
    – CJD
    Commented Aug 27, 2018 at 16:40
  • 1
    $\begingroup$ You have a sample size of 3, so you have very little power to detect a significant relationship. You claims that you have a significant coefficient but an insignificant F-test. However, with only one IV in the model, the p-value for the F-test for the model and the p-value for the t-test for the regression coefficient will be identical (0.08 in both cases). With only three observations, there is not much you can say about this relationship other than that for these three cases, the relationship is linear and positive. $\endgroup$
    – dbwilson
    Commented Aug 27, 2018 at 19:43
  • 1
    $\begingroup$ You certainly see that there's an increasing slope in the three sample values. But what you need to consider is whether such a thing could have arisen by chance for unrelated variables in the presence of noisy data (without some random noise you'd know the exact relationship from two values). Three random data points (ones whose deviation from no relation is just due to the noise) could either be in increasing or decreasing order quite easily (for continuous data there are 6 possible orders, one of which is purely increasing and one purely decreasing), ... ctd $\endgroup$
    – Glen_b
    Commented Aug 28, 2018 at 2:11
  • $\begingroup$ ctd... so - with that naive test at least - there's little reason to conclude that there's really anything but noise causing the appearance of a relationship. The p-value you got is simply based on a more sophisticated (and often more powerful) version of that sort of thinking. $\endgroup$
    – Glen_b
    Commented Aug 28, 2018 at 2:11
  • $\begingroup$ @Glen_b Thank you. I went back to my original data points and had to conclude that it is, in fact, insignificant as per the p-value. I understand the concept of noise and the need to filter it out. Your answer has assisted me, significantly. If you choose to post it as an answer, I will accept it. $\endgroup$
    – CJD
    Commented Aug 28, 2018 at 2:37

1 Answer 1

Reset to default
1
$\begingroup$

You certainly see that there's an increasing slope in the three sample values. But what you probably want to consider is whether such a thing could have arisen by chance for unrelated variables in the presence of noisy data.

(Without some random noise you'd know the exact relationship from just two values)

Three data points whose deviation from no relationship is just due to noise could either be in increasing or decreasing order quite easily (for continuous data there are 6 equally-likely orders, one of which is purely increasing and one purely decreasing).

On the basis of a naive test like that there's little reason to conclude that there's really anything but noise causing the appearance of a relationship.

The p-value you got is simply based on a more sophisticated* version of that sort of thinking.

You mention that your y-values are observed changes in means. If you have the original data you may be able to do more than you could with three points**, but you'd need to consider more about how the data were obtained (e.g. whether some model with a random-effects term is needed)

* it's often more powerful in that it's more likely to pick up a relationship - if one is present in the underlying "population" of possible samples you might take - under the specific assumptions it makes. For example, it assumes a linear relationship, from which much of the increase in power arises, while my naive calculation only looked at the more general case of monotonicity.

** for some reason there's a very common tendency to lose much of the original information by taking averages, while not even retaining the standard deviations of the averaged values

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.