Below is a scatter plot of some data with the best fit OLS line. From $R^2$ we can see it is a quite a poor fit. But is there a way to predict before running the regression that the results won't be reliable? My gut-feeling is that because we've got the cluster of data points on 1.5, 2 and 2.5, the linear regression is not a good model. But I am looking for a more formal explanation. 
$\begingroup$
$\endgroup$
1
-
$\begingroup$ Note that there is a single data point at X = 3.5, and 8 or 9 data points at X = 1.5 (for example). This has the effect of weighting the data away from the high end. I suggest making a test using the average value where there are multiple data points with the same X value, and see if that might work better for your modeling needs. My eyeballing the data indicates a straight line will not be a good model in such a test, and @NickCox's suggestion of using a quadratic (or even cubic) seems wise. $\endgroup$– James PhillipsJul 16, 2018 at 13:11
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
4
You're asking several different questions.
Why the predictor takes on particular distinct values (multiples of 0.25, it seems) could be an important detail. Ideally you would have information on why that happens from the data source or from a client or boss, say that the numbers arise from division by 4 and there is a good reason for that. At worst, even with no detail on that point in this case the number of distinct values still seems large enough to detect an overall pattern.
Calling $R^2 = 0.5306$ quite poor may or may not be echoed by people who know about the data in that field and what to expect. There are fields where anything less than say $0.9$ could mean nothing but disaster. There are fields where anything higher than that means a bogus or even tautologous relationship (just two different versions of the same underlying variable) or fraud, because correlations just aren't that strong otherwise.
To my eye, the scatter suggests curvature, so I would try a quadratic next. I would also want substantive detail on whether a relationship with a turning point makes sense and also on whether the response can ever be negative.
One way to tell how well a regression will work when there is just one predictor, or you can plot in two dimensions as here, is to look at the scatter plot. Otherwise I don't know what kind of prior analysis you're reaching for.
Reliable in many statistical contexts means reproducible rather than strong. A reliable relationship will show up in similar form in a similar sample.
I don't think that there is much scope to find good models without context on the data and how they arose.
answered Jul 16, 2018 at 11:32
-
$\begingroup$ +1. But in this particular a case a quadratic will work only poorly. A spline might be a better choice for many applications, or even settling for treating the discrete regressor values as categories. A direct answer to the stated question is "yes, it's obvious a linear fit will be poor, because the differences between the conditional means and any line are greater than the conditional standard errors." In other words, a standard goodness of fit test will provide the formal answer that was requested; and from the scatterplot alone one can readily see what the test will result in. $\endgroup$– whuber ♦Jul 16, 2018 at 13:10
-
$\begingroup$ @whuber "poorly" is a little strong, not least because splines aren't to everyone's taste and from the tone of the question would possibly be more complicated than the OP might prefer. I'd like to see the data to compare different ideas. By eye a quadratic would have turning point around $x = 2.75$. I'd also like to know whether the data should honour $(0, 0)$. The dopey axis titles "Axis Title" almost look like wilful attempts to obscure what the data represent! . $\endgroup$– Nick CoxJul 16, 2018 at 13:33
-
$\begingroup$ @NickCox, I am afraid I cannot share the data, as you have guessed. But I appreciate your insight. $\endgroup$ Jul 17, 2018 at 4:07
-
$\begingroup$ Well, in principle, someone can could digitise your graph and get very close to your real data. So, if your data are confidential you shouldn't even show a graph. I remain puzzled that you expect good advice when you can't give out very much real information. That's a situation in which you need to hire a consultant within a non-disclosure contract. $\endgroup$– Nick CoxJul 17, 2018 at 13:45