I have an empirically gathered dataset which relates two variables. Over a small range the relationship appears linear, however over a larger range there is clearly some second order polynomial relationship as can be seen in the image at http://imgur.com/W7f9p.

I'm trying to get a measure of linearity for different ranges considered. E.g. at 20 < x < 60 or 100 < x < 120 it is very linear, but at 20 < x < 180 it is not very linear. I have tried to fit a straight line to the data and calculate the R^2 data (goodness of fit) but this shows that the straight line over the larger range has a better fit than over the smaller range. While this may be true with MS Excel, from the image it is clear that the larger range is less linear...if you hold the side of a piece of paper against the points.

Is there a better way to measure the "linearity" of a dataset?

  • $\begingroup$ This question probably should (and likely will be) migrated to the statistics site. There are a ton of great stats software for free out there. You might lookup the program R and give it a try. $\endgroup$
    – AdamRedwine
    Jun 6 '12 at 12:27
  • $\begingroup$ You might be interested in this paper ncbi.nlm.nih.gov/pubmed/16724492 The idea is to fit a piecewise linear model and to test the equality of slopes (I do not comment anymore because I haven't read it) $\endgroup$ Jun 6 '12 at 17:54
  • 2
    $\begingroup$ Are you truly interested in the linearity of the data or in the linearity of the underlying curve? The distinction is that because the data (may) sample the curve unevenly, a measure based on the data would vary according to the nature of the sampling, whereas an estimate of the linearity of the curve would be more stable under changes of sampling program. Also, do you conceive of "linearity" as an absolute property (and thereby dependent on the units of measurement) or is it a property of the shape of the curve (and thereby invariant under affine transformations of x and y)? $\endgroup$
    – whuber
    Jun 6 '12 at 22:11

Fit a quadratic instead of a linear function. The absolute value of the estimate of the highest coefficient of the quadratic serves as a sensible measure of linearity, which is zero if the data lie exactly on a line. Moreover, if the data come from a linear model with Gaussian noise, the Gauss-Markov theorem guarantees that the coefficient estimates are unbiased, hence under repetition of the fit with multiple data from the same model distribution, the expected value of the coefficient will be zero.

Of course in a single fit, one usually doesn't get zero, so one would have to use some test for the significance of the coefficients.

  • $\begingroup$ could you elaborate a little on why it serves as a sensible measure of linearity? $\endgroup$
    – Lucas Reis
    Jun 6 '12 at 16:55
  • 1
    $\begingroup$ @LucasReis: I added some rationale. $\endgroup$ Jun 6 '12 at 17:20
  • $\begingroup$ (+1) It strikes me that any reasonable measure of "linearity" ought to be invariant under changes of location and scale (in both the dependent and independent variables). That rules out using the quadratic term, but suggests there may be merit in considering using the quadratic term when standardized variables are used in the regression. Note, though, that the quadratic term will not capture complex departures from linearity such as a wave-like pattern. $\endgroup$
    – whuber
    Jun 7 '12 at 21:54
  • $\begingroup$ @whuber: good point. The answer by Michael Chernick has this property and hence is to be preferred to mine. $\endgroup$ Jun 8 '12 at 15:43
  • $\begingroup$ Actually, I prefer your solution as modified by @Douglas Maynard (who, by using betas, achieves an invariant result). I, too, initially liked Chernick's answer, but when I examined it more deeply, it appeared wanting in some key respects. One is that it strongly depends on the sampling. E.g., heavy sampling at extreme $x$ values will drive $|\rho|$ to $1$ without changing the overall curvature. Another is that it mis-characterizes horizontal lines, which are perfectly linear but for which $|rho|\approx 0$! The quadratic term, on the other hand, is robust to such capricious changes. $\endgroup$
    – whuber
    Jun 8 '12 at 15:48

One way to go would be to run a hierarchical regression with your Y-axis variable as the outcome/criterion. In step/block 1 you would enter your X variable as a predictor, and in step/block 2, enter a product term (X squared or multiplied against itself). The X squared term represents your quadratic component. The standardized regression weights (betas) for X and X squared would give you some sense of the "strength" of the linear and quadratic components relative to each other, and the change in R-squared from step/block 1 to step/block 2 is an indication of how much better the model fits the data when you have added in the quadratic component.

See Ch. 8 in Keith, T. Z. (2005). Multiple regression and beyond. Allyn & Bacon. 978-0205326440


The best measure of linearity between two variables x and y is the Pearson product moment correlation coefficient. The closer it is to 1 in absolute value the closer the fit is to a perfect straight line. Now if you think there is good linearity in a subregion, calculate the correlation for just those pairs in the subregion. If there is achange in shape outside that region it should show up in a drop in the correlation when all the data are included.


The standard statistical tools are the correlation coefficient (see Michael Chernick's answer) which is a value between [-1,1] and is unit independent. Related to the correlation coefficient is the covariance. The covariance is affected by units but may be easier to interpret. However, I don't like either of those options in the general case. I don't like them because they are not conformal transformation independent. Consider that a straight horizontal or vertical line is deemed to be non-linear by both of those measures.

A better unitless option is to use a singular value decomposition (SVD). The SVD breaks data up into component pieces ranked by magnitude of their contribution to the whole. The ratio of the largest singular number to the second largest singular number is hence a metric of linearity. Note, that to use this method you must first centralize the data (make average X, Y, Z, etc coordinates equal zero).

Example: Pts: 1126640.141 233575.2013; 1126630.008 233572.8567; 1126625.829 233572.7434;
1126625.416 233577.3781;

Centralized Pts: 9.792639127 0.656480018; -0.340591673 -1.68817349; -4.519928343 -1.801499913; -4.932119113 2.833193384;

SVD, D matrix: 11.86500017 0; 0 3.813448344

Ratio of singular Values 3.111357

The ratio above can be roughly interpreted as the data being three times as long in the direction of the best fit line as it is cross-linear.

For a solution with units that has units and doesn't require an SVD. Do some line fit that has the center of the line as one of the parameters. Using the centralized data above this is simple: line pt = 0 0 (always the case for centralized data) line direction = -0.999956849 -0.009289783

Vectors from the center of the line to each point are the centralized coordinates of the points. Find the lengths of the of the projection of these vectors onto the line (absolute value of the vector dot the line direction), and the length of the perpendicular vector component (length of vector cross line direction). Length Parallel, Length Perpendicular 9.798315123, 0.565480194; 0.356259742, 1.684936621; 4.536468847, 1.759433021; 4.905586534, 2.878889448;

The maximum of parallel projections is the stretch of data along the line. The maximum length of the perpendicular projection is a measure of the non-linearity. The ratio of the two is an approximation of the singular values ratio above.

Notes 1. Affine invariance in linearity is not possible. Consider, in an affine transformation we could scale all but one of the coordinate axes to near zero (making any set of points linear). So conformal invariance is the best we can do. 2. These methods are NOT ROBUST to outlier data. 3. Examples are 2D but generalized to N-dimensional.

  • $\begingroup$ I think you meant to say that r is in [-1,1] not [0,1] $\endgroup$
    – mdewey
    Jan 26 '17 at 15:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.