5
$\begingroup$

Let as assume that we have a set of tuples of real numbers. Or, in other words, we have a set of (x,y) pairs. The simplest hypothesis (or assumption) about the relation between x and y is that there is a linear relation between them. We can even use a linear regression to determine what values the coefficients of the linear regression have.

But is there a way to determine if there is a statistically significant deviation of this linear dependency? Of course I am speaking about the cases when the deviation from a line is not obvious.

I assume that the question can be answered in the following way. If we have a convex function, (like square root) than a linear fit will give an underestimation in the middle and overestimation on the side of the range of x. Similarly, if we have a concave function (like exponent), we will have an overestimation in the middle and underestimation on the sides.

Is there a standard method to count (or somehow estimate) these under- and over-estimations and determine in this way if the observed measure is statistically significant?

Improve this question
$\endgroup$
7
  • $\begingroup$ Estimating the model $y\sim\sqrt{x}$ is also a linear model, so the more direct route would seem to be to fit both models and compare some metric of fit. $\endgroup$
    – Sycorax ♦
    Commented Oct 22, 2015 at 14:54
  • $\begingroup$ But $\sqrt{x}$ is not a linear function. Why do you call this model linear? $\endgroup$
    – Roman
    Commented Oct 22, 2015 at 14:56
  • $\begingroup$ Estimating a coefficient for $\sqrt{x}$ is a linear model, the same way that estimating the coefficient for $x^2$ is also a linear model. I mean that it's linear in the parameters to be estimated. $\endgroup$
    – Sycorax ♦
    Commented Oct 22, 2015 at 14:57
  • $\begingroup$ If we have completely linear dependency and then fit it with a linear function ($c + k \cdot x$) and then with something more complex and general (and non linear) (for example $c + k \cdot x + \alpha \cdot x^2$) we will get better results with the second fit. But it does not mean that assumption of non-linearity is correct. $\endgroup$
    – Roman
    Commented Oct 22, 2015 at 14:59
  • $\begingroup$ OK. Then we use different terminology. By linear dependency I mean $x + k \xdot x$, where $c$ and $k$ are model's parameters. I want to check if there is a statistically significant deviation from the linear model. $\endgroup$
    – Roman
    Commented Oct 22, 2015 at 15:00

2 Answers 2

Reset to default
1
+100
$\begingroup$

A structured approach to this problem is offered in [1] More precisely the following hypothesis test of linearity is performed: $$\text{H0: The data comes from a model with: $\text{med}(y|x) = \beta x + \alpha$ }$$

You will find more details these papers, more particularly in section 4.3 of 1 where the authors propose a test of linearity (the alternative is convexity/concavity).

If you have a vector of values of $y$ and a vector of values of $x$, this approach is fairly easy to implement. Check the description of the catline in 3

  1. The Deepest Regression Method (1997). S. Van Aelst, P.J. Rousseeuw, M. Hubert, A Struyf.
  2. Rousseeuw P., Struyf A., (2002). A Depth Test for Symmetry, in: Goodness-of-Fit Tests and Model Validity, Birkhauser Boston, pp.401-412.

Edit:

You might want to have a look at the recent conproj R package by X. Liao, M. C. Meyer. There is also a JoSS article associated with it by the same authors. Among other things, this package implements a (bootrap based) test of whether the function $f$ in the model:

$$y_i=f(x_i)+e_i,\; e_i\sim\text{i.i.d.}\;\mathcal{N}(0,1)$$

is convex (concave) or linear.

Improve this answer
$\endgroup$
1
$\begingroup$

Stephen Wright has a good discussion of one approach to understanding convexity in this NIPS tutorial from 2010. It's in the context of machine learning optimization algorithms where he gives a good definition of convexity about 3:57 minutes into the presentation with topic "First-Order Methods."

http://videolectures.net/nips2010_wright_oaml/

(Apologies for the absence of formulas in this answer -- which would help clarify things -- but I don't know how to integrate mathematical symbols into my response.)

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you for the answer. I will check. To get formulas, put LaTeX expressions between dollar signs. For example $\frac{x}{y}$ will give: $\frac{x}{y}$. $\endgroup$
    – Roman
    Commented Oct 22, 2015 at 16:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.