7
$\begingroup$

I now have seen several papers that analyze U-shaped or inverse U-shaped relations among variables (in a regression framework). The general understanding I have from there is that it is a specific type of a non-linear relationship that we can all easily visualize.

However, I am a bit confused about how exactly people mathematically define U-shaped regression functions. Suppose for simplicity there is only regressor $x$.

Does having U-shaped regression function mean the regression function is convex and decreasing in $x$ up to some point $c$ and then after $c$ is convex and increasing in $x$?

Or does it simply mean that the regression function is decreasing in $x$ up to some point $c$ and then after $c$ is increasing in $x$?

$\endgroup$
  • 3
    $\begingroup$ Different authors may have different definitions--should the relation be continuous? Differentiable? Convex? The most general definition consistent with the idea of "increasing then decreasing" or "decreasing then increasing" is: A map $f:A\to\mathbb{R}$ with $A\subset\mathbb{R}$ is "U-shaped" means there exists a decomposition of $A=B\cup C$ where (1) every element of $B$ is less than or equal to every element of $C$; (2) $f$ is monotonic on both $B$ and $C$; (3) the images $f(B)$ and $f(C)$ have at least two values each; and (4) the directions of monotonicity of $f$ differ on $B$ and $C$. $\endgroup$ – whuber Nov 24 '17 at 13:40
  • $\begingroup$ @whuber This is exactly what I am looking for -- if there is a general agreement on how to define it.... $\endgroup$ – Neznajka Nov 24 '17 at 14:39
  • 2
    $\begingroup$ I couldn't attest to a general agreement--and I'm sure that many authors would object that my definition is broader than they intended. That's why I have left it as a comment. $\endgroup$ – whuber Nov 24 '17 at 14:40
  • 3
    $\begingroup$ I don't think "U-shaped" is a mathematically well-defined term; there is no universally accepted definition and I don't think you should be looking for one. I changed the first two sentences of my answer to stress that. $\endgroup$ – amoeba Nov 24 '17 at 15:05
8
+50
$\begingroup$

The short answer to your question (as stated elsewhere) is that there is no single mathematical definition of a U-shape. The comment by @whuber is the best general definition that I have seen.

I do research on tests of U-shapes and for my presentation I have a slide with the title "What does a U mean to you?", meaning that it is subjective what people mean by the term "U-shape". The most important thing is that when you use the term "U-shape", you define exactly what you mean by it, without assuming that others will know what you mean.

Since you specified the case of only one regressor, I'll focus on that. I've seen the following definitions used in various articles:

  • A U-shape is a quadratic.
  • A U-shape means convexity (for an application along these lines, see Van Landeghem's 2012 "A test for the convexity of human well-being over the life cycle: Longitudinal evidence from a 20-year panel").
  • A U-shape is a function with weighted average derivative negative until a point, and weighted average derivative positive after that point (see Uri Simonsohn's Two-Lines: The First Valid Test of U-Shaped Relationships).
  • A U-shape is a function with exactly one turning point. This corresponds to a function that is quasi-convex but not monotone.

One complication that comes up is what if the turning point is close to the ends of the range of the x variable? Should we still consider such a function a U-shape? In my opinion, such a discussion should be had when you define what a U-shape means to you for your application, and when you specify your null hypothesis.

The definition I use in my paper, Non-Parametric Testing of U-Shaped Relationships, is the following:

Let $m(x)$ be the regression function and let $S\left(X\right)$ be the support of $X$. For a specified set $A_{0}\subset S\left(X\right)$, we are interested in testing the following:

$$\begin{align*} H_{0}\colon & \exists a\in A_{0}\mbox{ st }\forall x\in S\left(X\right)\\ & m^{'}\left(x\right)\left(x-a\right)\ge0\\ \text{versus}\\H_{A}\colon & \forall a\in A_{0},\,\exists x\in S\left(X\right)\mbox{ st}\\ & m^{'}\left(x\right)\left(x-a\right)<0 \end{align*}$$

For example, in an application I test for a U-shape of life satisfaction in age from age 20 to 70, where the turning point is between ages 30 and 60. Arbitrary decisions are necessary with this proposed framework. The important thing is to be open about them and check how sensitive results are to changes (and to challenge others to do the same).

In addition to stating the null hypothesis, as always you should state the assumptions you rely on. For example, a common assumption is that the regression function is either U-shaped on monotone. See, for example, Lind and Mehlum's 2009 "With or Without U? The Appropriate Test for a U-Shaped Relationship", where they propose an improvement on the vanilla OLS quadratic test by testing that the derivative of a specified functional form is negative at the beginning of the range, and positive at the end.

An additional point to consider is: Do you want a test that rejects the null hypothesis because of a small violation of U-shapedness? If yes, consider the R package qmutest, which implements a non-parametric tests based on splines of the null hypotheses that the regression function is quasi-convex, and separately that it is monotone. If you do not want a test that gives inference against a U-shape because of a small violation, Uri's two lines test might be best if you want to test that a regression function is mostly decreasing and then mostly increasing.

Since your question was about the use of the term "U-shape" and the definition, I find it relevant to list some terms here that are used often to refer to the same thing that "U-shape" and "inverted U-shape" are used to refer to: "valley-shaped", "trough-shaped", "hill-shaped", "unimodal", "single-peaked", and "bell-shaped". There is no inherent reason why "U-shape" is a better term than the others, but its use seems to have caught on.

I'm working on a general R package that will just be an interface to specific R packages (such as qmutest) that test for U-shaped relationships however they choose to define them. The goal will be to help users compare different tests and to think hard about the exact null hypothesis they want to test, and which assumptions they are prepared to make.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ +1. I am a bit confused by this sentence: "Do you want a test that rejects the null hypothesis because of a small violation of U-shapedness?" I assume that the null is that there is no U-shapedness, so that a sufficiently small p-value were an evidence of U-shapedness, is that correct? $\endgroup$ – amoeba Nov 25 '17 at 11:32
  • 1
    $\begingroup$ (I am glad to see that you favourably mention Uri's paper: I mentioned it in my answer here and it was heavily criticized in the comments.) $\endgroup$ – amoeba Nov 25 '17 at 11:33
  • 3
    $\begingroup$ (+1) Very nice, thoughtful, authoritative overview. Welcome to our site! $\endgroup$ – whuber Nov 25 '17 at 18:17
  • 1
    $\begingroup$ @amoeba When I use "U-shape" I'm referring to definition 4 above (a function with exactly one turning point). For my test, the null is U-shapedness. What I mean is that asymptotically the null of U-shapedness will be rejected if there is any violation of U-shapedness in the underlying regression function (e.g. there are two turning points). This is not the case for Uri's test because the two lines test is about the average derivative. So there can be wiggles without necessarily leading to asymptotic inference against U-shapes. $\endgroup$ – scottkosty Nov 25 '17 at 21:56
  • 1
    $\begingroup$ @amoeba As an example, see the function labeled "sin" in Figure 2 of my paper. I believe (although I have not checked) that the two lines test would give asymptotic inference suggesting that "sin" is a U-shape, even though it has three turning points. $\endgroup$ – scottkosty Nov 25 '17 at 22:05
7
$\begingroup$

"U-shaped relationship" is not a mathematically precise term and there is no universally accepted definition. It usually means that the relationship is first decreasing and then increasing, or vice versa.

In other words, it means that the relationship is not monotonic (non-monotonic), but instead has exactly one extremum (maximum or minimum). In computer science this is sometimes called "bitonic".

Uri Simonsohn has recently written an interesting paper about testing U-shaped relationships. See his preprint Two-Lines: A Valid Alternative to the Invalid Testing of U-Shaped Relationships with Quadratic Regressions which is very readable and amusing. Here is how the paper starts:

Is there such thing as too many options, virtues, or examples in an opening sentence? Researchers are often interested in these types of questions, in assessing if the effect of $x$ on $y$ is positive for low values of $x$, but negative for high values of $x$. For ease of exposition, I refer to all such relationships as 'u-shaped,' whether they are symmetric or not (i.e., U or J shaped), and whether the effect of $x$ on $y$ goes from negative to positive or vice versa (i.e., U or inverted-U).

This supports the definition I gave above.


For a short overview of Uri's paper, one can read his DataColada post Two-lines: The First Valid Test of U-Shaped Relationships. The main point is that using quadratic regression to test the presence of a U-shaped relationship is very very wrong. Apparently quadratic fits are often used in some fields to argue in favor of a U-shaped relationship (i.e. t-test for the quadratic term is taken to be the test of U-shape-ness); this is troubling.

Here is the key figure:

U-shaped relationships

Update: There is some criticism of Uri's paper in the comments. I would like to stress that he never suggests that discontinuous two-line fits are supposed to model the data well (or that the jump at the discontinuity has some physical meaning). No. This fit is used for the sole purpose of providing a statistical test of U-shape-ness.

Of course I agree with @FrankHarrell that it makes much more sense to use a spline model to fit such nonlinear relationships. But splines do not provide a test of U-shape-ness, whereas Uri's two-line fit does.

| cite | improve this answer | |
$\endgroup$
  • 4
    $\begingroup$ I'd say that a quadratic curve points towards, in the first place, a varying slope. Which I believe is a very very good way (or at least easy way, in many circumstances) to check that. Yet a very very bad way to represent the (true) underlying relationship, especially the, if you could say, U-ness of a relationship. $\endgroup$ – Sextus Empiricus Nov 24 '17 at 11:42
  • 4
    $\begingroup$ I just read it. He says 'forcing two lines to connect introduces bias'. What a strange argument. Allowing them not to connect introduces impossibilities. I find the whole two-line argument weak. It seems to just be avoiding splines. $\endgroup$ – Frank Harrell Nov 24 '17 at 12:49
  • 2
    $\begingroup$ @FrankHarrell Well, I imagine it's hard (if at all possible) to come up with a p-value for U-shape-ness based on a splines model. I guess in many cases it's enough to build a good splines model and then just eyeball it to see if there is any evidence of U-shape-ness. And you don't like p-values anyway. So that's fine. But this paper is trying to develop some instrument for the researchers that want to compute a p-value for U-shape-ness; and this instrument should not have obviously ridiculous false positive rate like quadratic term in regression does... At least that's my understanding. $\endgroup$ – amoeba Nov 24 '17 at 13:12
  • 3
    $\begingroup$ I don't see his arguments are strong. Splines are more likely to fit; why stop at bilinear or even present it seriously? With splines, testing for association (flatness) and nonlinearity are trivial. Testing for non-monotonicity is a challenge; would like to see a reference on that. Regarding just testing for nonlinearity (but ignoring accuracy of predictions) quadratics do a pretty decent job. The two-lines method is very dependent on where you place the discontinuity. $\endgroup$ – Frank Harrell Nov 24 '17 at 13:29
  • 3
    $\begingroup$ Even though I love Bayesian modeling I'm not convinced that the thought experiment of imagining a change point is the straightest way to go. I would rather see a flexible smooth fit with a prior distribution for the degree of non-monotonicity. $\endgroup$ – Frank Harrell Nov 24 '17 at 13:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.