Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I often read about a function being 'highly non linear'. In my understanding, there is "linear" and "non-linear", so what is this 'highly' about? Is there a formal difference from non linear? How is it defined?

share|improve this question
6  
Informally: "Don't expect to be able to easily map change in input to change in output." –  keshlam Apr 17 at 21:58
1  
Did you read this in a paper about Deep Learning? Highly non-linear function approximation is one of the motivations for deep learning because a shallow network has a hard time modeling the kinds of things Joe describes in his answer. –  Neil G Apr 18 at 5:24
    
I would say it depends on where you read it. If this is written by math-savvy people, then it could mean what answers here (so far) provide. If it was written by a practitioner, like a medical doctor or a biologist, it could mean the relationship is not straight, but highly curved. In my experience, most people think linear regression refers to fitting straight lines to data, which could be part of source of the confusion. –  Roman Luštrik Apr 18 at 6:45
    
No, I didn't @NeilG. –  Toby El Tejedor Apr 22 at 11:46
    
If one of these answers helped you understand this issue, consider accepting it by clicking the check mark to its left under the vote total. –  gung May 23 at 2:03

5 Answers 5

I don't think there's a formal definition. It's my impression that it simply means that not only is it non-linear, but attempting to model it with a linear approximation won't yield reasonable results and may even cause instability in the fitting method. Someone may also use it to simply mean that small input changes can result in counter-intuitively large changes in output.

share|improve this answer
    
(+1) for offering a very sensible criterion/content for "highly non-linear" (that linear approximation may make matters worse). –  Alecos Papadopoulos Apr 17 at 14:53

In a formal sense, I believe one could say that the second derivative differs substantially from zero. If 0 were a "reasonable" approximation to the second derivative over the domain of interest, it is close to linear, but if it's not, the nonlinear effects become very important to capture.

I've rarely heard terms like this apply to relatively simple polynomials, often in practical use it seems to apply to divergent dynamical systems (chaos-theory sort of things), or very non-smooth functions (where much higher-order derivatives are nonzero).

share|improve this answer
3  
Btw, "smooth" really is a technical term, meaning that every derivative exists. x -> e^x is smooth even though its derivatives of all orders are everywhere non-zero :-) –  Steve Jessop Apr 17 at 21:03

The important aspect missing from the other excellent answers is the domain. E.g., $f(x)=x^2$ is

  • highly non-linear on $[-10;10]$ but
  • not on either half of the domain (i.e., on both $[-10;0]$ and $[0;10]$ one can possibly use a linear approximation of $f$ without an immediate disaster).

Another example is $f(x)=x^3-x$ which is

  • highly non-linear on $[-1;1]$ but
  • not on the larger domain $[-10;;10]$
share|improve this answer
    
I disagree about $x^2$. It's always highly non-linear around 0. Consider $x =[0.1,0.2,0.3]$, where $f(x)=[0.01,0.04,0.09]$. How is this linear to you? It doesn't even have linear term (obviously), it can't be approximated linearly at all. –  Aksakal Apr 18 at 13:04
2  
@Aksakal: the function is certainly not linear (anywhere), but, as I said, "one can possibly use a linear approximation of f without an immediate disaster" –  sds Apr 18 at 14:19
1  
Any function can be approximated by a line, it's only a question of how bad the approximation is. And in x \in [0, 0.5], the error isn't that bad. –  Joe Apr 18 at 18:56

As other mentioned, I don't think there's a formal definition. I would define it as a function which can not be approximated linearly in the typical range of disturbances to the argument. For instance, you have $y=f(x)$, and $\sigma^2=var[x]$. Then if the approximation $f(x+\sigma)\approx f(x)+f'(x)\sigma$ breaks down, then it's highly non-linear. For instance, $f(x)=exp(x^2)$ would be highly non-linear for any $x$ around zero, because its Taylor series are $1+x^2+x^4/2+O(x^5)$.

share|improve this answer

Informally ... "highly non linear" means "even a blind man can see its not a straight line!" ;) Personally I take it as a danger sign, that it will somehow "blow up in your face" when used with real world examples.

The Tower of Hanoi could be called an example of highly non linear ... the legend being when the monks finish a 64 disk stack, the world will end. If you count total time spent in training, feeding, housing, and motivating everyone to support a thankless boring pointless multi-generational task, I would expect the total cost in man hours to really blow out!

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.