Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

we use models with multiplicative interaction effects when relationship between independent variable and dependent variable are non-additive.

My question is, Are all models with multiplicative interaction effects non-linear? and all models with additive interaction effects linear?

Also, With non-linearity, the effect of independent variable on dependent variable depends on the value of independent variable, in effect, independent variable somehow interacts with itself. Does that mean that an independent variable(x1) interacts with itself(x1)? or does it mean that independent variables(x1) interact with other independent variables(x2, x3...) and not with itself?

I am confused with concepts of of linearity, non-linearity, additivity and non-additivity (multiplicativity).

Any help is much appreciated.

share|improve this question
add comment

1 Answer 1

up vote 2 down vote accepted

we use models with multiplicative interaction effects when relationship between independent variable and dependent variable are non-additive.

My question is, Are all models with multiplicative interaction effects non-linear? and all models with additive interaction effects linear?

The answer to such a question depends on what you mean when you say 'linear' and 'nonlinear', and what domain of models you're restricting yourself to.

Usually the terms 'linear' and 'nonlinear' in statistical models refers to linearity in the parameters, not the variables.

So for example, $y = \alpha x^2 +\epsilon$ is linear in $\alpha$ though not in $x$, while $y = \exp(-\alpha) x +\epsilon$ is non-linear in $\alpha$, though it is in $x$. In usual parlance, the first is a linear model and the second is not. However, in those cases at least both may be turned into models that are linear in both the parameters and the predictors - in the first case by the transformation $x^* = x^2$, giving a model that has a linear relationship between $y$ and $x^*$, and in the second case by the reparameterization $\alpha^* = \exp(-\alpha)$.

As such a standard general linear model (regression-type model) with multiplicative interaction is linear in the parameters, even though it's not linear in either predictor (IV). However, note that even in terms of the IVs, it is conditionally linear - fix one of the IVs and the relationship is linear in the other.

[Minor mathematical aside: It should be noted that when we're taking about the relationships of $y$ and some $x$ being linear (in this sense rather than the 'makes a straight-line' sense), if we recognize we're using homogeneous co-ordinates in regression, it is linear. I mention it because I have seen people with just enough mathematics to be dangerous object that 'linear regression is not linear'.]

all models with additive interaction effects linear?

If I understand what you're even asking with 'additive interaction effects', there's really no such thing. If it's additive it's already in the main effects and there's nothing left over for some notional 'interaction'.

Also, With non-linearity, the effect of independent variable on dependent variable depends on the value of independent variable,

Only if you think of 'effect' as inherently linear

in effect, independent variable somehow interacts with itself.

This way lies much confusion. Why not just think of there being a relationship that's described by some curve rather than by a straight line?

--

Edit to address followup questions:

What do you mean when you say "what domain of models you're restricting yourself to"?

When you said "all models with multiplicative interaction effects" you presumably meant 'all models' in some class, such as regression models, or general linear models, or generalized linear models, or ... the list could go on for some time.

Thanks! for noting about linearity. For the longest time, even I thought being linear meant the relationship was a straight line.

Me too.

This does clear some doubts, but raises a few questions. So, if we recognize we're using homogeneous co-ordinates in regression, it is linear.

In terms of $x$'s - the actual columns of the $X$-matrix in a regression - it's linear in that linked mathematical sense if you realize you're working with homogeneous co-ordinates.

A multiple linear regression is already linear in the mean parameters (i.e. the $\beta$ vector, the parameters other than $\sigma^2$), without any such need to invoke homogeneous co-ordinates. That I was referring to the relationship with the $x$'s when I raised homogeneous co-ordinates was explicitly stated.

Also, Did you mean to say "Only if you think of 'effect' as inherently nonlinear instead of linear?

Nope. The way you phrased the question I was responding to only makes sense if you take the word 'effect' to imply linearity, otherwise the whole notion of 'interaction with itself' seems to be utterly meaningless. How is one to interpret the phrase?

What I meant to ask was. I read somewhere that "with non-linearity, the effect of X on Y depends on the value of X and X somehow interacts with itself".

I regard the statement as an unhelpful attempt at analogy, and, as already explained, I think you should not think about it this way. Not everything that someone writes down is useful.

Does this mean that X interacts with itself(X)? or does it mean that X interact with other variables(X,W etc.) if any?

I'm not going to make any further attempt to interpret something that makes no sense without first having a great deal of clarification of its intent. I've suggested a way to interpret it that makes at least a little sense, kind of. If you want to interpret it more generally, explaining what it means would be up to you - or the original author of it. As a general statement, it makes no sense whatever to me.

I expect if you were to ask what, exactly, it means, you would receive an answer that contained a lot of hidden premises, and one of those premises would rely, directly or indirectly, on taking underlying meaning of 'effects' to be linear, when we have no good reason to do that.

share|improve this answer
    
What do you mean when you say "what domain of models you're restricting yourself to"? Thanks! for noting about linearity. For the longest time, even I thought being linear meant the relationship was a straight line. This does clear some doubts, but raises a few questions. So, if we recognize we're using homogeneous co-ordinates in regression, it is linear. Is this true for parameters, third, fourth, fifth variables? or just X(IV)? –  user793468 Mar 19 '13 at 23:27
    
Also, Did you mean to say "Only if you think of 'effect' as inherently nonlinear instead of linear? What I meant to ask was. I read somewhere that "with non-linearity, the effect of X on Y depends on the value of X and X somehow interacts with itself". Does this mean that X interacts with itself(X)? or does it mean that X interact with other variables(X,W etc.) if any? –  user793468 Mar 19 '13 at 23:28
    
See my edited answer –  Glen_b Mar 19 '13 at 23:46
    
I understand your point on linear relationship that when we're taking about the relationships of y and some x being linear, if we recognize we're using homogeneous co-ordinates in regression, it is linear and that the curve plot does not matter. But wouldn't homogeneous co-ordinates always plot a straight line? –  user793468 Mar 20 '13 at 21:41
    
If the equation is linear in a single $x$, then an equation in homogenous co-ordinates will generate a straight line, yes. The original point was just an aside about what 'linear' means to different people - because there's at least three ways to interpret it (linear in parameters, linear in x's under homogeneous coordinates, and generating a straight line for the plot of $y$ vs $x$). All three apply. –  Glen_b Mar 20 '13 at 21:46
show 2 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.