0
$\begingroup$

I'm performing a case of polynomial regression. I use a power $k$ for the regressors (e.g. marketing spend), which helps me determine the nature of the response curve.

I also need to estimate the coefficient for each regressor.

Consider the simplistic case: $y = ax^k + c$ ; $c$ constant, $a$ a coefficient.

The values of $k$ and $a$ need to be determined (if polynomial, $k$, $x$ and $a$ would be vectors). I vary $k$ between $-2$ and $2$ and find the value of $k$ for which pow(x,k) correlates best with $y$ using a SAS macro. I take the top three $k$ which help $x$ correlate with $y$.

I start regressing $y$ on pow(x,k) and vary $k$ between the top values in priority and observe model fit and error structure to decide.

This is a slightly approximate approach (depending on the intervals of $k$ which I choose to iterate over, 0.01/0.1 etc.), but has worked well in polynomial situations because it is a SAS macro and runs pretty fast.

Is there a better approach?

Editing to add some more context as suggested by @Nick-Cox. The dependent is the sales of a product. The regressors (x) are marketing spends.

There is a strong hypothesis backing interaction effects between the x's.

Another requirement is that not all marketing spends should be forced to have a diminishing impact on sales.

$\endgroup$
5
  • $\begingroup$ What is the best approach depends on the conditional distribution of $y$; you might be looking at either nonlinear least squares (weighted or unweighted) or generalized nonlinear models. What are the y-values? $\endgroup$
    – Glen_b
    Commented Oct 19, 2013 at 5:53
  • $\begingroup$ Much overlap with stats.stackexchange.com/questions/59784/… Your data may be different, but in my experience additive errors are usually implausible for power functions. As in the thread cited, a power function is not best described as even a special case of of a polynomial. $\endgroup$
    – Nick Cox
    Commented Oct 19, 2013 at 7:49
  • $\begingroup$ How do you handle the potentially strong overfitting? By external validation? $\endgroup$
    – Michael M
    Commented Oct 19, 2013 at 8:20
  • $\begingroup$ How I'm handling over-fitting: Build a model with 75% of data. And, then fit the model with the same set of variables for each additional week of data. And, see how the coefficients for each variable vary across weeks. If the variance of coefficients is not very high, I conclude that the model is stable and responds well to new data/rejecting any fear of over-fitting. $\endgroup$ Commented Oct 19, 2013 at 10:33
  • $\begingroup$ Sales may tend to be fairly skew (I mean the conditional distribution); you might want to consider a GLM (perhaps a gamma family), with a log-link. Alternatively, transformation may make the distribution less skew. $\endgroup$
    – Glen_b
    Commented Oct 19, 2013 at 10:40

1 Answer 1

1
$\begingroup$

Although you are sensibly keeping a careful eye on what fits, your approach can fairly be described as rather home-grown or ad hoc. Depending on your target audience or readership, the consequence may range from practitioner puzzlement to statistician flak.

Your general model I take to be a sum of power functions. With some change of notation that could be

$y = b_0 + \sum_{j=1}^J b_j x_j^{k_j}$

with additive error. As @Glen_b comments, the usual approach to fitting such a model would be to use nonlinear least squares, which I take to be well supported in SAS, although I can't advise on details.

In many ways a simpler model is a multiplicative power function

$y = B_0 \prod_{j=1}^J x_j^{b_j}$, from which

$\ln y = b_0 + \sum_{j=1}^J b_j \ln x_j$, where $\ln B_0 =: b_0$.

That model is easy to fit by least squares as it is just multiple regression on the logged variables. Error with this is taken to be multiplicative on the original scale and additive on a logarithmic scale, which is often about right. A virtue of this model, as with power functions taken singly, is that it can be consistent with the limiting behaviour that $y$ tends to 0 as all the $x$s tend to 0, often important economically (physically, biologically). (However, the assumption is that all data are positive. Your own approach appears consistent with occasional zeros but not with negative values.)

More generally, however, we have no sight of your data and only a hint of what the regressors or predictors are, so it is difficult to say much more except to guess that your response variable is probably something zero or positive. If so, it is helpful to ensure that predictions are always positive for all data points. I can't see that is guaranteed by your present approach.

On a terminology question: I'd advise against calling any of these models a polynomial, even if you spell out very clearly that the powers are in general not integers. Either people don't know what a polynomial is or they will expect the powers to be integers, at least as a mathematical default: there is some obscurity either way, better avoided.

Edit:

The model

$\ln y = b_0 + \sum_{j=1}^J b_j \ln x_j$

can naturally be complicated according to taste and need, e.g.

$\ln y = b_0 + \sum_{j=1}^J b_j \ln x_j + \text{extra terms}$,

where the extra terms could be in the $x_j$, the $\ln x_j$ or both.

$\endgroup$
8
  • $\begingroup$ Thanks for the response @Nick. From my edited question above, it might start to appear why a multiplicative log-log model wont satisfy my requirements. It cannot isolate the interaction effects between each pair of x (which can be isolated in an additive model). Further, a log-log model necessitates that y increases at a slower rate as x tends to infinity (large values). $\endgroup$ Commented Oct 19, 2013 at 10:10
  • $\begingroup$ Your original model doesn't (obviously) have interaction terms either; the answer in both cases is to add further terms to the model if justified. Your comment on "diminishing impacts" is cryptic, but powers necessarily being positive is not assumed or implied by either model. $\endgroup$
    – Nick Cox
    Commented Oct 19, 2013 at 10:13
  • $\begingroup$ I am not sure how adding an interaction term to a multiplicative model will help isolate the pair-wise interaction effects. Because, its already a multiplicative (has all pairwise multiplicative effects) model. $\endgroup$ Commented Oct 19, 2013 at 10:17
  • $\begingroup$ See edits above. $\endgroup$
    – Nick Cox
    Commented Oct 19, 2013 at 10:22
  • 2
    $\begingroup$ A power function can have powers $>1$, so your comment (if I understand your wprding) that $y$ necessarily increases more slowly with $x$ is quite incorrect. $\endgroup$
    – Nick Cox
    Commented Oct 19, 2013 at 10:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.