Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 0 down vote favorite

(this question addresses an expanded case of How can factor-levels be automatically chosen in R to maximize the number of positive coefficients in a regression model?)

I am performing linear regression (using R) on data having both categorical (factor) and numeric variables, and fitting to a model having the form y ~ (.)^2 (i.e. including all first order and second-order interaction terms).

The question is: is there is a programmatic way of determining a coefficient vector among the set of optimal vectors $\Theta_{opt}$, where the length of the vector is minimized under the constraint that all elements of the vector are positive.

There may be cases where it is impossible to find an all-positive coefficient vector in $\Theta_{opt}$, but let's assume that the particular data which is being analyzed allows for such vectors to exist.

Perhaps one could start out with an initial (least-squares) optimal coefficient vector, and then manipulate this vector based on certain rules that depend on the nature of the terms which the coefficients are associated with. I can deduce some general rules for doing these manipulations, but don't know how to algorithmically perform the manipulations in a manner such that I arrive at a minimal-length parameter vector (parameters==0 don't count towards the vector's length). This may be heading the wrong direction though...?

share|improve this question
2  
Why do you want to do this? (For almost all statistical purposes, the choice of reference level makes no difference, nor does the number of coefficients that are positive) – guest Apr 10 '12 at 22:40
Why isn't $\Theta_{opt}=(0,0,\ldots,0)$ always a solution? – whuber♦ Apr 10 '12 at 23:11
@whuber: $\Theta_{opt}$ is a set of vectors, not a single vector. If a vector belongs to this set, it must provide a least-squares optimal solution (i.e. minimizes the squared residual sum). An all-zero vector would fail to minimize this sum. – Mark Apr 12 '12 at 19:42
@guest: statistics isn't the only purpose of regression -- it is often used for modelling systems from data. In these cases, one might have prior knowledge about the factors being used in the model, and desire to shape the model's form in such a way to improve human-interpretability of the model. For example, if I have a factor with 5 levels, and I know that only one of those levels causes an increase in the regressand variable, I would prefer a model form where that 1 level has a positive coefficient, instead of the 4 other levels having negative coefficients. – Mark Apr 12 '12 at 19:58
Mark, the reason for my comment was that $\Theta_{opt}$ does not have a clear definition. What do you mean by "optimal" vectors? Obviously $(0,0,\ldots,0)$ will be of minimal length. What criterion excludes it from the set $\Theta_{opt}$, then? You haven't given us any information in that regard. Your response sounds self-contradictory: you seek some parameter vector with certain characteristics (all elements are positive), yet you seem to focus on the LS solution, which apparently does not have those characteristics. What, exactly, are your objective function and constraints then? – whuber♦ Apr 12 '12 at 20:30
show 5 more comments

Know someone who can answer? Share a link to this question via email, Google+, Twitter, or Facebook.

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.