4
$\begingroup$

The situation: I'm struggling with a predictive analysis of food sales prices using a generalized linear model. My dataset contains different kinds of food (cheeses, vegetables, meats, spices etc.) and hence I am splitting the dataset completely by these kinds when doing the analysis, because they are very different by nature.

The current model: The dataset/model contains both factors such as "country of production" and numeric variables such as "transport distance" which is all used in the gamma based GLM i R.

The problem: Now in general my model fits pretty well, however sometimes in rare cases some of the metric variables gets the opposite sign (+/-) than you would expect it to have, because the model somehow catches other influences.

An example: An example would be spices. All spices have a relative long "transport distance" and a relative long shelf life and hence a pretty small impact on the sales price compared to e.g. meat. So in this case the model might by accident end up giving the "transport distance" variable a small but negative value - which is of cause wrong because it would mean that the longer the distance the food was transported the lower the price would be.

My question: What kind of model should I use in R if I wan't something similar to a GLM model but I want to be able to specify restrictions on some of the variables/coefficients? E.g. if I want to say that an increased "transport distance" should ALWAYS have a positive impact on the sales price?

Ideas: I have heard something about both "Bayesian GLM" models or using a so called "prior distribution" but I have no idea which one, if any, would be the best to use..?

UPDATE The answer below by @ACD is not, exactly what I'm looking for. I don't need an explanation of WHY this occurs, I need a solution to restricting the coefficient signs :-)

$\endgroup$
  • $\begingroup$ Why do you not use a transformation that guarantees that your predictive variable is positive. example : $Y = \alpha e^{X} + \beta $, <- this is equivalent to $Y = \alpha Z + \beta$ where $Z$ is positive ( $Z=e^{X}$) $\endgroup$ – dfhgfh Oct 29 '13 at 10:00
  • $\begingroup$ @DKK this will not guarantee that the coefficient is positive, which is what NK1 is looking for. $\endgroup$ – gregmacfarlane Jun 5 '14 at 18:09
3
$\begingroup$

The negative estimated coefficient on something that you KNOW is positive comes from omitted variable bias and/or colinearity between your regressors.

For prediction, this isn't so problematic, so long as you are sampling new data to predict the outcome (price?) of from the same population as your sample. The negative coefficient comes because the variable is highly correlated with something else, making the coefficient estimate highly variable, OR because it is correlated with something important that is omitted from your model, and the negative sign is picking up the effect of that omitted factor.

But it sounds like you are also trying to do inference -- how much does an exogenous change in $X$ change $Y$. Causal inferential statistics uses different methods and has different priorities than predictive statistics. It is particularly well developed in econometrics. Basically you need to find strategies such that you can convince yourself that $E(\hat\beta|X,whatever)=\beta$, which generally involves making sure that the regressor of interest is not correlated with the error term, which is generally accomplished by controlling for observables (or unobservables in certain cases). Even if you get to that point however, colinearity will still give you highly variable coefficients, but negative signs on something that you KNOW is positive will generally come with huge standard errors (assuming no omitted variable bias).

Edit: if your model is

$$ price = g^{-1}\left(\alpha + country'\beta + \gamma distance + whatever + \epsilon\right) $$

then country will be correlated with distance. hence, if you are in Tajikistan and you are getting a spice from Vanuatu, then the coefficient on Vanuatu will be really high. After controlling for all of these country effects, the additional effect of distance may well not be positive. In this case, if you want to do inference and not prediction (and think that you can specify and estimate a model that gives a causal interpretation), then you may with to take out the country variables.

$\endgroup$
  • $\begingroup$ Thanks, The thing is, that the model become too unstable if I add more coefficients/interactions in general. When this sometimes occur, the (wrong) negative effect is pretty small, and really doesn't affect the final price that much. The reason why I'm having issues with it anyways, is that I'm exporting the estimated model, and using it an interface where users can input parameters (such as "transport distance"). Now what they will then encounter is, that the price will wrongly be lowered when the distance is increased - which again seems very unlikely and wrong. $\endgroup$ – NK1 Oct 24 '13 at 8:26
  • $\begingroup$ Well, then the solution is to force the user to specify the country of origin AND the distance, and then to restrict input that doesn't make sense. E.g: the distance between Chad and Cameroon is never outside of 0 and whatever the maximum distance is. $\endgroup$ – generic_user Oct 24 '13 at 8:30
  • 1
    $\begingroup$ I mean, take the US, China, and Cuba. The price of American goods in Cuba will be high, but not because of distance. The price of Chinese goods in the US will be low, but not because of distance. The effect of distance that you are estimating is a residualized effect. If your mode was fit with only data from those three countries, you might find that the effect of distance decreases the price (if you didn't have country dummies). $\endgroup$ – generic_user Oct 24 '13 at 8:32
  • $\begingroup$ Thanks again, but unfortunately my reality isn't that simple :-) I would much rather use a model where I could specify a restriction on the coefficients. My ideas about using some kind of prior distribution is not something I could work with in your opinion? $\endgroup$ – NK1 Oct 24 '13 at 9:10
  • 1
    $\begingroup$ Its not about some abstract statistical correctness. Once you take out the effect of a given country on price, the effect of distance that is separate from the effect of that particular country might not be positive. $\endgroup$ – generic_user Oct 29 '13 at 11:47
6
$\begingroup$

You can do this in R with Lavaan by specifying the model as a structural equation model and adding constraints. I'm not sure if it's a good idea, but it can be done.

#load library and generate some data
library(lavaan)

d <- as.data.frame(matrix(rnorm(1:3000), ncol=3, dimnames=list(NULL, c("y", "x1", "x2"))))

Run it with GLM:

> summary(glm(y ~ x1 + x2, data=d))

Call:
glm(formula = y ~ x1 + x2, data = d)

Deviance Residuals: 
    Min       1Q   Median       3Q      Max  
-3.6385  -0.5899  -0.0224   0.6024   3.0131  

Coefficients:
            Estimate Std. Error t value Pr(>|t|)
(Intercept) -0.01855    0.03021  -0.614    0.539
x1           0.01208    0.03049   0.396    0.692
x2          -0.03676    0.03021  -1.217    0.224

(Dispersion parameter for gaussian family taken to be 0.912437)

    Null deviance: 911.2  on 999  degrees of freedom
Residual deviance: 909.7  on 997  degrees of freedom
AIC: 2751.2

Then run the same model with lavaan, to check equivalence:

> model1.syntax <- '
+ y ~ x1 + x2
+ '
> summary(sem(model1.syntax, data=d))
lavaan (0.5-14) converged normally after   1 iterations

  Number of observations                          1000

  Estimator                                         ML
  Minimum Function Test Statistic                0.000
  Degrees of freedom                                 0
  P-value (Chi-square)                           1.000

Parameter estimates:

  Information                                 Expected
  Standard Errors                             Standard

                   Estimate  Std.err  Z-value  P(>|z|)
Regressions:
  y ~
    x1                0.012    0.030    0.397    0.691
    x2               -0.037    0.030   -1.219    0.223

Variances:
    y                 0.910    0.041

In lavaan, you then add constraints, by naming the parameters and adding a constraint section:

> model2.syntax <- '
+ y ~ b1 * x1 + b2 * x2
+ '
> 
> model2.constraints <- 
+   ' 
+     b1 > 0
+     b2 > 0
+   '
> 
> summary(sem(model=model2.syntax, constraints=model2.constraints, data=d))
lavaan (0.5-14) converged normally after   1 iterations

  Number of observations                          1000

  Estimator                                         ML
  Minimum Function Test Statistic                1.484
  Degrees of freedom                                 0
  P-value (Chi-square)                           0.000

Parameter estimates:

  Information                                 Observed
  Standard Errors                             Standard

                   Estimate  Std.err  Z-value  P(>|z|)
Regressions:
  y ~
    x1       (b1)     0.012       NA
    x2       (b2)     0.000       NA

Variances:
    y                 0.911    0.041

Constraints:                               Slack (>=0)
    b1 - 0                                       0.012
    b2 - 0                                       0.000

Instead of being negative, the b2 parameter is fixed to zero.

Notice that you don't get any standard errors - if you want them, you have to bootstrap. (That's described in the lavaan manual).

$\endgroup$
2
$\begingroup$

For future references, if you don't mind switching to lasso type glm, you can use cv.glmnet with the argument lower.limits to specify which parameters should not go under 0.

It also has the nice property of removing a lot of spurious correlations from the fit: using as reference model, the model with lambda = "lambda.1se" all parameters not really relevant, based on cross-validation, will be set to 0.

In my experience on datasets with similar issues, just switching to lasso fixes most of the negative values, and obviously setting lower.limits fixes all.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.