2
$\begingroup$

This data is for a company who wants to minimize their expenses. The expenses is described out of the production costs . (Production, salary, capital and material prices). I need to create a regression model, which explains the expenses as good as possible by using R.

     Expenses   Production   Salary    Capital   Material prices
     30.8803877   5643.56430 10183.508 61.04610  27.85258
     8.6860685   1627.16270  7913.191 79.22792  31.68327
    29.8040801   4187.41870  7997.240 74.12741  47.43044
     2.2369237    198.01980  7047.205 64.95149  38.00380
    12.7842783   2632.26320  8470.687 70.30203  30.97410
     3.0430043    869.08690  8373.797 68.23382  21.06911
    46.8034799  11227.12260  8886.279 83.89439  23.62836
   119.3855374  33823.38200  7264.246 70.81508  22.15031
    58.1212115  17876.78750  6289.039 73.40234  20.62116
    1.3423342    183.01830  5063.996 74.43744  35.51355
   16.1720170   2863.28630  7510.121 81.55815  30.24042
    5.5976597    938.09380  7704.030 77.20472  25.40254
  282.2761248  48244.82400  9674.347 70.77708  36.51365
   31.9915988  10150.01490  6438.564 73.14731  18.53615
   90.3808372  21958.19560  7955.265 83.34633  22.91379
  277.3242296  72254.22470  7420.662 56.30663  25.09391
   21.5608559   3886.38860  9539.634 63.57536  30.89249
  737.4825409 115511.55000  8439.644 76.62766  38.85388
  125.3481336  24003.40010  8048.155 74.37944  33.09651
   28.7889786   9660.96600  6687.399 79.54995  20.26503
   67.8629856  15221.52200  6986.789 74.03240  25.66457
   17.2912290   2763.27630  8059.716 80.66807  45.66817
  183.2498231  27120.71180  9915.351 78.48785  41.76198
    3.1507150    378.03780  7896.220 60.28303  42.47255
   18.8981896   2020.20200 10807.281 79.57796  46.90469
   17.4819480   4148.41480  7537.644 74.03240  24.58616
   10.3146314   1886.18860  6834.613 67.68677  25.60256
   31.2192216   9057.90570 10600.920 31.72817  21.58216
   79.0784070  12707.27060  9283.438 70.86009  37.25142
   42.2556251   7320.73200  5880.098 92.07221  39.21432
   13.2692268   2445.24450  8765.626 76.14761  28.36764
   22.2645262   5601.56010  7582.448 68.86489  26.61766
    7.6243624   1340.13400  6439.064 79.22792  38.15211
    0.6159616     50.00500  9205.160 90.47905  32.07321
   31.6399637   4617.46170  6489.039 70.81308  41.99320
   24.7258723   5331.53310  7862.196 43.42234  28.48085
  469.2321185  66496.64900  9138.944 78.98790  41.01410
   32.5872584   7930.79300  7119.722 49.00190  22.84028
   31.2953292   7484.74840  8064.536 67.68677  23.52905
   42.4159412   8650.86500  7147.385 72.97430  27.87209
   29.4899487   5292.52920  8177.148 76.14761  22.83968
   68.4868480  12543.25420  8143.654 80.39304  35.79178
   30.1708168   9602.96020  7054.885 59.98300  20.20302
   19.9027901   5785.57850  7970.347 71.91719  22.24702
   55.1819176  10856.08550  8062.766 71.49715  31.76328
   66.1098103  11838.18370  8710.301 75.38654  31.33523
   78.7106703  17133.71320  8161.616 78.90689  25.54035
   35.1126109   5699.56990  7989.279 80.37804  42.63936
    0.4887489     14.00140  5439.434 86.11861  34.15341
  147.0036989  25149.51470  9901.750 75.73257  33.47755
  169.2523235  38346.83430  9118.072 65.99860  31.59286
    6.6227622   1500.15000  7589.459 82.84828  21.39724
   27.2470244   4568.45680  9620.722 76.14761  35.66807
  106.2835273  28374.83720  7503.680 66.58666  24.44544
   25.6119609   6604.66040  7715.111 90.87909  21.27113
   22.4443442   5648.56480  8955.015 78.44784  25.91859
   12.3255324   2816.28160 13045.304 58.86589  30.59306
    2.2589259    295.02950  8219.222 71.94719  39.20392
   27.0859083   7896.78960  7120.672 74.52045  20.11201
    1.3644364    154.01540  8291.819 87.47375  35.78358
    9.7852784   1627.16270  8356.586 76.14761  31.34053
   50.2143209  13031.30300  6735.673 55.92559  26.16662
   40.5321528   6891.68910  6770.227 74.12741  35.96870
    2.5595559    243.02430  7001.510 81.75817  33.43934
   35.5338530   6770.67700  7799.040 67.57676  29.82798
    0.7606761     65.00650  8972.787 41.24712  28.54185
   30.1910188   5317.53170  9587.589 78.01580  41.84418
   16.1780176   2487.24870  8089.259 66.62866  32.62186
   24.3999398   3490.34900  8478.078 75.42754  38.00380
  113.2668255  22524.25220  9501.730 76.73967  25.03140
   38.8510847  10058.00570  6036.554 81.58616  25.82658
  111.8974886  23219.32170  6874.417 83.88839  33.39774
    5.9286928    723.07230  6433.183 62.09221  43.88839
  240.5377514  29615.96130  9313.861 81.75817  41.89139
   47.3911386   9530.95300  7625.332 83.88839  31.58566
   22.8458844   3789.37890  6430.183 76.30763  34.26243
  134.2417228  19446.94450  9830.303 67.58676  38.80658
    0.3158316     16.30163  7728.543 38.10981  33.46335
   10.9675966   2506.25060  7413.101 57.24572  27.42344
    2.0534053    374.03740  7885.728 82.49325  26.30403
  168.3945378  34215.42120  5684.398 80.39304  40.53265
    9.4683467   2233.22330  6717.852 59.96300  22.69557
   45.1872183  12937.29360  8320.892 65.76658  22.03520
   57.7324727  10005.00040  6473.507 76.30763  28.09871
    3.7608760    377.03770  7432.983 74.12741  33.43934
   11.1102109   1961.19610  6965.506 82.49325  28.26623
   30.2097207   5286.52860  7084.808 73.33233  38.34223
   24.2927290   5316.53160  9760.806 74.03240  27.84078
    2.0360036    617.06170  9548.675 81.75817  18.49185
   30.4737471   8060.80600  7000.700 37.06171  30.77308
  282.9683940  53765.37600  9620.862 73.46735  34.64346
   79.6286621  16509.65080  9405.910 78.05180  42.21282
   30.2270224   6904.69040 11146.335 31.72817  25.18252
    1.7706770    153.01530 10964.996 57.61776  46.16462
    0.6363636     67.00670  6697.170 58.26383  25.40254
   88.1079099  26523.65210  6890.949 68.65086  20.51885
   13.6283627   2437.24370  7846.845 65.57156  30.67427
    6.6220621    856.08560  8034.523 67.68677  39.58516
   90.7258717  17281.72800  9192.389 72.97430  36.88529
   7.5499549   2969.29690  8184.158 80.66507   9.00090
  48.1173112  11115.11140  8414.701 69.98200  22.55586
  33.8847881   7382.73820  7513.471 72.36924  25.90269
  33.9766973   5708.57080 10025.202 78.10981  42.17022
   0.1304130      4.00040  6010.301 92.65927  33.20232
  22.5634561   3571.35710  7298.440 78.26283  41.59926
  72.0427035  12725.27240  8038.644 74.03240  37.52115
  21.5475545   3981.39810  8186.869 75.08951  35.20842
   8.6380637   1090.10900  8508.771 39.13091  33.90339
 113.6023591  22279.22770  5571.607 82.67627  36.04060
  55.3679362  13074.30730  7025.062 76.32763  26.28363
  19.7111709   2682.26820  9485.798 81.75817  50.45665
  41.9057902  12955.29540  6461.286 62.33623  21.75718
   9.6438643   2689.26890  6365.036 78.44784  23.49125
 107.9883978  18456.84550  6690.899 76.30763  32.96870
  14.0445043   2001.20010  8612.391 74.03240  30.64766
  97.3956386  13847.38460  7787.149 88.54885  44.16152
  40.8980894   9553.95530  7287.899 61.34113  26.84968
   9.4068406   1412.14120  7961.696 40.69607  41.53415
  77.8926885  13703.37020  7114.501 70.85708  34.96510
  80.3673359  16312.63110  7283.338 81.55815  40.97330
   4.5054505    467.04670  8411.181 76.30763  26.85269
 111.9354924  31911.19080  6972.237 75.22552  23.09391
   3.7258725    643.06430  7332.863 64.51245  21.16402
  52.7686763   9145.91450 10374.537 81.75817  35.81188
   6.8299829    946.69466 10643.224 43.60436  51.46815
   5.8493849   1025.10250  7094.029 68.23382  22.28153
   3.9825982    391.03910  5826.623 78.29583  44.63746

So far, the best fit i've made is:

lm(Expenses~production+I(Capital^3)+Salary+Material)
Improve this question
$\endgroup$
2
  • 6
    $\begingroup$ There may be some critical typographical errors in this question, such as "expenses pr production": what does this mean? Please edit your question to fix these errors. When you're doing that, consider also explaining the apparent changes that occur within the question statement: what begins as an optimization problem is restated as a regression which then turns into a model specification problem and finally a data description problem. Is that one question or four different ones? If it's one, exactly which one is it? $\endgroup$
    – whuber ♦
    Nov 29, 2013 at 17:00
  • 1
    $\begingroup$ I apoligize for an unprecise question. I have edited and hope it is more clear now, and maybe to receive a good answer. $\endgroup$
    – user3048944
    Nov 30, 2013 at 10:16

3 Answers 3

Reset to default
3
$\begingroup$

That model is qualitatively wrong in so far as it predicts negative values for Expenses for 22 out of 127 cases. This is a by-product of fitting a hyperplane to data for a non-negative response, which pays essentially no attention to the boundedness of the response, nonlinearity, or the side-effects of skewed distributions, including at least one outlier.

Your one concession to those features is to cube Capital, which doesn't seem justified to me.

It's just a result of a few minutes' tinkering, and emphatically not offered as any kind of best model, but I got not-too-bad-looking results from a Poisson regression of Expenses on the log of Production, Salary, Capital and Material prices with "robust" standard errors. See http://blog.stata.com/tag/poisson-regression/ for more on this approach.

Using log of Production is an attempt to tame the outlier and tackle the nonlinearity.

A much simpler model is a power function relating Expenses to Production, which yields a power round about 0.8, depending how you estimate it. The fact that this power is not 1 underlines that taking Expenses to depend linearly on Production is a poor approximation.

Here are observed vs fitted plots for my Poisson model with log of Product as one of the predictors and the other predictors taken as they come.

enter image description here

and with Product instead of log Product. This model is clearly performing poorly.

enter image description here

My main suggestions therefore are

  1. Fit the model on log scale using Poisson regression. (Depending on software, it may be convenient to use a generalized linear model command, function or routine.) How you calculate standard errors of estimates is important.

  2. The one predictor you must do something about is Production. It works much better when log-transformed and should not be left as it comes.

  3. Although four-predictor models can be found with fair behaviour, consider also just using (log of) Production as a single predictor.

EDIT Main command used: 

. glm Expenses logProduct Salary Capital Material_prices, vce(robust) f(poisson)
note: Expenses has noninteger values

Iteration 0:   log pseudolikelihood = -947.01393  
Iteration 1:   log pseudolikelihood = -379.14697  
Iteration 2:   log pseudolikelihood = -360.33023  
Iteration 3:   log pseudolikelihood = -360.29492  
Iteration 4:   log pseudolikelihood = -360.29492  

Generalized linear models                          No. of obs      =       127
Optimization     : ML                              Residual df     =       122
                                                   Scale parameter =         1
Deviance         =  89.59299713                    (1/df) Deviance =  .7343688
Pearson          =  92.57592166                    (1/df) Pearson  =   .758819

Variance function: V(u) = u                        [Poisson]
Link function    : g(u) = ln(u)                    [Log]

                                                   AIC             =  5.752676
Log pseudolikelihood = -360.2949178                BIC             = -501.3978

------------------------------------------------------------------------------
              |               Robust
     Expenses |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
 -------------+----------------------------------------------------------------
   logProduct |   .9827973   .0132301    74.28   0.000     .9568667    1.008728
       Salary |   .0000417   .0000122     3.43   0.001     .0000178    .0000656
      Capital |   .0028632   .0009619     2.98   0.003      .000978    .0047484
 Material_p~s |   .0255674   .0020059    12.75   0.000     .0216359    .0294988
        _cons |  -6.487564   .1684996   -38.50   0.000    -6.817817   -6.157311
Improve this answer
$\endgroup$
6
  • $\begingroup$ Thank you for the answer. In which software did you make this? and do you have the code? I am using R and when I use glm() i receive an infinite number AIC. And when I am using lm(), i get an appropriate model. I use this: fitc<-lm(log(expenses)~log(production)*log(capital)*log(salary)+log(material)) $\endgroup$
    – user3048944
    Nov 30, 2013 at 15:53
  • $\begingroup$ I used Stata. Command and results inserted above. A model which returns predictions for Expenses has, in my view, a practical edge over one that returns predictions for log of Expenses. Your functional form looks highly arbitrary without a justification. $\endgroup$
    – Nick Cox
    Nov 30, 2013 at 16:06
  • 1
    $\begingroup$ the command is just something like fit=glm(Expenses~logProduct Salary Capital Material_prices, data=data, family=quasipoisson()). You could also use sandwich estimator as in Stata command above, but I've had occasional errors with sandwich package (all my own errors I assume) so tend to avoid and calculate in less convenient ways. This is a good reference for "quasipoisson" since I don't think it is standard terminology: cran.r-project.org/web/packages/pscl/vignettes/countreg.pdf $\endgroup$
    – charles
    Nov 30, 2013 at 16:31
  • $\begingroup$ @charles If I set the family to poison i receive lower p values and than with the quasipoisson(). However I get some warnings for not using iterate numbers. $\endgroup$
    – user3048944
    Nov 30, 2013 at 18:55
  • 1
    $\begingroup$ I think that is what Nick Cox was trying to emphasize. The estimates for poisson model by be accurate, but the p-values and SE from such a model are bound to be to be incorrect since the data doesn't fit the assumptions of the poisson model. So you need some way to adjust those p-values/SE to provide a more appropriate estimate: quasipoission or sandwich estimator are two different approaches to achieving this end. Quassipoisson is just easier in R and sandwich estimator easier in Stata. (don't have much useful to say about the iterate numbers warning, sorry) $\endgroup$
    – charles
    Nov 30, 2013 at 19:03
0
$\begingroup$

Without making a log transformation, the best fit that I found was to the Harris yield density equation:

expenses = 1.0 / (a + b * pow(production, c))

Degrees of freedom (error): 124
Degrees of freedom (regression): 2
Chi-squared: 56041.8648389
R-squared: 0.946815254429
R-squared adjusted: 0.945957435952
Model F-statistic: 1103.7477973
Model F-statistic p-value: 1.11022302463e-16
Model log-likelihood: -566.899060305
AIC: 8.97478835126
BIC: 9.04197387299
Root Mean Squared Error (RMSE): 21.0065353233

a = -7.0568710281941160E-04
       std err squared: 8.41996E-08
       t-stat: -2.43196E+00
       p-stat: 1.64458E-02
       95% confidence intervals: [-1.28002E-03, -1.31356E-04]
b = 1.3275744481559514E+02
       std err squared: 6.71547E+03
       t-stat: 1.62002E+00
       p-stat: 1.07768E-01
       95% confidence intervals: [-2.94406E+01, 2.94955E+02]
c = -9.4941712799720679E-01
       std err squared: 4.08610E-03
       t-stat: -1.48526E+01
       p-stat: 0.00000E+00
       95% confidence intervals: [-1.07594E+00, -8.22896E-01]

Coefficient Covariance Matrix
[ 1.86302647e-10 4.91328980e-05 -3.90268446e-08]
[ 4.91328980e-05 1.48588673e+01 -1.15728001e-02]
[ -3.90268446e-08 -1.15728001e-02 9.04103457e-06]

I did not find that the other variables had much effect, and I did try them in combination with the production variable. I tested this in an attempt to be thorough.

Improve this answer
$\endgroup$
7
  • 1
    $\begingroup$ You can do much better with simpler models, such as expenses ~ production:material - 1, having just a single parameter. This was found automatically by Eureqa. You can improve on this by identifying suitable nonlinear transformations of the variables; logarithms would be a good start. $\endgroup$
    – whuber ♦
    Dec 2, 2013 at 21:47
  • $\begingroup$ That looks very interesting. Did Eureqa give any fit statistics for model comparison? It seems like it could be a very useful tool. $\endgroup$
    – James Phillips
    Dec 2, 2013 at 22:50
  • $\begingroup$ When i make a QQ plot of expenses ~production:material-1 I see the line is pretty bad at -2 and 2.. @JamesPhillips what software did you use for the code you uploaded? $\endgroup$
    – user3048944
    Dec 3, 2013 at 10:31
  • 1
    $\begingroup$ I note generally that the nonlinear least squares fitting presumably used by @JamesPhillips is very sensitive to initial values for the parameters. 3048944 My guess is that you are focusing too much on small details in the residuals. No model for this dataset, or most others, will catch all the detail. $\endgroup$
    – Nick Cox
    Dec 3, 2013 at 13:53
  • 1
    $\begingroup$ Since you ask what software I used, the fitting was done online at zunzun.com - a genetic algorithm is used to determine initial parameter estimates for the model, although the site does allow users to enter initial values for the parameters. The site's "function finder" was used in this way to fit the data to hundreds of named equations, and the Harris yield density equation looked like a good fit to me - thus my posting. $\endgroup$
    – James Phillips
    Dec 3, 2013 at 19:52
0
$\begingroup$

I just realized that the data you are modeling should be longitudinal, spanning some period of years, but might not be adjusted for inflation. To the degree that currency inflation affects the values in your data set, you would be modeling inflation itself. I thought of this possibility since the Harris yield density equation I suggested can be used to model currency inflation.

If your data does not yet account for currency inflation, making some correction to account for this and then re-modeling the data would generate a higher quality model.

Improve this answer
$\endgroup$
1
  • $\begingroup$ The data is not based on one company in multiple years, but one year with multiple different companies. Therefore inflation doesn't need to be taken into consideration. $\endgroup$
    – user3048944
    Dec 3, 2013 at 10:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.