# linear regression with categorical variables - what are the estimates of reference levels?

I've been trying with limited success to understand 'reference' levels when doing linear regressions on categorical variables in R. This post:

Effect of reference factor on T-test significance for linear regression in R

helped a bit, but there is still a lot I don't get. I wonder if someone could please answer my questions below and/or point me to relevant literature/posts.

Here's the example I used. I simulated some data about temperatures (Temp) measured at the same time of the day but in different months (Month) and places (Place). There are 12 months {Jan...Dec} and 6 places {A...F}. I assigned a constant value to each Month and Place, and calculated Temp as Month + Place + a random error. Here's what came out:

   Month Place  Temp
1    May     E 23.39
2    Dec     B 14.06
3    Nov     E 12.56
4    Nov     F 10.32
5    Aug     C 24.22
6    Jul     A 32.13
7    Oct     E 16.83
8    Feb     E  7.22
9    Jul     E 29.74
10   Aug     F 26.95
11   Jun     D 23.77
12   Jun     B 31.13
13   Dec     D  7.71
14   Sep     B 29.00
15   Jun     C 22.56
16   May     F 20.82
17   Nov     C  6.39
18   May     B 30.33
19   Aug     A 32.13
20   May     A 26.00
21   Apr     D 23.48
22   Jan     F  2.21
23   Oct     B 18.58
24   Mar     B 24.65
25   Dec     C  5.29
26   Jan     B 12.13
27   May     C 20.44
28   May     D 22.52
29   Oct     F 13.12
30   Jan     A  5.56
31   Apr     A 22.79


So I ran lm, with the intercept suppressed:

m1 <- lm(Temp~0+Month+Place,data=tdata)

sm1 <- summary(m1)

sm1

Call:
lm(formula = Temp ~ 0 + Month + Place, data = tdata)

Residuals:
Min      1Q  Median      3Q     Max
-1.9146 -0.7258  0.0000  0.6939  1.5626

Coefficients:
Estimate Std. Error t value Pr(>|t|)
MonthApr  24.3526     1.0980  22.179 2.64e-12 ***
MonthAug  30.9107     0.9721  31.797 1.87e-14 ***
MonthDec  10.3452     1.1508   8.990 3.44e-07 ***
MonthFeb   8.2012     1.7111   4.793 0.000286 ***
MonthJan   6.5432     0.9721   6.731 9.61e-06 ***
MonthJul  31.4256     1.0949  28.703 7.68e-14 ***
MonthJun  27.1452     1.1508  23.589 1.14e-12 ***
MonthMar  20.5690     1.6879  12.186 7.68e-09 ***
MonthMay  25.3779     0.8503  29.846 4.48e-14 ***
MonthNov  13.2277     1.1315  11.690 1.31e-08 ***
MonthOct  16.4136     1.1315  14.506 7.92e-10 ***
MonthSep  24.9190     1.6879  14.763 6.28e-10 ***
PlaceB     4.0810     0.9910   4.118 0.001045 **
PlaceC    -5.6213     0.9910  -5.672 5.76e-05 ***
PlaceD    -2.4352     1.0431  -2.335 0.034977 *
PlaceE    -0.9812     1.0299  -0.953 0.356896
PlaceF    -3.8106     0.9550  -3.990 0.001342 **
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 1.366 on 14 degrees of freedom
Multiple R-squared:  0.9981,    Adjusted R-squared:  0.9959
F-statistic: 438.7 on 17 and 14 DF,  p-value: 3.789e-16


The Estimates are pretty close to the values I used.
However, Place A has no Estimate or statistics.
I understand this is a necessity of the method used. But suppose that I need to make a report for an end user who wants to know about all places and all months.

Q1: What Estimate value should I put in that report for Place A? Zero? What error, t-value, Pr? Does it make sense at all to ask this?

Before using R, I used to run this kind of regressions in Maxima (function lsquares_estimates). It was much more complicated, and I could not calculate errors and other stats very easily, but it showed me something interesting.
I ran the exact same problem. One 'dependent equation' was eliminated, and I got non-numerical coefficients as the answer. I.e. there was a real variable added to each coefficient, like:

[[Jan=2.732253167457174-1.0*%R1,Feb=
4.386938346498089-1.0*%R1,Mar=16.76057521120221-1.0*%R1,Apr=
20.53682857932274-1.0*%R1,May=21.56761011969353-1.0*%R1,Jun=
23.33233046418485-1.0*%R1,Jul=27.61310220652083-1.0*%R1,Aug=
27.09988869583722-1.0*%R1,Sep=21.11123314105077-1.0*%R1,Oct=
12.60181221694821-1.0*%R1,Nov=9.416826587161458-1.0*%R1,Dec=
6.529998842026064-1.0*%R1,A=%R1+3.812202188790789,B=%R1+
7.892475858976436,C=%R1-1.80953066617677,D=%R1+
1.37794521871563,E=%R1+2.828665973718834,F=%R1]]


I guess this is equivalent to saying that F has been considered as a 'reference', and once we choose the value of F, all other values are determined.
In other cases I got more than one dependent equation eliminated, and multiple %R's were present in the solution (multiple references?).
At the time I looked a bit in the literature, and I found (I think in Wikipedia, but I can't find it back) that in such cases one can look for the values of the %R's such that the numerical range of the coefficients is minimal.
So in my example, if I take the RHS of the solution as if it were a vector, calculate its norm:

18.0*%R1^2-359.1752780077565*%R1+4061.64136679127


and differentiate it with respect to %R1 I get:

36.0*%R1-359.1752780077565


When this quantity is 0, the range spanned by the coefficients is minimised, and that happens for:

%R1=9.977091055771005


which, substituted in the original solution, yields:

[[Jan=-7.244837888313856,Feb=-5.590152709272942,Mar
=6.783484155431175,Apr=10.55973752355171,May=
11.5905190639225,Jun=13.35523940841382,Jul=17.6360111507498
,Aug=17.12279764006619,Sep=11.13414208527974,Oct=
2.624721161177185,Nov=-0.5602644686095726,Dec=-
3.447092213744966,A=13.78929324456182,B=17.86956691474747,C
=8.16756038959426,D=11.35503627448666,E=12.80575702948986,
F=9.97709105577103]]


Here the coefficients are all different from the ones I had used, but the ranking is correct within each explanatory variable. So this solution would be satisfactory if the end user only wanted to compare Months among them and Places among them; not really if he wanted to know if Places or Months are more important in determining the temperature. For that I guess R's result is better (although I still need to understand a lot about that).

Q2: What do you think about the above method to 'normalise' somehow the coefficients so that all of them can have a defined value? Is it statistically valid? Could one apply this in R? And would it then be possible to calculate the error, t-value and P for each (again, assuming it makes sense)?

Sorry for the long post; I am a beginner in statistics and I can't do technical jargon shortcuts.

• In particular, I tried the relevel command as explained in this post, setting Place B as the reference. I was relieved to see that the anova table didn't change compared to the case when Place A was the reference. – user6376297 Feb 12 '17 at 9:29