What is a contrast matrix? What exactly is contrast matrix (a term, pertaining to an analysis with categorical predictors) and how exactly is contrast matrix specified? I.e. what are columns, what are rows, what are the constraints on that matrix and what does number in column j and row i mean? I tried to look into the docs and web but it seems that everyone uses it yet there's no definition anywhere. I could backward-engineer the available pre-defined contrasts, but I think the definition should be available without that.
    > contr.treatment(4)
      2 3 4
    1 0 0 0
    2 1 0 0
    3 0 1 0
    4 0 0 1
    > contr.sum(4)
      [,1] [,2] [,3]
    1    1    0    0
    2    0    1    0
    3    0    0    1
    4   -1   -1   -1
    > contr.helmert(4)
      [,1] [,2] [,3]
    1   -1   -1   -1
    2    1   -1   -1
    3    0    2   -1
    4    0    0    3
    > contr.SAS(4)
      1 2 3
    1 1 0 0
    2 0 1 0
    3 0 0 1
    4 0 0 0

 A: A contrast compares two groups by comparing their difference with zero. 
In a contrast matrix the rows are the contrasts and must add to zero, the columns are the groups. For example:
Let's say you have 4 groups A,B,C,D that you want to compare, then the contrast matrix would be:
Group:    A  B C D
A vs B:   1 -1 0 0
C vs D:   0  0 -1 1
A,B vs D,C: 1 1 -1 -1  
Paraphrasing from Understanding Industrial Experimentation: 
If there's a group of k objects to be compared, with k subgroups averages, a contrast is defined on this set of k objects by any set of k coefficients, 
[c1, c2, c3, ... cj, ..., ck] that sum to zero.
Let C be a contrast then,
$$
C = c_{1}\mu_{1} + c_{2}\mu_{2} + ... c_{j}\mu_{j} + ... c_{k}\mu_{k}
$$
$$
C = \sum_{j=1}^{k} c_{j}\mu{j}
$$
with the constraint
$$
\sum_{j=1}^{k} c_{j} = 0 
$$
Those subgroups that are assigned a coefficient of zero will be excluded from the comparison.(*)
It is the signs of the coefficients that actually define the comparison, not the values chosen. The absolute values of the coefficients can be anything as long as the sum of the coefficients is zero.
(*)Each statistical software has a different way of indicating which subgroups will be excluded/included.
A: "Contrast matrix" is not a standard term in the statistical literature. It can have [at least] two related by distinct meanings:

*

*A matrix specifying a particular null hypothesis in an ANOVA regression (unrelated to the coding scheme), where each row is a contrast.  This is not a standard usage of the term. I used full text search in Christensen Plane Answers to Complex Questions,  Rutherford Introducing ANOVA and ANCOVA; GLM Approach, and Rencher & Schaalje Linear Models in Statistics. They all talk a lot about "contrasts"  but never ever mention the term "contrast matrix". However, as @Gus_est found, this term is used in Monahan's A Primer on Linear Models.


*A matrix specifying the coding scheme for the design matrix in an ANOVA regression. This is how the term "contrast matrix" is used in the R community (see e.g. this manual or this help page).
The answer by @Gus_est explores the first meaning. The answer by @ttnphns explores the second meaning (he calls it "contrast coding matrix" and also discusses "contrast coefficient matrix" which is a standard term in SPSS literature).

My understanding is that you were asking about meaning #2, so here goes the definition:
"Contrast matrix" in the R sense is $k\times k$ matrix $\mathbf C$ where $k$ is the number of groups, specifying how group membership is encoded in the design matrix $\mathbf X$. Specifically, if a $m$-th observation belongs to the group $i$ then $X_{mj}=C_{ij}$.
Note: usually the first column of $\mathbf C$ is the column of all ones (corresponding to the intercept column in the design matrix). When you call R commands like contr.treatment(4), you get matrix $\mathbf C$ without this first column.

I am planning to extend this answer to make an extended comment on how the answers by @ttnphns and @Gus_est fit together.
