A dummy variable regression is equivalent to an ANOVA, and the beta coeffns are equal to the means of particular category with respect to the base category.
However, I am unable to interpret the "standard errors" of the beta coeffns se(beta1) etc. What do the standard errors mean and how are they calculated? Are these standard errors also related to the standard deviation of the sample for the particular category?
While @Kjetil is of course right that there is nothing special about s.e.s in a dummy variable regression, it may be instructive to look at how the expressions look like explicitly.
Take the model (which slightly differs from yours in that there is a full set of dummies rather than an intercept and dummies for all but one category) $$ y_i=\beta_1D_{1i}+\beta_2D_{2i}+u_i $$ where the dummies are such that $D_{1i}+D_{2i}=1$ for all $i=1,\ldots,n$.
Let there be $n_1$ observations such that $D_{1i}=1$ and $n_2$ such that $D_{2i}=1$. Then, the formula for the variance of the regression coefficients, $\sigma^2(X'X)^{-1}$, simplifies to $$ Var(\hat\beta)=\begin{pmatrix}\sigma^2/n_1&0\\0&\sigma^2/n_2\end{pmatrix}, $$ which are indeed nothing but the respective variances of the sample means of the $y_i$ belonging to the two different groups. The off-diagonal entries must be zero as the second regressor always has a zero entry when the first has a unit entry, so when computing the off-diagonal entries of $X'X$, we must multiply zeros and ones.
When we have one unit column and one dummy (here, $D_{1i}$), $X'X$ will become $$ \begin{pmatrix}n&n_1\\n_1&n_1\end{pmatrix} $$ so that $$ (X'X)^{-1}=\frac{1}{nn_1-n_1^2}\begin{pmatrix}n_1&-n_1\\-n_1&n\end{pmatrix} $$ or $$ (X'X)^{-1}=\begin{pmatrix}\frac{1}{n_2}&-\frac{1}{n_2}\\-\frac{1}{n_2}&\frac{n}{nn_1-n_1^2}\end{pmatrix} $$ Hence, the off-diagonal is no longer zero, but basically cancels out the variance of the baseline category. I am not so sure what to make of this finding, but note that this implies that the variance of the sum of coefficients in the regression with intercept, $$ Var(\hat\beta_0+\hat\beta_1)=\frac{1}{n-n_1}-2\frac{1}{n-n_1}+\frac{n}{nn_1-n_1^2}, $$ equals the variance of $D_{1i}$, $1/n_1$, in the regression with two dummies.
Here is a little numerical illustration of these ideas.
n <- 1000
y <- rnorm(n) # some dependent variable
x1 <- rbinom(n, size=1, p=.4) # a dummy regressor
x2 <- 1-x1 # its complement
(reg1 <- summary(lm(y~x1+x2-1)))# the regression with full dummies
n1 <- length(y[x1==1]) # the y's belonging to the first regressor
n2 <- length(y[x2==1]) # the y's belonging to the second regressor
mean(y[x1==1]) # the means are indeed the point estimates
mean(y[x1==0])
reg1$sigma*sqrt(1/n1) # reproduces the standard errors
reg1$sigma*sqrt(1/n2)
sd(y[x1==1])*sqrt(1/n1) # not exactly the same because the regression uses a "pooled" estimator for sigma^2_u
vcov(reg1) # off-diagonal element is zero
(reg2 <- summary(lm(y~x1))) # point estimate of the baseline category unaffected, also its standard error, but that of remaining dummy has changed
vcov(reg2) # now, the off-diagonal entry is no longer zero
sqrt(diag(vcov(reg2))) # another way to look at standard errors
