Why is it bad to dichotomize categorical variables in regression? Say we have a research question of: "Does a specific cancer type increase the risk of seizure?"
We then have 10 cancer types and a seizure 0/1 (no/yes) dependent variable.
One way we could answer our question would be to dichotomize the cancer types variable into ten 0/1 variables for each cancer type and then run a logistic regression on each type to see if one has a higher risk than the others.
But this is apparently a bad idea. Can anyone tell me why I must have a baseline in a category? I understand that it is to compare the results with, but to me it also seems that I am answering the research question by simply regressing against a cancer type since it keeps the rest of my data baseline and looks at whether or not people in my data set with a specific cancer type have a higher risk of seizure.
 A: This problem is not conceptually different from a standard multi-category 1-way ANOVA, except that it involves logistic regression instead of (the equivalent of) standard linear regression. You thus have two issues to address: whether there are any significant differences at all among the 10 cancer types with respect to the probability of seizures, and if so which cancer types differ in that respect.
A logistic regression of seizure (binary 0/1) against a single 10-category cancer-type variable addresses the first issue. Software will typically report coefficients and p-values for differences of each of 9 cancer types from a single reference cancer type, but what you first care about here is the model as a whole. If the model is not significant overall, then you simply stop and say that there are no significant differences among these cancer types.
If the model is significant overall then you can proceed with examining differences among cancer types. Absent pre-specified hypothesis you must take into account the issues raised by multiple comparisons. This page shows a way to proceed for logistic regressions.
I understand the initial appeal of testing each cancer type against the average of all the other 9, but that can lead to all sorts of problems. For example, say that 5 types of cancer had no seizures while all patients with the other 5 types had seizures. There clearly are differences among cancers with respect to seizure probability, but if the number of cases is limited you might find no individual cancer "significantly different" from the approximately 40% - 60% seizure incidence in the average of the other 9. Combined with the lack of any test of overall difference, you also face a substantial problem with multiple comparisons as it's pretty easy to get one "significant" result by chance out of 10 comparisons even if there are no actual differences (happens about 40% of the time at p < 0.05).
A joint test for any differences among cancers followed by an analysis of individual differences that controls for multiple comparisons is the tried-and-true way to approach this type of problem. 
A: The baseline, or base case, or control variable is necessary absolutely necessary. In your model, if you were to include all cancer types as regressors, then 


*

*The regression will fail. Each of the 10 dummy variables will have complete dependency on the other 9 (i.e. if all other variables are zero, the last one MUST be one, assuming the dummies are mutually exclusive and collectively exhaustive). If you have a model where one of the variables must hold a certain value given the others, then there is no randomness between variables. Perfect negative multicollinearity would exist between each variable and the summation of rest of them.



*There would be no interpretation of coefficients even if the model did run. A given coefficient wouldn't tell you anything, because if the x(i)=0, then βx=0, but then another variable would hold a {1} value, and that βx≠0. 


A: Since you don't have covariates like age or gender, you might simply run case-control 2$\times$2 tables to determine odds ratio for each cancer vs seizure combination.  For example, consider the colon cancer cases: you'll have maybe several hundred with colon cancer and several hundred without (cancers other than colon).  Set exposure ($E=1$) for colon and $E=0$ for other cancers.  Now set seizure to $D=1$, and $D=0$ for no seizure, and run a 2$\times$2 disease vs exposure ($D\times E$) table to determine the odds of having seizures among those with colon cancer vs other cancers.  Do this same setup for each cancer-seizure combination.  There may be one cancer type that is more greatly associated with seizure -- and remember any association is not causal -- i.e. apply the Bradford Hill criteria for causation.      
Your question about baseline and regression is simply related to whether a constant term is needed when you use $k-1$ binary (0,1) dummy indicators (e.g., 9) to represent the 10 cancers.  This is called corner point coding. If you want a regression coefficient for each of the 10 dummy variables, you have to specify that you don't want an intercept term (constant) in the regression model.  This is called sum-to-zero constraints coding. However, for logistic this gets complicated.  If you had a continuous $y$-variable for outcomes instead of binary (0,1) for seizure, the 10 regression coefficients would be the mean value of $y$ among those with the various cancers.   Whereas if you used 9 ($k-1$) dummy indicators and selected a cancer for the baseline, the 9 regression coefficients would represent the delta $y$ away from the baseline mean.   
