Recently, I was thinking about the following question - suppose we have some continuous variables and some categorical variables and we want to fit a regression model to this data: How will this change the overall estimation and optimization process?
The way I see it, optimizing the Maximum Likelihood Equation for this regression model might become a "Mixed Integer Optimization Problem" - whereas when there were only continuous variables, optimizing the Maximum Likelihood Equation was a standard optimization problem.
Many times, standard Maximum Likelihood Equations have closed form solutions and can be "directly" solved - other times when these equations do not have closed form solutions (e.g. estimating the parameters in a Multinomial Logistic Regression), we can use standard optimization algorithms such as Newton-Raphson or Gradient Descent and iteratively find a solution.
However, now that some of the variables are categorical, (in theory) this would mean that the Likelihood is no longer "differentiable" in the classical sense, seeing as integer constraints have now been imposed on this optimization problem. I have heard that for such problems, optimization algorithms such as "Branch and Bound" are used instead of Gradient Based Optimization Algorithms.
This leads me to my question:
When we have a Likelihood Equation that involves both continuous and categorical variables, how do usually optimize the Likelihood - can this equation still have closed form solutions or be solved using some Gradient Method?
In classical Maximum Likelihood Estimation, the parameter estimates have certain "desirable" properties such as "Unbiasedness" and "Consistency" - when there are categorical variables in the Likelihood, do the parameter estimates still "maintain these attractive properties"?
Note: I am interested in applying a Cox Proportional Hazards Model to some data with continuous and categorical variables. Although R seems to be able to estimate the regression parameters, I was wondering which optimization algorithm was being used in the background and if the resulting parameter estimates still maintain their "attractive statistical properties".