# Clarification for $\beta = {\{\beta_{10},\beta_1}\}$ when fitting logistic regression and the number of classes is k=2

I was learning from Elements of statistics p.120 under section 4.4.1 Fitting Logistics Regression Models

The log likelihood function was given as

$$l(\beta) = \sum_{i=1}^N {y_i\log p(x_i;\beta) + (1-y_i)log(1-p(x_i;\beta))}$$

Here

$$\beta = {\{\beta_{10},\beta_1}\} \qquad (1)$$

and we assume that the vector of inputs $$x_i$$ includes the constant term 1 to accommodate the intercept.

Assuming we have only two inputs $$X$$ = $$X_1$$ + $$X_2$$ and adding the intercept or constant term ($$X_0)$$ that contains only 1's, we will have $$X$$ = $$X_1 + X_2 + X_0$$. When we find $$\beta$$ using linear regression, it will be a vector in $$R^3$$ or the vector will contain three elements i.e $$\beta = \{ b_1,b_2,b_3 \}$$

How did they get $$\beta_{10}$$ in $$(1)$$ and also I want to know if $$\beta_{10}$$ and $$\beta_1$$ in $$(1)$$ are scalars or their vectors

To address your first question, assuming that you have two features, $$X_1$$ and $$X_2$$, and you choose to include an intercept, the number of parameters that you would expect to estimate is 3, i.e. $$\beta = \{\beta_0, \beta_1, \beta_2\}$$. However, if you define a new variable $$X = X_{1}+X_{2}$$, we could treat this as a single feature and therefore model using $$\beta = \{\beta_0, \beta_1\}$$
To address your second question regarding $$\beta_{10}$$ and $$\beta_1$$. If you look at the next statement on page 120
$$\begin{equation} l(\beta) = \sum_{i=1}^N {y_i\log p(x_i;\beta) + (1-y_i)log(1-p(x_i;\beta))} \\ = \sum_{i=1}^N {y_i\beta^Tx_i - log(1+e^{\beta^Tx_i})} \end{equation}$$
We see that $$\beta$$ is treated as vector which suggest that $$\beta_{10}$$ and $$\beta_{1}$$ are the components of $$\beta$$. Based on this, I would assume that the authors are implying these are scalar values.
• Please meaning you can't use th logistics regression when you have 8 or less features. This is because we can't get $\beta_{10}$ Nov 1, 2020 at 9:48
• You can have more than 8 features - the 10 does not mean Ten. If you have a look on page 119, the authors state that a model with K classes is specified in terms of $K-1$ logit odds - each with their own set of parameters. In your example $K=2$, meaning their is only 1 set. The 1 in $\beta_{10}$ corresponds to this first and only set, while the 0 corresponds to the intercept. Nov 1, 2020 at 15:55