1
$\begingroup$

I was learning from Elements of statistics p.120 under section 4.4.1 Fitting Logistics Regression Models

The log likelihood function was given as

$l(\beta) = \sum_{i=1}^N {y_i\log p(x_i;\beta) + (1-y_i)log(1-p(x_i;\beta))}$

Here

$$\beta = {\{\beta_{10},\beta_1}\} \qquad (1)$$

and we assume that the vector of inputs $x_i$ includes the constant term 1 to accommodate the intercept.

Please my question is :

Assuming we have only two inputs $X$ = $X_1$ + $X_2$ and adding the intercept or constant term ($X_0)$ that contains only 1's, we will have $X$ = $X_1 + X_2 + X_0$. When we find $\beta$ using linear regression, it will be a vector in $R^3$ or the vector will contain three elements i.e $\beta = \{ b_1,b_2,b_3 \}$

How did they get $\beta_{10}$ in $(1)$ and also I want to know if $\beta_{10}$ and $\beta_1$ in $(1)$ are scalars or their vectors

$\endgroup$

1 Answer 1

1
$\begingroup$

To address your first question, assuming that you have two features, $X_1$ and $X_2$, and you choose to include an intercept, the number of parameters that you would expect to estimate is 3, i.e. $\beta = \{\beta_0, \beta_1, \beta_2\}$. However, if you define a new variable $X = X_{1}+X_{2}$, we could treat this as a single feature and therefore model using $\beta = \{\beta_0, \beta_1\}$

To address your second question regarding $\beta_{10}$ and $\beta_1$. If you look at the next statement on page 120

\begin{equation} l(\beta) = \sum_{i=1}^N {y_i\log p(x_i;\beta) + (1-y_i)log(1-p(x_i;\beta))} \\ = \sum_{i=1}^N {y_i\beta^Tx_i - log(1+e^{\beta^Tx_i})} \end{equation}

We see that $\beta$ is treated as vector which suggest that $\beta_{10}$ and $\beta_{1}$ are the components of $\beta$. Based on this, I would assume that the authors are implying these are scalar values.

$\endgroup$
3
  • $\begingroup$ Please meaning you can't use th logistics regression when you have 8 or less features. This is because we can't get $ \beta_{10} $ $\endgroup$
    – EA Lehn
    Nov 1, 2020 at 9:48
  • 1
    $\begingroup$ You can have more than 8 features - the 10 does not mean Ten. If you have a look on page 119, the authors state that a model with K classes is specified in terms of $K-1$ logit odds - each with their own set of parameters. In your example $K=2$, meaning their is only 1 set. The 1 in $\beta_{10}$ corresponds to this first and only set, while the 0 corresponds to the intercept. $\endgroup$
    – nwaldo
    Nov 1, 2020 at 15:55
  • $\begingroup$ please can you help me with this question stats.stackexchange.com/questions/494532/parameters-of-lda $\endgroup$
    – EA Lehn
    Nov 1, 2020 at 23:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.