0
$\begingroup$

When we are using logistic regression, we can get the probability that $y$ belongs to class $1$ as follows:

$P(y=1|x;\theta)=\frac{1}{1+\exp(-\theta^Tx)}$.

PDF of a logistic distribution is given as follows:

$f(x)=\frac{\exp(-x)}{(1+\exp(-x))^2}$

and this is a continuous distribution.

Here are my questions:

  1. Conditional distribution of $y|x$ is logistic distribution. Is this right? Or is it a categorical distribution?
  2. PDF at a single point is zero. So how can we get probabilities in logistic regression if it is a continuous distribution?
  3. As a side question, what is it we get when we evaluate a PDF at a single point? For example, in scipy.stats, norm().pdf(3) returns a value, what is it?
$\endgroup$
2
  • 1
    $\begingroup$ @kjetilbhalvorsen I'm sure it is because I'm very bad at statistics, but I couldn't make use of the answers there. The first one is related, but my other questions are much more basic, like the definition of pdf. Unfortunately I'm getting confused again and again, so I need a more intuitive and basic explanation. $\endgroup$
    – groove
    Commented Oct 11, 2020 at 22:52
  • $\begingroup$ Then I am afraid what you need is a good book or a good course ... $\endgroup$ Commented Oct 24, 2020 at 19:40

1 Answer 1

0
$\begingroup$
  1. Conditional distribution of 𝑦|𝑥 is logistic distribution. Is this right? Or is it a categorical distribution?

No, $y$ is a binary variable so cannot have a logistic distribution. The conditional distribution of $y\mid x$ is binomial.

  1. PDF at a single point is zero. So how can we get probabilities in logistic regression if it is a continuous distribution?

$y$ is categorical, so does not have a pdf, it has a pmf (probability mass function.) By the way, the PDF at a single point does not need to be zero, see Can a probability distribution value exceeding 1 be OK? for intuition.

  1. As a side question, what is it we get when we evaluate a PDF at a single point? For example, in scipy.stats, norm().pdf(3) returns a value, what is it?

See the answer above.

For the question in the title about relationship logistic distribution and logistic regression see How is Logistic Regression related to Logistic Distribution?

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.