# Why the probability in my logistic regression is far from 0.5 in R?

Logistic regression models the relationship between a set of independent variables and the probability that a case is a member of one of the categories of the dependent variable. If the probability is greater than 0.5, the case is classified in the modeled category. If the probability is less than 0.50, the case is classified in the other category. The problem is that when I run the model with my dataset, the probabilities are far from 0.5, in fact it never gets to that value.

Here is part of My dataset:

  sum_profit   direction   profit_cl1
10           up          0.00
0            Not_up     -0.03
-5            Not_up      0.04
-5            Not_up     -0.04


I want to find a relationship between the price of oil and the stock price of a Colombian oil company. So the variable 'sum_profit' is the sum of the change in the stock price in the next ten minutes. The variable 'profit_cl1' shows me the net change in the oil price in the last 10 minutes.

So what I want to know is that if the oil price changes in the last 10 minutes how would I expect the stock price direction to be in the following 10 minutes (Up or Down).

The problem is that my probabilities once I run the logistic regression are far from 0.5 even though the model is significant

    glm.fit=glm(formula = direction ~ profit_cl1, family = binomial, data = datos)

Deviance Residuals:
Min       1Q   Median       3Q      Max
-0.6786  -0.6786  -0.6131  -0.6131   1.8783

Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept)     -1.57612    0.01618 -97.394   <2e-16 ***
profit_cl1       0.22485    0.02288   9.829   <2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

Null deviance: 48530  on 50309  degrees of freedom
Residual deviance: 48434  on 50308  degrees of freedom
AIC: 48438

Number of Fisher Scoring iterations: 4


The code to get the probabilities:

   log.probs=predict(glm.fit, type="response")
mean(log.probs)=0.1873


the 0.1873 is very far from 0.5.

Sorry but I did not know where else to look for help! I appreciate any suggestion!

• why do you expect the average response to be ~ 0.5 ? What is the value of mean(datos\$direction-1)? I'm guessing the stock goes down 72% of the time, either because it is generally falling or because the mean size of the downward jumps is smaller than the mean size of the upward jumps ... – Ben Bolker Oct 6 '15 at 19:29
• This isn't a coding question and is better suited for CrossValidated than StackOverflow, but here we are, so: The average of the predicted probabilities from a logistic regression model should not necessary approximate 0.5. Instead, it should approximate the base rate for the classification or event of interest. – ulfelder Oct 6 '15 at 19:29
• Thank you very much @ulfelder, I did not know CrossValidated – Juan Trujillo Oct 6 '15 at 19:40
• you can either re-post on CV yourself (probably best to delete this question as well; cross-posting is discouraged) or wait for this one to be migrated (after 3 more high-reputation users vote to close&migrate it). – Ben Bolker Oct 6 '15 at 19:48
• As @BenBolker noted, your assumption that the average response rate should be 0.5 is incorrect. Further, this is typically a very poor decision rule for classifying your predictions... Also, this appears to be a time-series question, ie correlated errors. – Alex W Oct 6 '15 at 20:20