Trouble implementing logistic regression

Question

I am trying to implement logistic regression in my app. I know this question is software based but the concept is mathematical.

I have a table. The X axis represents minutes in a day and starts from 1 all the way to 1440. 1 represents 12am, 3 represents 12:03am etc. The y axis represents occupied / vacant, 0 and 1 respectively:

I am using the XLMiner analysis tool (it is a Excel add on as well as a google drive add on) to create the coefficients and it generated the following coefficients: 0.7861840684, -0.0006480269158.

I've plugged in the relevant variables in the following equation which is used in logistic regression: prediction = (Math.exp(betaZero + betaOne * currentTime)) / (1 + Math.exp(betaZero + betaOne * currentTime)); where betaZero = 0.7287119626, betaOne = -0.0006480269158 and currentTime is the current time on the phone represented in the amount of minutes that has passed since 00:00am.

When I compute the value of the prediction, is always predicts >0.5, even during the time where the data specifies that it should be <0.5. I'm not sure if I've calculated this right. This is my first exposure to machine learning and in this type of mathematics.

I am using Java.

EDIT = After calculating the probability for all time (1 - 1440), I can see that the probability NEVER drops below 0.5... infact, it starts at 0.6744390764013958 and as time increases, the probability drops only till 0.5451722142011737.

Your data has a striking pattern. What question are you trying to address with your regression? You may have a situation where logistic regression doesn't help much given the obvious pattern in your data. — Todd D, Commented Jan 17, 2018 at 18:30
You don't have 1440 observations: you have three, of which two are incomplete! They consist of an interval ending at 480, the interval [481,1088], and an interval beginning at 1089. There's not a whole lot anyone can legitimately infer from such a small amount of data. What is the purpose of your app? Do you in practice have much more information? — whuber, Commented Jan 17, 2018 at 18:49
What is the trouble? You have a balanced quadratic trend in risk, so the prediction using a linear term is a flat response. The implementation was a success. The model choice was poor. — AdamO, Commented Jan 17, 2018 at 19:43
I disagree with the vote to close here. This is essentially about what logistic regression does and doesn't do and what to do instead. It's not focused on code or programming. The use of Stata in my answer is just to confirm the OP's calculations (their code being invisible here) . — Nick Cox, Commented Jan 18, 2018 at 15:36

Nick Cox · Accepted Answer · 2018-01-17 18:46:01Z

This is what I get with Stata for your data.

. logit occ time

Iteration 0:   log likelihood = -980.63877  
Iteration 1:   log likelihood = -968.10592  
Iteration 2:   log likelihood = -968.09925  
Iteration 3:   log likelihood = -968.09925  

Logistic regression                             Number of obs     =      1,440
                                                LR chi2(1)        =      25.08
                                                Prob > chi2       =     0.0000
Log likelihood = -968.09925                     Pseudo R2         =     0.0128

------------------------------------------------------------------------------
    occupied |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        time |   -.000648   .0001305    -4.97   0.000    -.0009037   -.0003923
       _cons |   .7861841   .1102782     7.13   0.000     .5700429    1.002325
------------------------------------------------------------------------------

We agree on the coefficients to the precision shown above, so your code looks good. I didn't ask Stata for more decimal places.

I don't see where 0.7287119626 comes from, however.

The real question is whether your results are surprising. A graph of the data resolves the puzzlement.

Logit regression can't be smarter than what you ask it to do. Here with just time as a predictor the best it can do is fit a monotonic curve, which for these data doesn't vary that much, as you report. In broad terms you have three blocks of points that vote in different ways. The 1s pull up the curve; the 0s pull it down. The first 480 points are 1s and the last 352 are 1s, so that asymmetry drives a discernible downward slope.

To get closer to the data you'd need more predictors (e.g. a quadratic would give you a turning point too), but it's hard to see why that would be interesting or useful in this case. The data are deterministic and defined by two steps at particular times. Logit regression can't get very close to that unless you tautologically use stepped predictors for the purpose (and there are subtle problems with such deterministic cases any way).

@NickCox Thanks for the in depthanswer. It actually helped me understand it more. I came up with this data as I am using it to predict when a parking bay is occupied. These are pretty much dummy values. In the above data, person A parks his car until 9amish and parks it back at 6pmish. I was under the assumption that logistic regression would be suitable for this approach as there are only 2 outcomes; 0 or 1.. what do you reckon? — dfqupwndguerrillamailde, Commented Jan 17, 2018 at 19:25
Broadly, my view is that it's unsuitable for your data for the reasons stated. In real data you could presumably focus on times of arrival and departure, which could be expected to vary. — Nick Cox, Commented Jan 17, 2018 at 19:30
@NickCox I could add days into the data so add data for 7 days of the week as the bay will be used differently on weekdays than it is used on weekends? More data means more variety hence a better prediction...? — dfqupwndguerrillamailde, Commented Jan 17, 2018 at 19:38
You need to think about what is random in your problem. Logistic regression assumes that the response is random, but governed by a probability. In your case, the boundaries seem random, while the observations between the boundaries are deterministic. — Matthew Drury, Commented Jan 17, 2018 at 20:55

Stack Exchange Network

Trouble implementing logistic regression

1 Answer 1

Your Answer

Not the answer you're looking for? Browse other questions tagged
regression
machine-learning
probability
logistic
or ask your own question.

Hot Network Questions

Trouble implementing logistic regression

1 Answer 1

Your Answer

Sign up or log in

Post as a guest

Not the answer you're looking for? Browse other questions tagged regressionmachine-learningprobabilitylogistic or ask your own question.

Related

Hot Network Questions

Not the answer you're looking for? Browse other questions tagged
regression
machine-learning
probability
logistic
or ask your own question.