# Training a Logistic Regression Model

I created a quick fun Excel Spreadsheet tonight to try and predict which video games I'll enjoy if I buy them. I'm wondering if this quick example makes sense from a Logistic Regression perspective and if I am computing all of the values correctly.

Unfortunately, if I did everything correctly I doubt I have much to look forward to on my XBOX or PS3 ;)

I laid out a few categories and weighted them like so (Real spreadsheet lists twice as many or so):

 4                   4             3         1

 

Visually Stunning   Exhilirating  Artistic  Sporty 

Then I went through some games I have and rated them in each category (ratings of 0-4). I then set a separate cell to be the value of Beta_0 and tuned that until the resulting percentages all looked about right.

Next I entered in my expected ratings for the new games I was looking forward to and got percentages for those.

Example: Beta_0 := -35

 4                   4             3         1

 Visually Stunning   Exhilirating  Artistic  Sporty 

4                  4             0         1 

Would be calculated as

P = 1 / [1 + e^(-35 + (4*4 + 4*4 + 3*0 + 1*1)] P = 88.1%

If I were to automate the regression am I correct in thinking I'd be tuning Beta_0 to make it so the positive training examples come out high and the negative training examples come out low?

I'm completely new to this (just started today thanks to this site actually!) so please have no concern about bruising my ego, I'm eager to learn more.

Thanks!

-
Also help with laying out tables on this site is much welcomed and appreciated. –  Justin Bozonier Jul 28 '10 at 3:15

Like drknexus said, for a logistic regression, your outcome measure needs to be 0 and 1. I'd go back and recode your outcome as 0 (didn't like it), or 1 (did like it). Then, abandon excel and load the data into R (it's really not as intimidating as it looks). Your regression will look something like this:

glm(Liked ~ Visually.Stunning + Exhilarating + Artistic + Sporty, family = binomial, data = data)


The regression will return betas for each feature in terms of log-odds. So, for every 1 point increase in Artistic, for instance, you'll have a value for how much that increases or decreases the log-odds of your enjoyment. Most of the betas will be positive, unless you dislike sporty games or something.

Now, you'll have to ask yourself some interesting questions. The assumption of the model is that the values on each of these scores affect your enjoyment independently, which probably isn't true! A game that is very Visually.Stunning and Exhilarating is probably way better than you would expect given those component parts. And it's probably the case that if a game gets scores of 1 on all features except Sporty, which gets a 4, that high Sporty score is worth less than if the other scores were higher.

That is, many or all of your features probably interact. To fit an accurate model, then, you'll want to add in these interactions. That formula would look like this:

glm(Liked ~ Visually.Stunning * Exhilarating * Artistic * Sporty, family = binomial, data = data)


Now, there are two points of difficulty here. First, you need to have more data to fit a good model with this many interactions than the pure independence model. Second, you risk overfitting, which means that the model will very accurately describe the original data, but will be less good at making accurate predictions for future data.

Needless to say, some people spend all day fitting and refitting models like this one.

-
I'd argue from a standpoint of trying to create good guess as to what video game a person likes coding 1s and 0s for like and didn't like isn't a good approach. A scale of 1-5 or 1-7 is quite easy to elicit and will require fewer datapoints to generate a good model (because each data point provides more information). With caveats about treating ordinal data as interval data of course applying, but probably not really that important in this context. –  rpierce Jul 28 '10 at 19:23
Agreed. I just answered in terms of the question, which was about logistic regression. I've fit proportional odds logistic regressions for ordered data like this in R using MASS:polr. –  JoFrhwld Jul 28 '10 at 19:42
Thanks guys! TONS O' INFO. :D So then what I'm doing now isn't exactly logistic regression and is instead proportional odds logistic regression? –  Justin Bozonier Jul 28 '10 at 23:06
What you want to do is a proportional odds logistic regression. I'm not exactly sure what to call what you have done, –  JoFrhwld Jul 29 '10 at 3:47
Hahaha ok thanks man –  Justin Bozonier Jul 29 '10 at 5:35

Usually in logistic regression you'd want "successes" to be 1 and "failures" to be 0, but so long as you are consistent in how you enter your data and interpret it, the coefficients don't really care.

-
Interesting... I'm not exactly sure how to apply that to what I've done but I'll think on it a bit more so I can articulate a question. Thanks! –  Justin Bozonier Jul 28 '10 at 4:22