2
$\begingroup$

I am having some difficulty understanding something. Let's say that I have data and construct a logit model on that data. Now, let's say I have a similar and newer data set with those same variables, and I'm wondering if the original model predicts the same on the new data. Are the values expressed in the first logit model also similar on the newer data. Right, so I was wondering how one does this in R. Is it just a matter of specifying the new data when attempting to predict.

Not totally understanding the conceptual or application of this idea. Any help is appreciated.

df = data.frame(sell=c("0","1","0","0","1"), home=c("own","rent","rent","rent","own"),
                income=c(50,20,20,50,50), gender=c("M","M","F","F","F"))
df$sell = as.factor(df$sell)
df$home = as.factor(df$home)
df$income = as.factor(df$income)
df$gender = as.factor(df$gender)
str(df)
m1 = glm(factor(sell) ~ home + income + gender,
         data=df, family=binomial(link="logit"))
summary(m1)

new.df = data.frame(sell=c("1","1","1","0","0"), home=c("own","own","rent","rent","own"),
                    income=c(30,30,30,50,50), gender=c("M","M","F","M","F"))
m2 = glm(factor(sell) ~ home + income + gender,
         data=new.df, family=binomial(link="logit"))
summary(m2)

So how do I answer the question of did the original model (m1) do a good job of predicting the values in new data (new.df).

Thanks!

$\endgroup$
1
$\begingroup$

Look at the crossval function in the bootstrap package to check the proportion of correctly predicted responses on new data.

Example code:

fitLogistic = function(x,y){
  tmp=data.frame(cbind(y,x))
  fit=glm(y~.,family=binomial,data=tmp)
  fit
} 
predict.logistic = function(fitLog,x){
  x=as.data.frame(x)
  pred.logistic=predict(fitLog,newdata=x,type='response')
  ifelse(pred.logistic>=.5,1,0)
} 
resCVLog = crossval(data1[,-1], data1[,1], fitLogistic, predict.logistic, ngroup = 10)
sum(resCVLog$cv.fit==data1[,1]) / length(data1[,1])

This assumes your data is stored in a data frame called data1 where the first column are the responses and the cutoff to determine classification is .5.

A 'good' prediction rate depends on your field, although you should do better than randomly guessing.

$\endgroup$
1
$\begingroup$

What about if you just fitted the first model m1 and then used the 'predict' function to predict the responses (sell) for the new data (see e.g. http://stat.ethz.ch/R-manual/R-patched/library/stats/html/predict.glm.html).

Then you can use a Hosmer-Lemeshow test (since your response is binary) to compare your predicted values to the observed response (sell) in the new data set to assess whether your model is a good fit (see http://sas-and-r.blogspot.co.uk/2010/09/example-87-hosmer-and-lemeshow-goodness.html).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.