Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Hello: I am a computer science student working as a research assistant in an undergrad IR lab, feeling spectacularly out of my element.

Given an input of a single continuous value and a vector of several dozen boolean values, I must estimate the expected value of a single continuous value and output it. I have several thousand training examples where I have both the full input and the actual output. I suspect there is some sort of machine learning algorithm that will produce a function mapping the input to the output, but I'm unsure enough of the terminology involved that I'm not sure how best to seek these resources.

What I am looking for reminds me of classic IR classifiers such as the naive Bayesian classifier, but classifiers give you the probability of a sample belonging to a discrete class, not an expected continuous variable. Is this a form of regression? If so, what type?

Does anybody have any insight? Any webpages to help fill in the gaps in my knowledge? Or even any helpful search terms? Also, if this is an inappropriate question for stats.stackexchange.com, is there a more appropriate community based QA site you could direct me to?

Thanks.

share|improve this question
    
Similarly to the respondents to your question, I think you should try using linear regression. If you want to try fitting the model in R, the function is lm(). But any statistical software should have such a function. –  jkd Jun 20 '12 at 19:01

2 Answers 2

The question is a good fit for this site. You can almost certainly use an ordinary multiple regression model to perform this task. It would be helpful if the continuous value you want to predict is normally distributed, but that's not really necessary. More important is that the examples are independent, and that the variance of the output is constant across the training examples. You will need some software to perform the model fit. Then you will need to organize the data into a matrix with one row for each example, with the output value in the first column, and the other values represented in subsequent individual columns. For example, imagine you have 5k examples and 24 Booleans, then you will have a 5k (row) by 26 (column) matrix for your data, with each case in it's own row, the output value for that case in the first column, the Boolean values that correspond to that case in the next 24 columns (in order), and the continuous value in the last column. Terminology for multiple regression varies widely, unfortunately, but the most generic would be to refer to the output as the response variable, and the others as covariates.

share|improve this answer
    
This is quite helpful! Most multiple regression analysis I have seen take in continuous variables, not booleans. Do I just put in a "1" for true and a "0" for false? Or do I need something more advanced than that? –  Brennan Smith Jun 20 '12 at 19:26
    
Nope, that's it. It is perfectly acceptable to use Booleans as predictors / covariates in a multiple regression model. It is very commonly done. –  gung Jun 20 '12 at 19:27

This does very much sound like regression. In regression analysis you estimate the conditional expectation of the outcome variable y given the predictors which in your case are the 12 binary variable and the one continuous predictor variable. The form of the model is not clear from your description but it might be simply that the regression function is a simple linear function of the predictors. You could try ordinary linear regression fit by least squares as a first attempt.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.