1
$\begingroup$

I have a dataset of tasks. Each task:

  • has a duration (difftime in seconds)
  • belongs to a project (project name factor)
  • has an owner (user name factor)

In R , I can simulate my data by something like:

set.seed(1)
dat <- sample(data.frame(duration=runif(N,1,8),
                         owner=sample(gl(N/50,50)),    
                         project=sample(gl(N/60,60))))

Is it possible to estimate duration using owner / project categorical variables? What are the best methods to explore: classification, estimation, clustering, trees, something else?

I am not looking for an exact solution but rather I want to have a methodology to deal with this kind of problem.

$\endgroup$
  • $\begingroup$ Are you simply looking for survival analysis? $\endgroup$ – gung Dec 7 '13 at 1:05
  • $\begingroup$ @gung thanks. maybe but it looks that survival analysis answers question like( correct me please) what is the proportion of a tasks that will not finish after a certain time? This is a part of an answer , I am looking also to characterize this tasks, can I do this using survival? $\endgroup$ – agstudy Dec 7 '13 at 1:12
  • $\begingroup$ The most popular model in SA is the Cox proportional hazards model, however, if you use a parametric SA model, you can get exact predicted durations. They provide a characterization of the whole distribution of survival times. You could fit dummies for your categories & get the expected duration for each. If you wanted to think of your categories as random effects, there are frailty models, albeit more complex. $\endgroup$ – gung Dec 7 '13 at 1:22
  • $\begingroup$ The proportion that survives longer than some time in SA is not really different than the proportion of the conditional distribution of Y (the residuals) that is greater than some number in linear regression. The expected value of Y given xi in regression is y-hat; in SA, the expected value of Y is y-hat, too. $\endgroup$ – gung Dec 7 '13 at 1:26
1
$\begingroup$

This website gives a pretty good description of how to create a multiple regression to predict a continuous dependent variable (e.g., duration) based upon categorical independent variables (such as owner/project). A key point to understand is that binary valued (0 or 1) categorical input variables can be used to extend the more "typical" style of regression model (i.e., one normally based upon continuously valued input variables rather than categorical) in a relatively straightforward way, but multiple-valued categorical variables cannot. It's therefore customary to work around this problem by using a trick: if you have a categorical variable with $k$ different possible values, you first transform it by "dummy coding" it into $(k-1)$ binary yes/no variables, and then you can substitute these newly created dummy variables as the actual direct inputs into the regression itself, and simply calculate the regression in the usual way.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.