11
$\begingroup$

I want to associate code defects with code complexity metrics like closeness. One common model is to view this as a Poisson process, where the duration is how much time is spent coding and the density is a function of the code complexity. I am able to do a regression and get significance values etc.

However, it's hard for me to visualize the results (and even harder for my less mathematically inclined colleagues). Is there a good way to view this to look at things like outliers, if it's a linear trend etc.? (Links to R packages are appreciated.)

For example, I could just plot Defects / Time ~ Complexity but this is fairly noisy and since Defects is discretized and very small, it's hard to view the trend. One thing I was thinking of is that I could slice the data into quantiles and then do a regression per quantile and plot the resulting densities - I'm not certain how valid this is though, especially since my data are not normally distributed so it might mislead people about what a quantile is.

$\endgroup$
6
$\begingroup$

After you've fit the model, why not use predicted defects as a variable to compare to the others using whatever standard techniques are meaningful to them? It has the advantage of being a continuous variable so you can see even small differences. For example, people will understand the difference between an expected number of defects of 1.4 and of 0.6 even though they both round to one.

For an example of how the predicted value depends on two variables you could do a contour plot of time v. complexity as the two axes and colour and contours to show the predicted defects; and superimpose the actual data points on top.

The plot below needs some polishing and a legend but might be a starting point.

enter image description here

An alternative is the added variable plot or partial regression plot, more familiar from a traditional Gaussian response regression. These are implemented in the car library. Effectively the show the relationship between what is left of the response and what is left of one of the explanatory variables, after the rest of the explanatory variables have had their contribution to both the response and explanatory variables removed. In my experience most non-statistical audiences find these a bit difficult to appreciate (could by my poor explanations, of course).

enter image description here

#--------------------------------------------------------------------
# Simulate some data
n<-200
time <- rexp(n,.01)
complexity <- sample(1:5, n, prob=c(.1,.25,.35,.2,.1), replace=TRUE)
trueMod <- exp(-1 + time*.005 + complexity*.1 + complexity^2*.05)
defects <- rpois(n, trueMod)
cbind(trueMod, defects)


#----------------------------------------------------------------------
# Fit model
model <- glm(defects~time + poly(complexity,2), family=poisson)
# all sorts of diagnostic checks should be done here - not shown


#---------------------------------------------------------------------
# Two variables at once in a contour plot

# create grid
gridded <- data.frame(
    time=seq(from=0, to=max(time)*1.1, length.out=100),
    complexity=seq(from=0, to=max(complexity)*1.1, length.out=100))

# create predicted values (on the original scale)
yhat <- predict(model, newdata=expand.grid(gridded), type="response")

# draw plot
image(gridded$time, gridded$complexity, matrix(yhat,nrow=100, byrow=FALSE),
    xlab="Time", ylab="Complexity", main="Predicted average number of defects shown as colour and contours\n(actual data shown as circles)")
contour(gridded$time, gridded$complexity, matrix(yhat,nrow=100, byrow=FALSE), add=TRUE, levels=c(1,2,4,8,15,20,30,40,50,60,70,80,100))

# Add the original data
symbols(time, complexity, circles=sqrt(defects), add=T, inches=.5)

#--------------------------------------------------------------------
# added variable plots

library(car)
avPlots(model, layout=c(1,3))
$\endgroup$
2
$\begingroup$

So you run a poisson regression on the count data variable "defects" and want to visualize significant differences. What you do when you estimate that regression is estimate the conditional mean $E[y_i| x_i]$, where $y_i$ are the defects and $x_i$ are your regressors, like code complexity.

The easiest way to visualize it is to just take two different values for $x_i$, e.g., high complexity and low complexity, and then plot the predicted frequency of $y_i=0, y_i=1$ etc. for both complexity levels. The resulting histogram would look like this on slide 3. If the differences are strong, the histrograms of those frequencies will look very different (e.g., many zeros for low complexity and many high defects for high complexity). It might look nice of you overlay these two histograms, or you plot the difference in predicted defects.

If you feel reducing your $x_i$ to 2 categories is oversimplification, you can do the same for any discrete set of $x_i$, but it doesn't make for a good visual presentation if you throw more than, say, 4 graphs at your audience at once.

Alternatively, if your $x_i$ is continuous, you could plot one predicted count (e.g., defects=0) depending on that $x_i$. Maybe this results in a nice monotone fuction that is in line with your results.

Btw: have your also run a negative binomial regression on your data? Are the results very different? If so, your Poisson assumption (conditional mean equal to conditional variance) might be too restrictive (see "overdispersion"). That could be the case, for example, if your variable "defect" has many zeros.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.