# Plot by cdplot in R doesn't fit with data

I am trying to understand how cdplot in R behaves, but I am missing something. When I copy/paste the following example given in the documentation for cdplot:

> fail <- factor(c(2, 2, 2, 2, 1, 1, 1, 1, 1, 1, 2, 1, 2, 1, 1, 1,
1, 2, 1, 1, 1, 1, 1),
levels = 1:2, labels = c("no", "yes"))

> temperature <- c(53, 57, 58, 63, 66, 67, 67, 67, 68, 69, 70, 70,
70, 70, 72, 73, 75, 75, 76, 76, 78, 79, 81)
> cdplot(fail ~ temperature)


this is the plot that I get:

Based on this graph, my expectation is that $P(\text{fail} = 1 | \text{temperature} = 67) \approx 0.77$ But when I look at the data:

> which(temperature == 67)
6 7 8

> which(temperature == 67 & fail == 1)
integer(0)


which together imply that $P(\text{fail} = 1 | \text{temperature} = 67) = 0$ (at least based on this data). What am I missing here? Why is there such a huge difference between what cdplot gives and a basic hand calculation? Also, for some variables, I get plots of this shape: cdplot gone wild

What has happened in such a plot?

-

With regard to the example you've given, the problem is not with the plot but with you apparently forgetting that you have turned fail into a factor, so fail==1 will always return a vector of FALSEs. Try:

> which(temperature == 67 & fail == "no")
[1] 6 7 8


Or actually, more useful for you would be the result of

prop.table(table(temperature, fail),1)

-
Thank you Peter for your answer. I'm afraid the problem is still there. With your correction, we have P(fail == no | temperature == 67) = 1 from the data, which is still very far from 0.77. My expectation was that at the values of t where t is in our sample, P(fail == no | temperature == t) from the data would be exactly the same as the value coming from cdplot, and for other values, cdplot would interpolate somehow. But this doesn't seem to be how cdplot operates. –  user765195 Mar 28 '12 at 3:47
Well, is 1 really that far from 0.77? Although there are no observed fails at temp=67, there are two at temp=70 and one at temp=63. –  Peter Ellis Mar 28 '12 at 9:15
Hi Peter. Yes, in my line of work, that's a huge difference. I guess my problem with the plot is that if it deviates from a simple back of the envelope calculation, I have to be able to understand completely why. Otherwise, I won't be able to use it. Sadly the person who wrote the code for cdplot gives an unpublished manuscript as his reference and I couldn't find that manuscript online. –  user765195 Mar 30 '12 at 17:32
My point in saying "that far from .77" is that I think .77 is plausible estimate of a fail rate at that point, given the fails can happen at either side of it in your very small data set. In fact, considerably more plausible than 1.0. The back of the envelope calculation only takes into account when temperature exactly equals 67, which is clearly not a satisfactory way of doing things given one would effect some degree of continuity in temperature's impact on fail. –  Peter Ellis Apr 1 '12 at 4:41
You're right Peter. I hadn't thought about the continuity correction. It makes a lot more sense now. It is probably some sort of moving average. I really wish there was a document with the full explanation of how this works. Thanks for your help in clarifying! –  user765195 Apr 1 '12 at 16:26