Deriving a "confidence relationship"?

I understand the basics of confidence intervals, the central limit theorem, etc, to be able to know things like given N samples of random variable, we're 68/95/99.7 percent sure the variable is within these two values.

I'm wondering about how this generalizes to cases where we're trying to derive a relationship between two variables. Ie, if we're trying to measure how some variable Y depends on a variable X, one thing I guess we could do is set X to 1, then get a 95% confidence interval for X = 1. Then set X += 0.1 and do that again, and again again etc, to derive a 95% "confidence relationship", or something.

This seems kind of sloppy though. Is there some other statistical mechanism designed for doing things like this (ie, finding a relationship between X and Y that we are 95% sure is the right relationship).

Say I have a bunch of (x,y) pairs. I'm trying to express the y's as a function of x. x values might repeat. I also don't necessarily care about expressing the relationship algebraically. So, the naive thing to do would be to average over the ys for every possible x value we have data for, and plot that. (so if the dataset was {(1,2), (1,3), (2, 2)}, I'd plot the points {(1,2.5), (2,2)}).

This totally disregards data about how much y data we got for each x value, though. It also seems to be ignoring the fact that we suspect the data to be correlated in some way. Like if our data points were {(1 + 1e-10, 4), (1, 17)}, my naive approach would plot {(1,17), (1 + 1e-10, 4)}. But considering how close 1 and 1e-10 are, we'd think that the values 4 and 17 should be close to each other. So maybe something like {(1, 10.5), (1 + 1e-10, 10.5)} would be more accurate.

Or, in another sense, if our dataset was {(1,16), (1+1e-10,16.2}), the fact that we got two values so close to 16 both times should seem to "inspire our confidence". But since the x values are technically different we won't notice this.

• You can get confidence intervals on a correlation, and you don't do it in the way that you imagine. There is a very simple introduction at onlinestatbook.com/2/estimation/correlation_ci.html The key is to use Fisher's z transformation, which is an atanh scale. May 28, 2013 at 0:40
• Important: a confidence interval IS NOT the interval in which there is a 67/95/99.7 percent probability of finding the parameter of interest! Another easy way of finding confidence intervals is to perform sub-sampling (e.g. bootstraping). May 28, 2013 at 1:29
• @Néstor raises a good point, but it may be hard to understand. If so, this thread: why does a 95% CI not imply a 95% chance of containing the mean? may help you. May 28, 2013 at 2:24
• @NickCox so a lot of that seemed a bit over my head, but I got the impression it was working off a model of linear correlation, right? what if I'm not expecting the values to be correlated linearly? May 28, 2013 at 5:30
• You have edited your post. It does seem as if you don't want a confidence interval for a correlation coefficient. I can't follow what you do want. There is going to be a persistent communication problem if you are using terms like correlation and confidence in your own informal senses, when there are technical uses of the same words. May 28, 2013 at 7:47

The word you missed is regression. You have $(x,y)$ pairs, and wants to "express the y's as a function of x". That is regression, and usual regression procedures will take care of "x values might repeat".