How to learn the relationship between two variables

Question

I have a data set where there are two variables X and Y, X being a set of values and Y being a set of corresponding values, like X = [1, 2, 3, 4] and Y = [100, 200, 300, 400].

Ideally I should have a directly proportional relationship. When X increases Y does too, and vice versa. What I would like to do is learn what might be the relationship, is it linear or non-linear or something else. How are such things explored?

More generic question when give two such variables how do you start exploring it?
How to check if there is correlation?

Attaching the scatter plot image

Answer 1

A correlation implies a $linear$ relationship between two variables, which may not always be the case. See this for an example.

Exploratory graphics are your friend here, I like to plot scatterplots between variables with smoothers to explore relationships initially.

Answer 2

For two variables, you should definitely first plot them and look at them since it is so easy and gives great insight into the data. Humans are great at visually identifying trends. Check to see if the scatter plot is approximately linear first, and then go from there.

Answer 3

Statistician Francis Anscombe demonstrates both the importance of

1. graphing data before analyzing it, and
2. checking the effect of outliers and other influential observations on statistical properties

by showing that evaluating data relationships based solely on statistical measures is inadequate compared to preliminary visual inspection. He does this with four different datasets, each consisting of two variables $x$ and $y$, different for each:

Each of the $x$ vectors in the four graphs have the same mean and variance, same goes for the $y$ vectors. Pairwise, $(x,y)$ have the same correlation and regression line in all four graphs.

1. The first scatter plot (top left) appears to be a simple linear relationship, corresponding to two variables correlated where $y$ could be modelled as gaussian with mean linearly dependent on $x$.
2. The second graph (top right) is not distributed normally; while a relationship between the two variables is obvious, it is not linear, and the Pearson correlation coefficient is not relevant. A more general regression and the corresponding coefficient of determination would be more appropriate.
3. In the third graph (bottom left), the distribution is linear, but should have a different regression line (a robust regression would have been called for). The calculated regression is offset by the one outlier which exerts enough influence to lower the correlation coefficient from 1 to 0.816.
4. Finally, the fourth graph (bottom right) shows an example when one high-leverage point is enough to produce a high correlation coefficient, even though the other data points do not indicate any relationship between the variables.

If there is non-linearity between data series, (Pearson's) correlation is inadequate and other dependence measures such as Spearman's rank correlation and mutual information should be used.