Interpreting regression and $R^2$ with small $n$

Question

I am trying to create a regression comparing resampled data from 2 sampling periods using $n=6$ samples. I have tried a few different models (e.g., linear, log-transform, and exponential), and have received mixed results.

For example, the linear models look best graphically, but because I have so few data points, each point has a strong effect on the trend and therefore points with greater distance from the mean are resulting in the model having a negative adjusted $R^2$ value.

An exponential model has a much improved, and positive, adjusted $R^2$ value, but graphically doesn't make as much sense as the linear model. Essentially, the exponential model is reacting to the dispersion of points around the linear regression line and trying to make up for the residual error of more outlier-ish points. However, these points would almost certainly be within reasonable distance from the mean if more points were added.

So I have 2 questions:

1. Do I go with what looks more sensible graphically, or just blindly believe the adjusted $R^2$?
2. Obviously, more data points would be beneficial, but what are my options if I can't supplement with more data?

Examples:

3 variables demonstrating this issue. Solid color lines are model fits and dashed lines are 95% CI. Black = linear, red = log-trasnformed, green = exponential. x-axis = 1977, y-axis = 2015. Created in R using lm().

Update:

Background info: I have 21 soil variables I resampled in 1977 & 2015. Each of these variables had 6 sample plots shared b/w each sampling period. However, I have a number of 1977 samples & 2015 samples that were only sampled once (in 1 year & not the other). The soil processing was different b/w years, so the 2 years aren't directly comparable. I'm trying to 'correct' the non-resampled 1977 samples to match 2015 data (not sampled in 1977) by regressing my 6 resampled samples for each variable. I'm assuming soil has not changed in 40 years, so I'm not trying to compare soil change, but rather trying to lump all extant samples (resampled and not) for common analyses.

My example data:

dput(soil.dat)
structure(list(plot.2015 = c(5L, 10L, 24L, 25L, 44L, 51L), Silt.2015 = c(28.88, 21.84, 22.89, 25.34, 16.96, 22.36), Sand.2015 = c(63.92, 73.67, 60.38, 69.03, 81.6, 70.18), BS.2015 = c(70, 60.4, 43.2, 41, 84.4, 54), Silt.1977 = c(44L, 38L, 40L, 42L, 50L, 36L), Sand.1977 = c(37L, 49L, 37L, 36L, 39L, 42L), BS.1977 = c(36.6, 26.48, 44.08, 36.6, 53.32, 44.08)), .Names = c("plot.2015", "Silt.2015", "Sand.2015", "BS.2015", "Silt.1977", "Sand.1977", "BS.1977"), class = "data.frame", row.names = c(1L, 2L, 15L, 16L, 18L, 19L))

Answer 1

The problem is that your data is both very small and very noisy. This leads to the very different outcomes you get from your models. In such small samples outliers have very big influence on your outcome. Imagine, for example, that you removed case 44 from the first plot - then we would see "obvious" upward, quadratic trend. Now imagine we removed case 5 - the "trend" would be very different. If we removed case 10 from the second plot we would get very steep positive trend. All this makes the approach you would choose for analyzing this data have great influence on the outcome.

One approach you could possibly apply is to use Bayesian model with informative prior based on your out-of-data knowledge about the phenomenon you observe.

With such a small sample overfitting is a significant risk, so simpler (e.g. linear) models are safer that more complicated ones (e.g. including polynomials).