Observing negative correlation between CV accuracy and test set accuracy I've been trying to evaluate whether it is more appropriate to split my dataset into test and training sets to estimate model accuracy or to use leave-one-out CV. 
While comparing the variation in accuracies reported by the two approaches I noticed that they tend to be negatively correlated, which doesn't make any sense to me. 
To illustrate this problem I used a different dataset that I saw mentioned in this post and also used the same pre-processing commands as used in that post. As mentioned above, I have seen the same weird negative correlation in accuracies in a different dataset as well so I don't think the pre-processing steps and/or the quirks of this specific dataset are responsible. 
I think it's more likely that I'm missing something trivial - does anyone have an idea of what is going on here? See my Rscript and scatterplot of the accuracies below.
library(caret)

### import doMC library and allow up to 30 cores to be used for multi-threading:
library(doMC)
registerDoMC(cores = 30)

### GermanCredit dataset, which is packaged with caret:
data(GermanCredit)

### prep df for all results:
out <- data.frame( matrix( NA , nrow=0 , ncol=5 ))
colnames(out) <- c("Rep" , "mtry" , "train_accuracy" , "test_accuracy" , "Kappa")

GermanCredit <- GermanCredit[, -nearZeroVar(GermanCredit)]
GermanCredit$CheckingAccountStatus.lt.0 <- NULL
GermanCredit$SavingsAccountBonds.lt.100 <- NULL
GermanCredit$EmploymentDuration.lt.1 <- NULL
GermanCredit$EmploymentDuration.Unemployed <- NULL
GermanCredit$Personal.Male.Married.Widowed <- NULL
GermanCredit$Property.Unknown <- NULL
GermanCredit$Housing.ForFree <- NULL

### use leave-one-out CV and a grid search for the mtry parameter:
fit_control <- trainControl( method = "LOOCV" ,  search="grid" )

for ( i in 1:100) {

  ### change random seed for every replicate (for both dividing data into test/training sets and for the RF itself)
  set.seed(1991 + i)

  ### divide data into test/training sets:
  inTrain <- createDataPartition(GermanCredit$Class, p = .8)[[1]]
  GermanCreditTrain <- GermanCredit[ inTrain, ]
  GermanCreditTest  <- GermanCredit[-inTrain, ]

  set.seed(10847 + i)
  ### run RF model with tuneLength and ntree set to small values so this script can be re-run relatively quickly 
  ### I observed the same overall trend of negative correlation when tuneLength=30 and ntree = 1001, so this isn't the problem.

  credit.rf <- train( Class ~ . , data = GermanCreditTrain, trControl=fit_control , model="rf", tuneLength=3, ntree = 11)

  ### get accuracy on test set:
  test.rf <- predict( credit.rf, GermanCreditTest)
  test_confusionMatrix <- confusionMatrix( test.rf , GermanCreditTest$Class )

  result <- data.frame( matrix(c( i , credit.rf$bestTune[[1]] , credit.rf$results$Accuracy[which(credit.rf$results$mtry==credit.rf$bestTune[[1]])] , test_confusionMatrix$overall[[1]] ,  test_confusionMatrix$overall[[2]]  ), nrow=1, ncol=5) )

  colnames(result) <- c("Rep" , "mtry" , "train_accuracy" , "test_accuracy" , "Kappa")
  out <- rbind( out, result)

}


plot( out$train_accuracy , out$test_accuracy , pch = 16 , xlab="Training set leave-one-out CV accuracy" , ylab="Test set accuracy")


cor.test( out$train_accuracy , out$test_accuracy )

# Pearson's product-moment correlation
# 
# data:  out$train_accuracy and out$test_accuracy
# t = -5.128, df = 98, p-value = 1.476e-06
# alternative hypothesis: true correlation is not equal to 0
# 95 percent confidence interval:
# -0.601990 -0.289714
# sample estimates:
# cor 
# -0.4599582 

EDIT: I think I understand the issue now, see my answer below.
 A: Suppose you have dogs and cats. You have a feature "barks" which predicts dogs with 100% accuracy. However 50% are missing. In any sample the more "barks" you have, the more dogs you can find. 
The number of barks in train is 100% inversely correlated with the number of barks in your test sample. Therefore the scores on train and test are also inversely correlated.
Whilst this is an extreme case the same argument would apply with any predictive feature. The more it appears in train the less it appears in test. The train and test samples are not independent if they are drawn from the same data. 
A: I ran cross-validation on the training set and compared that accuracy to the accuracy calculated to fitting the test data to the model. This isn't a fair comparison since the model is being assessed on 2 different datasets.
I think the reason for the negative correlation above is that the RF fits some samples better than others. In cases where I got high accuracy with CV I think those samples were probably included in the training set and not in the test set (and vice versa when I calculated higher accuracy in the test set).
