Compare a panel versus a reduced panel I am facing a sort of situation I never encountered so far.
Basically I would like to compare a set of data from a panel versus a set of data from the same panel but reduced (see picture)
My point is to see whether the mean from the reduced panel is significantly different to the one from original panel.

I am pretty sure I can not rely on paired t-test since the data are not all paired between the two panels.
Also, I do not think I should use unpaired t-test as the observation are not independent since the data in the reduced panel are also in panel 1.
Does anyone know an alternative suitable for this situation?
 A: Disclaimer: I might be wrong here. 
I think the main consideration here is how much you think the panel members 'correlate'. Do you want to consider the variance among them as a measurement error on an underlying 'truth', or do you think that they are completely independent thinkers, or is it somewhere in between? The first two cases are probably too simplistic, but the last one (as is the case with nuance), is trickier. 
First of all, if they are all just measurement errors of the same thing, you can use a simple bootstrapping method (or an unpaired t-test). 
If they are all completely independent thinkers, you can just throw away the measurements on the panel members that are missing in the second test, as there is not much you can derive from the other's values anyway. 
In between, there is some kind of multilevel / hierarchical approach, where you try to estimate how much overlap there is between the different panel members so that you can incorporate the information learned by the panel members that were only in the first panel. 
But then there is also the question on how much variation over time from a panel member on the same topic you are willing to believe. 
No hard answers here, but rather the idea that you need to think about the underlying model that caused this data. 
A: If I understand your question correctly, I would recommend to compare the mean of the individuals in the reduced panel to the mean of the individuals not included in the reduced panel using a t test for independent samples. 
The t test assesses the null hypothesis that two group means are equal, which means that both groups come from the same population. So you do not need to test the reduced panel versus the original panel.
A: You can use a random-effects model in place of a paired t-test.
Say you have two sets of measurements denoted by $i=1$ and $i=2$ one of the population mean $\mu_1$ and one of the population mean $\mu_2$. The measurement consists of points $y_{ij}$, which have errors within participants $\epsilon_j$ and from test to test $\epsilon_{ij}$
$$y_{ij} = \mu_i + \epsilon_j + \epsilon_{ij}$$
it may look as following for 50 subjects when $\mu_1 = 0.1, \mu_2 = -0.1, \epsilon_j \sim N(0,1), \epsilon_{ij} \sim N(0,0.01)$

The points in this example have a lot of scattering due to the error $\epsilon_j$ related with each individual. The estimates of the means for each separate group would involve large errors. However, the estimate in terms of an intercept term plus a difference term might be well estimated.

*

*With a paired t-test you use the differences and compute it as a single sample t-test.

*With a mixed-effects model, you estimate the parameter/slope/coefficient for the difference and estimate its standard deviation/error. This estimate is done with the covariance matrix of the errors (which is itself estimated as well). Because this covariance matrix includes terms for the correlation of the measurements within the same subject, you end up with a small standard error in the effect for the difference between the $\mu_i$.

This mixed-effects model can still be used when you do not have all of the data points paired. (see R code example below) Some aspects:

*

*With the mixed-effects model you can express a t-value based on the estimated effect and its estimated standard error. However, there is no straightforward formulation for the related degrees of freedom and the t-distribution that relates to this t-value. But, for significance testing, you could also use a likelihood ratio test.


*You could also use a paired test with only the paired points available, but the mixed-effects model is doing slightly better because it will be better able to estimate the group for which you have more points.
To see this intuitively, imagine a worst case when you have a lot of points for both groups (and thus the means of both groups can be estimated accurately) but only a small overlap with the pairing between the points (in which case just using the paired points will reduce the precision).
You can see it work as follows: The error for the paired differences is due to the $\epsilon_{ij}$. By using additional information from the un-paired points you can slightly bias the paired points to get a more precise estimate.

set.seed(1)

### settings
ni <- 50     # group size
nr <- 10     # reduced group size
d = 0.2      # difference in group means
sig <- 0.1   # stdev of sampling error 
tau <- 1     # stdev of subjects 

### subject error
err_subj <- rnorm(ni, 0 , tau)

### observations 
###       mean      + subject effect/error + sampling error
y <- c(rep(d/2,ni)  + err_subj             + rnorm(ni,0,sig),
       rep(-d/2,ni) + err_subj             + rnorm(ni,0,sig))
### regressors (group and subject)
xgroup <-  c(rep(1/2,ni), rep(-1/2,ni))
xgroup2 <- c(rep((ni)/(ni+nr),ni), rep(-(nr)/(ni+nr),ni))
xsubj <- c(c(1:ni),c(1:ni)) 

### creat data tables
data <- list(y=y,
             xgroup = as.numeric(xgroup),
             xsubj = as.factor(xsubj))
data2 <- list(y=y[-c(1:(ni-nr))],
             xgroup = as.numeric(xgroup[-c(1:(ni-nr))]),
             xsubj = as.factor(xsubj[-c(1:(ni-nr))]))


### perform regression
mod <- lme4::lmer(y ~ 1 + xgroup +  (1|xsubj),  data = data)
mod2 <- lme4::lmer(y ~ 1 + xgroup +  (1|xsubj),  data = data2)
mod3 <- lm(y  ~ 1 + xgroup, data = data2)

### test statistics summary
summary(mod2)
summary(mod3)

### significance test using chi-squared / likelihood-ratio
drop1(mod2,test="Chisq")
drop1(mod3,test="Chisq")

plot(y[c(1:ni)], ylim = c(-2.3,2.3), 
     xlab = "subject i", ylab = "y")
points(y[c(1:ni)+ni], col  =2)

