I have never used SPSS, their documentation is very sparse (nowhere does it show which model is being fit) and I don't own a copy to test, but the terminology is sufficiently similar to SAS that I can wager a guess as to what's going on.
In SAS (and possibly in SPSS), random and repeated can be used alongside one another to define similar models using either, or models that are more complex than what several R implementations allow. Very briefly, the linear mixed model fit by SAS is the following:
$$
\textbf{y} = \textbf{X}\beta + \textbf{Z}\gamma + \epsilon
$$
$\textbf{y}$ is your outcome, $\textbf{X}$ the fixed effects design matrix, $\textbf{Z}$ the random effects design. $\beta$ contains the fixed effect parameter estimates, $\gamma$ and $\epsilon$ the random-effect parameters and residual variance. The key point of these last two is the following assumed normal distribution:
$$
\text{E}\begin{bmatrix} \gamma \\ \epsilon\end{bmatrix}=\begin{bmatrix} 0 \\ 0\end{bmatrix},\ \text{Var}\begin{bmatrix} \gamma \\ \epsilon\end{bmatrix}=\begin{bmatrix} \textbf{G} & 0 \\ 0 & \textbf{R}\end{bmatrix}
$$
Specifically, they have mean zero and (co)variances $\textbf{G}$ and $\textbf{R}$. The whole point of random and repeated is to specify the structure of $\textbf{G}$ (via $\textbf{Z}$) and $\textbf{R}$ respectively.
Let's start with a longitudinal example because that's where repeated is most intuitive, but it should become clear that there need not be a time component to such models (remember, it only defines the structure of the residual covariance $\textbf{R}$). Suppose we have a small dataset with the following fixed effect design, where the leading letters identify subjects, and we then have an overall intercept and column for a second timepoint:
$$
\textbf{X}=\begin{matrix} \text{a}\\\text{a}\\\text{b}\\\text{b} \end{matrix}\begin{bmatrix}1 & 0 \\ 1 & 1 \\ 1 & 0 \\ 1 & 1\end{bmatrix}
$$
If you specify that subject is repeated, the residual covariance will have the following blocked structure (one per level of subject, blanks are zero):
$$
\textbf{R}=\begin{bmatrix}\sigma_0^2 & \sigma_c^2 & & \\ \sigma_c^2 & \sigma_1^2 & & \\ & & \sigma_0^2 & \sigma_c^2 \\ & & \sigma_c^2 & \sigma_1^2 \end{bmatrix}
$$
$\textbf{Z}$ and $\textbf{G}$ are undefined and not fit for now. The actual structure and values that these $\sigma^2$ can take can be further specified (e.g. you can set compound symmetry which forces $\sigma_0^2=\sigma_1^2$ or some kind of variance components/banded structure which forces $\sigma_c^2=0$). As an aside, in SAS the only effect of specifying a repeated variable is to set the row order in the $\textbf{R}$ blocks. If you don't, the assumption is that all subjects have the same number of observations (present with missing $\textbf{y}$ if need be) and their order is the same. Failure to do so may lead to e.g. $\sigma_1^2$ being estimated on different timepoints from different subjects instead of always the last.
You can specify the exact same model as random. For this, we make $\textbf{Z}$ equal to $\textbf{X}$ above - you now have to specify an overall and timepoint intercept as random effects in SAS - producing the following (co)variances:
$$
\textbf{G}=\begin{bmatrix}\sigma_0^2 & \sigma_c^2 \\ \sigma_c^2 & \sigma_1^2 \end{bmatrix},\ \textbf{R}=\sigma_r^2
$$
Again, the structure of $\textbf{G}$ can be further specified, but assuming it is the same as before the final response covariance $\textbf{V}=\textbf{ZGZ}^\text{T}+\text{diag}(\textbf{R})$ will be identical to the previous $\textbf{R}$ (where $\textbf{Z}$ and $\textbf{G}$ didn't exist). The actual fitting process is very different and this model is technically quite a bit more complex, but with an unstructured covariance the above $\textbf{G}$ will match within numerical precision to a block of the previous $\textbf{R}$ with $\sigma^2_r$ subtracted from its diagonal.
Returning to the specification of repeated and your limb example: I would assume that the same might happen in SPSS as in SAS. While subject defines the blocking level, repeated matches each observation to a specific row/column in $\textbf{R}$ (just as your random effect design would with $\textbf{G}$). Not doing so might get a left knee matched to a right arm when fitting a single variance parameter. If you don't care about that and you consider limbs exchangeable within subject, you should add a single random intercept at the subject level, because $\textbf{R}$ will still contain one row/column per observation within subject.
To summarize, random and repeated implement two different ways of imposing additional structure on the response covariance. In certain simple cases they can even be made to match (where repeated is more stable & faster so preferred if possible), but random effects are a lot more flexible (e.g. repeated doesn't do slopes, and requires at most one observation per row/column in $\textbf{R}$). For more complex models, such as a mixed effects with additional fixed effect heteroscedasticity, you can even combine them. At least, that's how it works in SAS!
About R
You ask why this is so confusing compared to R implementations. The answer is simple: several of those will let you do either $\textbf{G}$-side (lme4
and many others) or $\textbf{R}$-side effects (e.g. mmrm
), but not both. For example, (time | subject)
will be identical to random time / subject=subject
in SAS if you're using lme4
, or to repeated time / subject=subject
for mmrm
(ignoring covariance structure specification, lme4
only does unstructured I believe).
I recall reading somewhere that nlme
might be sufficiently flexible to have both, but I wouldn't know how & assume it will be roughly equally complicated. edit: as mentioned in the comments, the corresponding nlme::lme
arguments are random
and correlation
. glmmTMB
was raised as another package that can do both.
Addendum
A few additional questions were raised in the comments, or I neglected to answer them from the OP. All of this still assumes SAS and SPSS behave similarly, which I can't test or find in the documentation for the latter.
I need to know exactly which variables should be put as REPEATED and why?
To restate what repeated asks from your model: instead of treating all observations as coming from a single error distribution (independent & identical), subject defines a block of observations where each observation within that block has a different (structured, no longer necessarily all identical) error distribution. If you don't use repeated or random there will only be a single residual variance parameter, my $\sigma_r^2$ above, and it'll apply to all observations in the input. You've just done basic linear regression.
If you specify repeated without a variable the assumption is that all blocks have the same number of observations in the same order. You can think of this as putting 'observation number within subject' as the repeated variable. If you do specify a repeated variable, observations will be matched based on these. Had we provided a dataset like $\textbf{X}$ in below order to the model, swapping rows 3 & 4, and we did not specify the second column which I'll just call 'class' instead of timepoint or limb or.. as a repeated variable:
$$
\textbf{X}=\begin{matrix} \text{a}\\\text{a}\\\text{b}\\\text{b} \end{matrix}\begin{bmatrix}1 & 0 \\ 1 & 1 \\ 1 & 1 \\ 1 & 0\end{bmatrix}
$$
The model will estimate $\sigma_0^2$, the residual variance for the first observation in each subject, from class 0 in subject a and class 1 in subject b. Conversely, $\sigma_1^2$ is estimated from class 1 in subject a and class 0 in subject b. This obviously runs counter to our assumption that the error is still independent & identical within class! Had we provided that repeated variable, the fitting routine would correctly figure out that rows 1 and 4, and rows 2 and 3 are the ones belonging to the same class and match those observations to the corresponding rows/columns of $\textbf{R}$.
if those REPEATED ones should be necessarily nested or not?
I'm not 100% sure what you mean by 'nesting', I understand this as for example limb (arm/leg) being nested in side (left/right) being nested in subject. The model expects you to specify 'subject' as an identifier where different values mean those observations are always assumed to be independent. Specifying nested variables (for subject) is really just a shorthand for expanding dummy variables, as far as I know. The following input data and specifications will lead to the same blocking structure in your model:
Subj Side Limb
A Left Arm
A Left Leg --> repeat LIMB within SIDE(SUBJ)
A Right Arm
A Right Leg
Subj_Side Limb
A_Left Arm
A_Left Leg --> repeat LIMB within SUBJ_SIDE
A_Right Arm
A_Right Leg
Both will result in separate variances for arms/legs, but pool observations from sides and subjects in fitting these.
Another example where you want nested effects is if you have subjects within hospitals, and the subjects start numbering from 1 in each hospital: SUBJECT(HOSPITAL) or subject nested in hospital tells the model that subjects with number 1 aren't actually the same if they also don't share hospital. If all subject numbers are unique across hospitals the non-nested specification gives you the same result.
whether or not those REPEATED ones should be necessarily modeled as random effects or not?
No, most likely not. You can usually turn a repeated model into random as I explained above: repeated is technically simpler but less flexible. Specifying the same design as both repeated and random will lead to confounded variance parameters in your model, which is not what you want if it'll work at all. You can certainly combine repeated and random effects however, I very commonly see models that assume time follows some auto-regressive repeated structure with an additional subject-level random intercept (applying to all timepoints).
SPSS allows a variable (eg. Limb) be designated as REPEATED but not as RANDOM. And in that case, SPSS treats the REPEATED variable as a form of RANDOM effect.
I'm not sure what the limitations are here for SPSS; I would expect repeated not to handle continuous predictors but only intercepts/categorical dummies. Possibly SPSS does not allow you to enter the same effect in both arguments, but you probably don't want that - see above. You are correct that repeated is kind of like random: a model with repeated is the same as a model with a random intercept for every observation within block when using the same subject blocking & specific covariance structures.
Still, one can designate a variable (eg. Limb) as REPEATED, and also as set it as RANDOM at the same time! Or one can designate Limb as REPEATED and RANDOM, and then nest it within SUBJECT!
This runs counter to your previous statement, and goes back to my uncertainty on what you mean by 'nesting': the latter applies to the blocking (i.e. the subject or which observations you consider as independent), not the effects that determine the structure of non-independent observations (repeated/random). I don't want to confuse you more, but it is for example possible to specify a model where you assume limbs within one side follow some covariance structure (the same for each side), and then sides as a whole have another structure within subject. This can only be achieved by using at least one random effect however, just repeated won't get you there.
Another edit: after further clarification by 'nested' you mean "crossed versus nested effects". This is only really relevant for random effects, of which you can have arbitrarily many, and not so much repeated. When you do have multiple random effects you need to think very carefully whether and how they might correlate (going back to the 3-hierarchy example above: does a subject's limb effect depend on its side effect or not?). There can only ever be one level of repeated effect, so that doesn't apply here. Through specification of each covariance structure you can still allow or prevent (certain kinds of) correlation within the effect however, this can go from banded/variance components which has zero correlation, to compound symmetry/auto-regressive which has certain fixed correlations, to unstructured which is 'anything goes' (except negative correlation), and many others.