If I find that my covariate (reaction time) alters over the length of my experiment (e.g. due to fatigue), can I somehow build that into my model?
So what I am saying is that the effect of my covariate is not constant (between subjects and within subjects).
Your covariate is not only different across subjects, but also gets different during the multiple measurements on the same subject. This has implications on the study design and the analysis method as well. (I'm not sure if you really meant the effect of the covariate in your second sentence.)
Study design: if your multiple measurements on the same subject are of various natures it's important that you run the measurements in various orders so as to be able to separate the effect of increasing fatigue from the effect of the natures of measurement.
Analysis: some statistical software require the dataset of repeated measure ANCOVA be formatted in the wide format, some other software require the long format (example for wide/long, permalink). Only in the long format will you be able to specify multiple RTs per subject. You will need a statistical software that supports the long format for repeated measures ANCOVA.
If you're interested in parsing a measure of fatigue from your RT data for use as a covariate, then I'd suggest computing the slope of RT as a function of time. An additional measure of "noisiness" might be the variance of RT once the effect of time has been removed.