On top of Gavin's answer I'll add that you can use the {mvgam} package if you need to allow each fish to have a potentially different $\rho$ parameter. The package allows all of the same effects that you want to use I'm gam() or bam() so hopefully it won't be too much of a learning curve. You could of course then expand up to more complex latent dynamics should you need them, such as higher order AR terms, vector autoregressions or dynamic factors.

On top of Gavin's answer I'll add that you can use the {mvgam} package if you need to allow each fish to have a potentially different $\rho$ parameter. The package allows all of the same effects that you want to use in gam() or bam() so hopefully it won't be too much of a learning curve. You could of course then expand up to more complex latent dynamics should you need them, such as higher order AR terms with correlated errors, Vector Autoregressions or dynamic factors, which might be helpful for capturing the types of multivariate processes that likely occur in your data.