The setting you describe resembles non-stationary bandits, in which the means of the arms vary over time.

The following papers might be of interest:

1. A Linear Bandit for Seasonal Environments
2. Stochastic Multi-Armed-Bandit Problem with Non-stationary Rewards
3. On Slowly-varying Non-stationary Bandits

Here there are also have these lecture notes which seem relevant.

The setting you describe resembles non-stationary bandits. In this setting, the means of the arms are allowed to vary over time.

The following papers might be of interest:

1. A Linear Bandit for Seasonal Environments
2. Stochastic Multi-Armed-Bandit Problem with Non-stationary Rewards
3. On Slowly-varying Non-stationary Bandits

Here there are also have these lecture notes which seem relevant.

The setting you describe resembles non-stationary bandits, in which the means of the arms vary over time.

The following papers might be of interest:

1. A Linear Bandit for Seasonal Environments
2. Stochastic Multi-Armed-Bandit Problem with Non-stationary Rewards
3. On Slowly-varying Non-stationary Bandits

Here you also have these lecture notes which seem relevant.

The setting you describe resembles non-stationary bandits, in which the means of the arms vary over time.

The following papers might be of interest:

1. A Linear Bandit for Seasonal Environments
2. Stochastic Multi-Armed-Bandit Problem with Non-stationary Rewards
3. On Slowly-varying Non-stationary Bandits

Here there are also have these lecture notes which seem relevant.