Creating a better prior based on past observations

Question

Based on this post,

In plain english, update a prior in bayesian inference means that you start with some guesses about the probability of an event occuring (prior probability), then you observe what happens (likelihood), and depending on what happened you update your initial guess. Once updated, your prior probability is called posterior probability.

Of course, now you can:

• stop with your posterior probability;
• use you posterior probability as a new prior, and update such a probability to obtain a new posterior by observing more evidence (i.e. data).

To the best of my understanding, this describes the following scenario.

1. There existed some distribution $P[\theta]$, from which exactly one set of parameters $\theta_1$ was randomly generated. The parameter $\theta_1$ is unknown to the experimenter.
2. The parameter $\theta_1$ was then used to generate some observed data $D_1, D_2$ and $D_3$ according to some model $P[D|\theta]$.
3. One can make a better guess at the value of $\theta_1$ than the prior $P[\theta]$ by incorporating the data, for example by computing the posterior $P[\theta_1 | D_1]$, and then using posterior as a prior two more times to also incorporate $D_2$ and $D_3$.

I am interested in a slightly more complicated scenario

1. From $P[\theta]$ two sets of parameters $\theta_1$ and $\theta_2$ are generated. These can be thought of as two different experiments, where former is done earlier than the latter.
2. Each set of parameters is used to generate their own data $D_1$ and $D_2$ correspondingly.
3. Initially experimenter only has access to a not-so-good prior $\tilde P[\theta]$ which they seek to improve

Question: Can the data from the first experiment be used to get a better prior distribution than $\tilde P[\theta]$ for use in the second experiment? If yes how?

Naively, I had thought that the posterior

$$P[\theta_1 | D_1] = \frac{P[D_1 | \theta_1] \tilde P[\theta_1]}{\sum_{\theta_1} P[D_1 | \theta_1] \tilde P[\theta_1]}$$

from the first experiment can be used as prior to the second experiment, but that is surely wrong. For example, if data $D_1$ is so crisp and clear that it tells us exactly what $\theta_1$ is, then our posterior would be a delta function $\delta(\theta = \theta_1)$. That is a bad prior for future experiments, because future values of $\theta$ may be different from $\theta_1$.

In general, I am interested in the formal procedure of constructing a new for future use using data from several past experiments.