Unfortunately the algebraic structure of the normal likelihood does not allow us to separate the $\tau_i$ and $\tau$ in a way that allows for exact conjugate relationships to be used here. The exponential family conjugate relationships are a direct consequence of the sum/product properties of exponentials.. to see the problem look at the log likelihood of the data: $$ \text{LL}(\text{data}) = \text{constant} + \frac{1}{2}\sum_i \log(\tau_i) + \frac{1}{2}\sum_i \tau_i (Y_i - \theta_i)^2. $$ There is no way to combine terms involving $\theta_i$ with the prior for $\theta_i$, $$ \log(p(\theta_i)) = \text{constant} + \frac{1}{2} \log(\tau) + \frac{1}{2} \tau (\theta_i - \mu)^2. $$
To combine these two (and the other distributions involved naturally) and get a posterior distribution as a function of the parameters, we would normally combine the squared sum at the end of each of these through completing the square (see section 3 here). This is not possible here because each of the $\tau_i$ infront of each $(Y_i - \theta_i)^2$ are not the same. By not being able to complete the square we are not able to put the parameters in a normal distribution form, so there is by definition no conjugacy. This is without even investigating the additional levels of depth necessitated by the prior (it's really just one Normal-Gamma prior) on $\tau$ and $\mu$.
Further, this non-conjugacy result would hold if you use the convolution of normals mentioned by SeanEaster due to the way the variances are added, making them impossible to wrangle into a Normal-Gamma posterior.
An alternative approach to solving this problem would be computationally through MCMC, especially Metropolis-Hastings.working with a similar model specification is given by jaradniemi