The basic idea of the prior is, that it describes your knowledge about the weights before you get to know the observations (the data). Thus, a prior with a mean equal to zero and a small prior variance keeps the parameters near zero, i.e. it is used as regularization. Alternatively, a large prior variance means that you don't have much knowledge about the weights and the main source of your information are your data. If there is no prior knowledge about the weights, people use uninformative priors, which don't contain information. This could go as far as using improper priors, which are not even proper densities anymore but can still be computed with as far as the purposes of Bayesian inference are concerned.
Note, that the priors are not the only source of the "noise of the weights", i.e. the variance of the posterior, but it is also very much determined by the data.
Furthermore, what is really useful about priors, is that you can learn them, too. You can make (some of) the hyperparameters, that describe the prior, random variables as well, creating a hierarchical model, and also infer them from the data. That way, you could e.g. even create priors that help you learn sparse models, as in the Relevance Vector Machine.
Thus, while the basic purpose of the prior is to convey prior knowledge about your weights, they can have additional purpose if they are learned, too.