Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I have a question about the AR(1) model. Expressed mathematically as:

$$ Z_{t} = \rho Z_{t-1} + \epsilon_{t}, t=1,..,T$$ $$ \epsilon_{t} \sim iid \ N(0,1) $$

My question is about the "transformation group" method of creating non-informative priors, which I believe initially was suggested by Edwin Jaynes (and is discussed in Chapter 12 of his book Probability Theory: The Logic of Science).

One possible suggestion for a transformation group is to consider "reversing" the time series and then rescaling. Thus my transformation group is the following:

$$\rho^{(1)} = \rho^{-1}$$ $$Z_{t}^{(1)} = \rho^{(1)}Z_{T-t+1}$$

Using the original AR distribution, you can show that this transformation basically just "shuffles" the $\epsilon_{t}$ terms, which by definition of the model are exchangeable. So, estimating $\rho$ using $Z_{t}$ is equivalent to estimating $\rho^{(1)}$ using $Z_{t}^{(1)}$ (i.e., the joint distribution of the noise is the same in both cases). Thus the prior for $\rho^{(1)}$ must be the probability transform of the prior for $\rho$. Or, in mathematical terms, the prior must satisfy the following functional equation:

$$f(\rho)=|{\frac{\partial \rho^{-1}}{\partial \rho}}| f(\rho^{-1})=\rho^{-2}f(\rho^{-1})$$

Unfortunately this does not describe a unique function. In fact, any function with the following form will satisfy the above functional equation:

$$ f(\rho) = (constant) \times \begin{bmatrix} \ \rho^{2b} (1-\rho^{2})^{a} & |\rho|<1 \\ \ \rho^{-2(b+a+1)} (\rho^{2}-1)^{a} & |\rho|>1 \end{bmatrix}.$$

For $a > -1$ and $b>-\frac{1}{2}$ this distribution is proper, with the normalizing constant being the reciprocal of $2\beta(b+\frac{1}{2},a+1)$ where $\beta(a,b)$ is the "beta integral". Note that this class includes the "symmetric reference prior" recommended in Berger, J. O. and Yang, R. (1994). Noninformative priors and Bayesian testing for the AR(1) model. Econometric Theory 10 461–482.

Usually specifying a transformation group makes the solution unique, so I am perplexed as to how this group of transformations does not produce a unique solution. Have I done something wrong in the process of creating the transformation group?

Please let me know if I need to explain my question better. It is quite a doozy!

UPDATE:Perhaps there is no transformation group which uniquely determines the prior in this case?

share|improve this question
    
After a bit of maths, the class of distributions above all give $p(|\rho|<1)=\frac{1}{2}$ so that can't be used as a criterion for making the solution unique. –  probabilityislogic Jan 19 '11 at 3:03
    
Clearly $\Pr(|\rho| \lt 1) = 1/2$ by symmetry: no calculations are needed. –  whuber Jan 19 '11 at 4:19
add comment

1 Answer 1

up vote 4 down vote accepted

This transformation group is discrete and finite: it contains exactly two elements, the identity and inverting $\rho$. It's simply not big enough to determine a prior. In fact, you can choose any measurable function $f$ defined on $[-1,1]$ provided (a) it is integrable and (b) $\rho^{-2}f(1/\rho)d\rho$ is integrable on $[1, \infty]$. The latter restricts $f$ only in a neighborhood of $0$.

BTW, for this model to be practical you need to introduce a nuisance parameter $\sigma$: $\epsilon_t \sim N(0, \sigma^2)$.

share|improve this answer
    
I could also introduce a location parameter $\mu$ So that $Z_{t}$ effectively gets replaced with $\frac{X_{t}-\mu}{\sigma}$, but then the prior would just get multiplied by $\frac{1}{\sigma}$, so the example I have given doesn't make the task any easier –  probabilityislogic Jan 19 '11 at 5:16
    
@probabilityislogic Yes, you're right that leaving out these parameters does not simplify the problem. –  whuber Jan 19 '11 at 6:08
    
I'm thinking from the lack of articles on it, perhaps no transformation group big enough exist? this group was the only one I could think of which left the problem unchanged to someone who was "completely ignorant". I may update my question to reflect this –  probabilityislogic Jan 19 '11 at 8:25
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.