1
$\begingroup$

Let me preface this by saying that if someone manages to provide a solution to my problem, I will forever be indebted to them, as this problem has driven me crazy. Let us first assume that the discrete variables $S_1,S_2,...,S_n$ are independent and can be explained by an information matrix, say $n\times k$ matrix $X=[x_0,x_1,...,x_{n-1}]$, where $X\in\mathbb{R}^{n\times k}$. That would imply that their log-likelihood function can be expressed by \begin{eqnarray} L(\mathbf{S_n};\theta)=\log P\left[S_1=s_1,...,S_n=s_n\mid X\right]&=&\log\prod_{t=1}^{n}P\left[S_t=s_t\mid X\right]\\ &=&\sum\limits_{t=1}^{n}\log P\left[S_t=s_t\mid X\right] \end{eqnarray} Now let us assume that a test statistic, say $S_{n,\text{independent}}$ is based on the above log-likelihood function. Therefore, the characteristic function of the statistic $S_{n,\text{independent}}$ w.r.t $X$ can be written as \begin{eqnarray} \phi_{S_{n,\text{independent}}}(u)&=&\mathbb{E}_{X}\left[\exp{\left(iuS_{n,\text{independent}}\right)}\right]\\ &=&\mathbb{E}_{X}\left[\exp{\left(iu\left\{\sum\limits_{t=1}^{n}\log P\left[S_t=s_t\mid X\right]\right\}\right)}\right]\\ &=&\mathbb{E}_{X}\left[\prod\limits_{t=1}^{n}\exp{\left(iu\left\{\log P\left[S_t=s_t\mid X\right]\right\}\right)}\right]\\ \end{eqnarray} and due to independence of $S_1,...,S_n$

\begin{equation} \phi_{S_{n,\text{independent}}}(u)=\prod\limits_{t=1}^{n}\mathbb{E}_{X}\left[\exp{\left(iu\left\{\log P\left[S_t=s_t\mid X\right]\right\}\right)}\right] \end{equation}

Dependent case:

Now consider an identical problem to the one before, where now $S_1,...,S_n$ are dependent. Then from given the chain rule in probability, the log-likelihood function will change to \begin{eqnarray} L(\mathbf{S_n};\theta)=\log P\left[S_1=s_1,...,S_n=s_n\mid X\right]\\ =\log\prod_{t=1}^{n}P\left[S_t=s_t\mid S_1=s_1,...,S_{t-1}=s_{t-1},X\right]\\ =\sum\limits_{t=1}^{n}\log P\left[S_t=s_t\mid S_1=s_1,...,S_{t-1}=s_{t-1} ,X\right] \end{eqnarray} where now \begin{eqnarray} \phi_{S_{n,\text{dependent}}}(u)&=&\mathbb{E}_{X}\left[\exp{\left(iuS_{n,\text{dependent}}\right)}\right]\\ &=&\mathbb{E}_{X}\left[\exp{\left(iu\left\{\sum\limits_{t=1}^{n}\log P\left[S_t=s_t\mid S_1=s_1,...,S_{t-1}=s_{t-1}, X\right]\right\}\right)}\right]\\ &=&\mathbb{E}_{X}\left[\prod\limits_{t=1}^{n}\exp{\left(iu\left\{\log P\left[S_t=s_t\mid X\right]\right\}\right)}\right]\\ \end{eqnarray}. Am I correct that since $S_1,...S_n$ are no longer independent, such decomposition does not exist? \begin{equation} \phi_{S_{n,\text{dependent}}}(u)\neq\prod\limits_{t=1}^{n}\mathbb{E}_{X}\left[\exp{\left(iu\left\{\log P\left[S_t=s_t\mid S_1=s_1,...,S_{t-1}=s_{t-1},X\right]\right\}\right)}\right] \end{equation} Or could we decompose as above due to conditional independence?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.