Bayesian approach to infer whether a patient has a particular disease

Question

Suppose for each of $p$ symptoms we have a measurement of the degree of severity from $1$ to $5$. We have a dataset $\{(x_i,y_i)\}_{i=1}^n$, where $x_i=(x_{ij},j=1,\cdots,p)$ are the measured severities of the symptoms of patient $i$, and $y_i=0$ or $1$ indicates whether or note patient $i$ has the disease.

Let $\pi$ be the prior of class $Y_i=1$.

Then the joint probability over the class variable $Y_i$ and observation $X_i$ is $$ P(X_i,Y_i)=\pi^{Y_i}(1-\pi)^{1-Y_i}\prod_{k=0}^1\prod_{j=1}^p\prod_{v=1}^5\theta_{kjv}^{I(Y_i=k)I(X_{ij}=v)} $$

where $\theta_{kjv}$ is the probability that $X_{ij}=v$ given $Y_i=k$.

Now I am asked to take a Bayesian approach to derive the posterior distribution of parameters $\theta$ and $\pi$, given the dataset $D=\{(x_i,y_i)\}_{i=1}^n$, under the assumption that the prior of $\pi$ is $\text{Beta}(1,1)$ (i.e. uniform) and the prior of $\{\theta_{kjv}\}_{v=1}^5$ is $\text{Dirichlet(1,1,1,1,1)}$ (i.e. uniform on the simplex $\sum_v\theta_{kjv}=1$).

Also, assume the parameters are independent.

Could anyone show me how to do this?

My try is as follow:

$$ \begin{align*} f(\pi|D)&=\frac{P(D|\pi)f(\pi)}{P(D)}\\ &\propto P(D|\pi)\\ &=\int_{\theta\in\text{simplex}}P(D,\theta|\pi)d\theta\\ &=\int_{\theta\in\text{simplex}}P(D|\pi,\theta)f(\theta)d\theta\\ &=\mathbb{E}_\theta[P(D|\pi,\theta)] \end{align*} $$

It is the most I can do, how can I move on to have a closed form for $f(\pi|D)$? It is requires to write it in $n_k$ and $m_{kjv}$ where $n_k$ is the number of data in class $k$, and $m_{kjv}$ is the number of patients in class $k$ with severity level $v$ for symptom $j$.

Answer 1

The posteriors for $\pi$ and all $\theta_{kj}$ are conditionally independent given the data. You can use Beta-Binomial conjugacy for the $\pi$ and Dirichlet-Multinomial on each $\theta_{kj}$ vector.

That is $$ \pi \mid \text{data} \sim \text{Beta}\left(1 + \sum_i I(Y_i = 1), 1 + n - \sum_i I(Y_i = 1) \right) \\ \theta_{kj} \mid \text{data} \sim \text{Dirichlet}\left( \left\{ 1 + \sum_i I(Y_i = k) \times I(X_{ij} =v) \right\}_{v=1}^5 \right) \text{for each $j=1,\dots,p$; $k = 0,1$} $$

---

The approach you took ignored the prior. Try and reformulate your approach as $$ \text{Posterior} \propto \text{Likelihood across all observations} \times \text{Prior} $$ and infer a distribution kernel from playing with the algebra. I used the fact that there was independence in the prior for your parameters and that the total likelihood function allowed me to algebraically separate parameters (i.e. other parameters behaved like constants when trying to infer a posterior for some of the parameters).

Answer 2

For the record, I believe that the formulation and solution to this problem correctly belongs in the realm of Bayesian Linear Discriminant Analysis.

To quote a source, 'A Comparison among Classification Accuracy of Neural Network, FLDA and BLDA in P300-Based BCI System':

4. BAYESIAN LINEAR DISCRIMINANT ANALYSIS BLDA can be considered as an extension of Fisher‘s Linear Discriminant Analysis (FLDA). Here, the main problem in FLDA is solved. In BLDA, to prevent overfitting to high dimensional and possibly noisy datasets, the regularization is used and through this analysis the degree of regularization can be estimated automatically and quickly from training data without the need for time consuming cross-validation. Algorithms that are closely related to the method used here are the Bayesian least-squares support vector machine (Van Gestel et al., 2002) and the algorithm for Bayesian non-linear discriminant analysis described by Centeno and Lawrence (2006). BLDA is also closely related to the so-called evidence framework for which detailed accounts are given by MacKay (1992) and Bishop (2006) [5].

Here is another example of work in this area, 'Epileptic Seizure Detection Using Lacunarity and Bayesian Linear Discriminant Analysis in Intracranial EEG. I quote the abstract from this BLDA application as it directly relates to medical diagnostics:

Automatic seizure detection plays an important role in long-term epilepsy monitoring, and seizure detection algorithms have been intensively investigated over the years. This paper proposes an algorithm for seizure detection using lacunarity and Bayesian linear discriminant analysis (BLDA) in long-term intracranial EEG. Lacunarity is a measure of heterogeneity for a fractal. The proposed method first conducts wavelet decomposition on EEGs with five scales, and selects the wavelet coefficients at scale 3, 4, and 5 for subsequent processing. Effective features including lacunarity and fluctuation index are extracted from the selected three scales, and then sent into the BLDA for training and classification. Finally, postprocessing which includes smoothing, threshold judgment, multichannels integration, and collar technique is applied to obtain high sensitivity and low false detection rate. The proposed algorithm is evaluated on 289.14 h intracranial EEG data from 21-patient Freiburg dataset and yields a sensitivity of 96.25% and a false detection rate of 0.13/h with a mean delay time of 13.8 s.

More theoretical background includes, for example, the following works: https://www4.stat.ncsu.edu/~sghosal/papers/QDA.pdf and http://www2.ece.ohio-state.edu/~aleix/pami07.pdf