1
$\begingroup$

I have in mind a Bayesian Learning model . The idea of the model is described as the following. The probability of an event (A) happening is P. P depends on observables Xand parameters $\beta$. For instance , $P=\frac{exp(\beta X)}{exp(\beta X)+1}$.Each time when the event A happens , the belief about $\beta$ is updated.

My question is : is there a commonly used conjugate distribution for these type of bayesian learnings. Can you give me some references that dealt with similar problems.

Update : The event is multinomial. I understand the Dirichlet distribution is used as the prior . But this problem goes one step further. It's not simply updating the probability of each event , but updating the parameter values of the underlying causes.

The model is the following :

Agents make one decision each period , smoke or not, $d$. Smoking history is summarized using one variable , $A$ . Health is determined by the arrival of several smoking related diseases. Agents have prior about the parameters in the probability function of the arrival of the diseases on the smoking stock. In each period, after seeing diseases arrival, the belief about the parameters are updated.

$\endgroup$
0
$\begingroup$

I'm not sure about your notation, but you should have one $\beta$ vector for each element of $X$. Each $\beta$ vector is the natural parameter of a Multinomial distribution. This model is multinomial logistic regression.

The likelihood induced on the output multinomial distribution when observing the event $A$ is Dirichlet. The likelihood induced on each vector $\beta$ is not Dirichlet anymore and so if you want to do such a Bayesian update to $\beta$, you'll have to make some assumption (e.g., approximate it as Normal).

$\endgroup$
  • $\begingroup$ I only have one $\beta$ to estimate. And I can't do multinomial logisti regression. The reason is I am estimating a dynamic value function and the bayesian updating is part of the dynamic problem. Is there a way to link the $\beta$ to the parameters in the Dirichlet distribution ? $\endgroup$ – Yan Song Oct 15 '14 at 19:50
  • $\begingroup$ @YanSong What you're saying doesn't make any sense. Why don't you write down your model in full? If $A$ is multinomial, I assume your prediction for $A$ is also multinomial with parameters say $\eta$. The loss is the cross entropy. How are you arriving at the $m$-vector $\eta$ given the $n$-vector $X$? Surely $\beta$ is an $m\times n$ matrix with one column vector corresponding to each element of $X$. $\endgroup$ – Neil G Oct 15 '14 at 20:09
  • $\begingroup$ I wrote the model as concisely as possible. I don't agree with you that X has to be multidimensional. $\endgroup$ – Yan Song Oct 15 '14 at 20:19
  • $\begingroup$ @YanSong you said that $X$ represents the "observables". Also if you mean for it to be a dot product between $\beta$ and $X$, you should make that clear. If $X$ is not a vector, but just a scalar, that's fine: $\beta$ will be a vector since $n=1$. $\endgroup$ – Neil G Oct 15 '14 at 20:36
  • $\begingroup$ The main issue is I have to solve the problem in dynamic programming. And in each period of the dynamic programming, I need agents to update their belief on the distribution of $\beta$, not the parameters of the Dirichlet distribution. It's also why I can not run regression. $\endgroup$ – Yan Song Oct 15 '14 at 20:38
0
$\begingroup$

There is no commonly used conjugate distribution for $\beta$ in this problem.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.