1
$\begingroup$

I have in mind a Bayesian Learning model . The idea of the model is described as the following. The probability of an event (A) happening is P. P depends on observables Xand parameters $\beta$. For instance , $P=\frac{exp(\beta X)}{exp(\beta X)+1}$.Each time when the event A happens , the belief about $\beta$ is updated.

My question is : is there a commonly used conjugate distribution for these type of bayesian learnings. Can you give me some references that dealt with similar problems.

Update : The event is multinomial. I understand the Dirichlet distribution is used as the prior . But this problem goes one step further. It's not simply updating the probability of each event , but updating the parameter values of the underlying causes.

The model is the following :

Agents make one decision each period , smoke or not, $d$. Smoking history is summarized using one variable , $A$ . Health is determined by the arrival of several smoking related diseases. Agents have prior about the parameters in the probability function of the arrival of the diseases on the smoking stock. In each period, after seeing diseases arrival, the belief about the parameters are updated.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
0
$\begingroup$

I'm not sure about your notation, but you should have one $\beta$ vector for each element of $X$. Each $\beta$ vector is the natural parameter of a Multinomial distribution. This model is multinomial logistic regression.

The likelihood induced on the output multinomial distribution when observing the event $A$ is Dirichlet. The likelihood induced on each vector $\beta$ is not Dirichlet anymore and so if you want to do such a Bayesian update to $\beta$, you'll have to make some assumption (e.g., approximate it as Normal).

Improve this answer
$\endgroup$
7
  • $\begingroup$ I only have one $\beta$ to estimate. And I can't do multinomial logisti regression. The reason is I am estimating a dynamic value function and the bayesian updating is part of the dynamic problem. Is there a way to link the $\beta$ to the parameters in the Dirichlet distribution ? $\endgroup$
    – Yan Song
    Commented Oct 15, 2014 at 19:50
  • $\begingroup$ @YanSong What you're saying doesn't make any sense. Why don't you write down your model in full? If $A$ is multinomial, I assume your prediction for $A$ is also multinomial with parameters say $\eta$. The loss is the cross entropy. How are you arriving at the $m$-vector $\eta$ given the $n$-vector $X$? Surely $\beta$ is an $m\times n$ matrix with one column vector corresponding to each element of $X$. $\endgroup$
    – Neil G
    Commented Oct 15, 2014 at 20:09
  • $\begingroup$ I wrote the model as concisely as possible. I don't agree with you that X has to be multidimensional. $\endgroup$
    – Yan Song
    Commented Oct 15, 2014 at 20:19
  • $\begingroup$ @YanSong you said that $X$ represents the "observables". Also if you mean for it to be a dot product between $\beta$ and $X$, you should make that clear. If $X$ is not a vector, but just a scalar, that's fine: $\beta$ will be a vector since $n=1$. $\endgroup$
    – Neil G
    Commented Oct 15, 2014 at 20:36
  • $\begingroup$ The main issue is I have to solve the problem in dynamic programming. And in each period of the dynamic programming, I need agents to update their belief on the distribution of $\beta$, not the parameters of the Dirichlet distribution. It's also why I can not run regression. $\endgroup$
    – Yan Song
    Commented Oct 15, 2014 at 20:38
0
$\begingroup$

There is no commonly used conjugate distribution for $\beta$ in this problem.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.