I am trying to solve a classification model using a bayesian approach. In particular I am using as reference the following work: Modeling Analysts’ Recommendations via Bayesian Machine Learning

The supervised learning problem consists in a dataset of analysts reccomendations for a population of stocks in a given period of time. Each row represents a given stock in a given day, while each column (that we call feature) represent a given Broker. The table is populated with the broker reccomendations for a given stock in a given day grouped in 4 classes:

  • 0 if the reccomendation is missing
  • 1 hold
  • 2 sell
  • 3 buy

The dataset contains also a label vector composed of 3 classes:

  • 0 if the stock return in the following 2 months has been lower than 5%
  • 2 if the stock return in the following 2 months has been higher than 5%
  • 1 otherwise

Since most of 99% of the dataset is composed of missing data (zeros), this classification problem is suitable for a bayesian approach.

Let's reproduce a simplified version of the data:

import pandas as pd
import numpy as np
index = pd.Index(range(100))
columns = ['B1', 'B2', 'B3', 'B4', 'B5']
recommendations = pd.DataFrame(np.random.randint(0, 4, size=(len(index), len(columns))), index=index, 
label = pd.DataFrame(np.random.randint(0,3, size=(len(index), 1)), index=index, columns=['label'])
df = recommendations.join(label)

     B1  B2  B3  B4  B5  label
0    0   0   2   1   3      0
1    3   2   0   2   2      2
2    1   3   3   3   3      2
3    0   1   3   3   2      1
4    1   1   1   0   0      0
..  ..  ..  ..  ..  ..    ...
95   3   2   0   2   3      1
96   2   0   2   0   1      2
97   3   0   3   1   0      0
98   1   2   3   0   3      0
99   3   2   1   0   0      1

We assume independence both among the rows and columns. The probabilistic setting is the following: we assume that the label probabilities are $k={k_0, k_1, k_2}$ and have a three-dimensional Dirichlet distribution with parameter $V=(v_0, v_1, v_2)$. For each broker we have 4 different probabilities conditioned on the 3 label classes: $B_k|T=t$, $t=0,1,2$, $Pr(B_k|T=t)=\pi_{tk}$ (for each broker, conditional to each truth we have 4 probabilities) and assume for each Bk a four-dimensional Dirichlet distribution with parameters {$\alpha_0, \alpha_1, \alpha_2, \alpha_3$}. In our simplified framework we have 5(broker)x4(classes recommendation)x3(classes label)=60 parameters. We set all the parameters of the model equal to 1.

Now, I want to divide the database into a train set in which I observe both features and label, and a test set in which I observe the broker reccomendadion and I want to perform a prediction of the label class.

train = df[df.index < 70]
test = df[df.index >=70]

The task I am struggling to perform is to build an appropriate model for this framework using the library bayespy. We can represent the problem as follows:

enter image description here

Where A0 contains all the 3x5 four-dimensional hyperparameters of the Dirichlet distributions for the broker forecasts, V the three-dimensional hyperparameters of the Dirichlet distribution for the label classes, Pi are the 3x5 4D-Dirichlet random variables for the broker recommendations, K is the 3D-Dirichlet random variable for the label classes, $T=t_j$ are the categorical label classes observed during trining but that must be inferred during testing and $B_{kj}$ are the categorical broker reccomentdations observed both during trining and test.

I built the following model:

import numpy as np
from bayespy import nodes
from bayespy.nodes import Mixture, Categorical

K = nodes.Dirichlet(np.ones(3), name='K')
T = nodes.Categorical(K, plates=(len(train),))

Pi = nodes.Dirichlet(np.ones(4), plates=(3,5), name='Pi') # four classes, one rv for each of the 3 truth, 5 brokers

B = Mixture(T, Categorical, Pi)

But here I get the error:

No automatic conversion from CategoricalMoments to CategoricalMoments with different number of categories

I would continue as follows:

from bayespy.inference import VB
Q = VB(T, K, B, Pi)




I know I am doing something wrong in the construction of the model, but after several trials I cannot reach an appropriate setting. Moreover, I searched some examples in the web about the construction of categorical mixture models, but I did not find anything similar to this.

Any help would be appreciated.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.