# Bayespy model construction

I am trying to solve a classification model using a bayesian approach. In particular I am using as reference the following work: Modeling Analysts’ Recommendations via Bayesian Machine Learning

The supervised learning problem consists in a dataset of analysts reccomendations for a population of stocks in a given period of time. Each row represents a given stock in a given day, while each column (that we call feature) represent a given Broker. The table is populated with the broker reccomendations for a given stock in a given day grouped in 4 classes:

• 0 if the reccomendation is missing
• 1 hold
• 2 sell

The dataset contains also a label vector composed of 3 classes:

• 0 if the stock return in the following 2 months has been lower than 5%
• 2 if the stock return in the following 2 months has been higher than 5%
• 1 otherwise

Since most of 99% of the dataset is composed of missing data (zeros), this classification problem is suitable for a bayesian approach.

Let's reproduce a simplified version of the data:

import pandas as pd
import numpy as np
index = pd.Index(range(100))
columns = ['B1', 'B2', 'B3', 'B4', 'B5']
recommendations = pd.DataFrame(np.random.randint(0, 4, size=(len(index), len(columns))), index=index,
columns=columns)
label = pd.DataFrame(np.random.randint(0,3, size=(len(index), 1)), index=index, columns=['label'])
df = recommendations.join(label)
df

B1  B2  B3  B4  B5  label
0    0   0   2   1   3      0
1    3   2   0   2   2      2
2    1   3   3   3   3      2
3    0   1   3   3   2      1
4    1   1   1   0   0      0
..  ..  ..  ..  ..  ..    ...
95   3   2   0   2   3      1
96   2   0   2   0   1      2
97   3   0   3   1   0      0
98   1   2   3   0   3      0
99   3   2   1   0   0      1


We assume independence both among the rows and columns. The probabilistic setting is the following: we assume that the label probabilities are $$k={k_0, k_1, k_2}$$ and have a three-dimensional Dirichlet distribution with parameter $$V=(v_0, v_1, v_2)$$. For each broker we have 4 different probabilities conditioned on the 3 label classes: $$B_k|T=t$$, $$t=0,1,2$$, $$Pr(B_k|T=t)=\pi_{tk}$$ (for each broker, conditional to each truth we have 4 probabilities) and assume for each Bk a four-dimensional Dirichlet distribution with parameters {$$\alpha_0, \alpha_1, \alpha_2, \alpha_3$$}. In our simplified framework we have 5(broker)x4(classes recommendation)x3(classes label)=60 parameters. We set all the parameters of the model equal to 1.

Now, I want to divide the database into a train set in which I observe both features and label, and a test set in which I observe the broker reccomendadion and I want to perform a prediction of the label class.

train = df[df.index < 70]
test = df[df.index >=70]


The task I am struggling to perform is to build an appropriate model for this framework using the library bayespy. We can represent the problem as follows:

Where A0 contains all the 3x5 four-dimensional hyperparameters of the Dirichlet distributions for the broker forecasts, V the three-dimensional hyperparameters of the Dirichlet distribution for the label classes, Pi are the 3x5 4D-Dirichlet random variables for the broker recommendations, K is the 3D-Dirichlet random variable for the label classes, $$T=t_j$$ are the categorical label classes observed during trining but that must be inferred during testing and $$B_{kj}$$ are the categorical broker reccomentdations observed both during trining and test.

I built the following model:

import numpy as np
from bayespy import nodes
from bayespy.nodes import Mixture, Categorical

K = nodes.Dirichlet(np.ones(3), name='K')
T = nodes.Categorical(K, plates=(len(train),))

Pi = nodes.Dirichlet(np.ones(4), plates=(3,5), name='Pi') # four classes, one rv for each of the 3 truth, 5 brokers

B = Mixture(T, Categorical, Pi)


But here I get the error:

No automatic conversion from CategoricalMoments to CategoricalMoments with different number of categories

I would continue as follows:

from bayespy.inference import VB
Q = VB(T, K, B, Pi)

Pi.initialize_from_prior()
K.initialize_from_prior()

B.observe(train[train.columns[:-1]])
T.observe(train.label)

Q.update(repeat=1000)


I know I am doing something wrong in the construction of the model, but after several trials I cannot reach an appropriate setting. Moreover, I searched some examples in the web about the construction of categorical mixture models, but I did not find anything similar to this.

Any help would be appreciated.