1
$\begingroup$

I am trying to solve a classification model using a bayesian approach. In particular I am using as reference the following work: Modeling Analysts’ Recommendations via Bayesian Machine Learning

The supervised learning problem consists in a dataset of analysts reccomendations for a population of stocks in a given period of time. Each row represents a given stock in a given day, while each column (that we call feature) represent a given Broker. The table is populated with the broker reccomendations for a given stock in a given day grouped in 4 classes:

  • 0 if the reccomendation is missing
  • 1 hold
  • 2 sell
  • 3 buy

The dataset contains also a label vector composed of 3 classes:

  • 0 if the stock return in the following 2 months has been lower than 5%
  • 2 if the stock return in the following 2 months has been higher than 5%
  • 1 otherwise

Since most of 99% of the dataset is composed of missing data (zeros), this classification problem is suitable for a bayesian approach.

Let's reproduce a simplified version of the data:

import pandas as pd
import numpy as np
index = pd.Index(range(100))
columns = ['B1', 'B2', 'B3', 'B4', 'B5']
recommendations = pd.DataFrame(np.random.randint(0, 4, size=(len(index), len(columns))), index=index, 
                               columns=columns)
label = pd.DataFrame(np.random.randint(0,3, size=(len(index), 1)), index=index, columns=['label'])
df = recommendations.join(label)
df

     B1  B2  B3  B4  B5  label
0    0   0   2   1   3      0
1    3   2   0   2   2      2
2    1   3   3   3   3      2
3    0   1   3   3   2      1
4    1   1   1   0   0      0
..  ..  ..  ..  ..  ..    ...
95   3   2   0   2   3      1
96   2   0   2   0   1      2
97   3   0   3   1   0      0
98   1   2   3   0   3      0
99   3   2   1   0   0      1

We assume independence both among the rows and columns. The probabilistic setting is the following: we assume that the label probabilities are $k={k_0, k_1, k_2}$ and have a three-dimensional Dirichlet distribution with parameter $V=(v_0, v_1, v_2)$. For each broker we have 4 different probabilities conditioned on the 3 label classes: $B_k|T=t$, $t=0,1,2$, $Pr(B_k|T=t)=\pi_{tk}$ (for each broker, conditional to each truth we have 4 probabilities) and assume for each Bk a four-dimensional Dirichlet distribution with parameters {$\alpha_0, \alpha_1, \alpha_2, \alpha_3$}. In our simplified framework we have 5(broker)x4(classes recommendation)x3(classes label)=60 parameters. We set all the parameters of the model equal to 1.

Now, I want to divide the database into a train set in which I observe both features and label, and a test set in which I observe the broker reccomendadion and I want to perform a prediction of the label class.

train = df[df.index < 70]
test = df[df.index >=70]

The task I am struggling to perform is to build an appropriate model for this framework using the library bayespy. We can represent the problem as follows:

enter image description here

Where A0 contains all the 3x5 four-dimensional hyperparameters of the Dirichlet distributions for the broker forecasts, V the three-dimensional hyperparameters of the Dirichlet distribution for the label classes, Pi are the 3x5 4D-Dirichlet random variables for the broker recommendations, K is the 3D-Dirichlet random variable for the label classes, $T=t_j$ are the categorical label classes observed during trining but that must be inferred during testing and $B_{kj}$ are the categorical broker reccomentdations observed both during trining and test.

I built the following model:

import numpy as np
from bayespy import nodes
from bayespy.nodes import Mixture, Categorical

K = nodes.Dirichlet(np.ones(3), name='K')
T = nodes.Categorical(K, plates=(len(train),))

Pi = nodes.Dirichlet(np.ones(4), plates=(3,5), name='Pi') # four classes, one rv for each of the 3 truth, 5 brokers

B = Mixture(T, Categorical, Pi)

But here I get the error:

No automatic conversion from CategoricalMoments to CategoricalMoments with different number of categories

I would continue as follows:

from bayespy.inference import VB
Q = VB(T, K, B, Pi)

Pi.initialize_from_prior()
K.initialize_from_prior()

B.observe(train[train.columns[:-1]])
T.observe(train.label)

Q.update(repeat=1000)

I know I am doing something wrong in the construction of the model, but after several trials I cannot reach an appropriate setting. Moreover, I searched some examples in the web about the construction of categorical mixture models, but I did not find anything similar to this.

Any help would be appreciated.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.