0
$\begingroup$

I'm trying to fit an ordered logistic regression on Python statsmodel using statsmodels.miscmodels.ordinal_model.OrderedModel. I'm wrapping my head around this but I can't find a solution.

Long story short here is the main code snippets:

  df['Ricontatto'] = pd.Categorical(df['Ricontatto'], ordered=True)
  ordine_desiderato = ['sinistra', 'centro-sinistra', 'centro', 'centro-destra', 
  'destra']
  df['Or Pol'] = pd.Categorical(df['Or Pol'], categories=ordine_desiderato, 
  ordered=True)
   df['Extr conf Isr'] = pd.Categorical(df['Extr conf Isr'], ordered=True)

  #Model
  predittori = ['Mc_Isr', 'Or Pol', 'Extr conf Isr', 'SSP']
  responso= ['Ricontatto']
    
  mod_prob = OrderedModel(df[responso], 
                                df[predittori],
                                distr='probit') 
        
  #Fit
  results = mod_prob.fit(method='bfgs', full_output=True)
  print(results.summary())

The variables: Mc_Isr is continuous, Or Pol is categorical ordered, as Extr conf Isr; SSP is continous.

result: ValueError
ValueError                                Traceback (most recent call last)
Cell In[27], line 5
      2 predittori = ['Mc_Isr', 'Or Pol', 'Extr conf Isr', 'SSP']
      3 responso= ['Ricontatto']
----> 5 mod_prob = OrderedModel(df[responso], 
      6                             df[predittori],
      7                             distr='probit') 
      9 # Fit del modello
     10 results = mod_prob.fit(method='bfgs', full_output=True)

.......

File ~\ANACONDA_LUGLIO_2024\Lib\site-packages\statsmodels\base\data.py:509, in PandasData._convert_endog_exog(self, endog, exog)
    507 exog = exog if exog is None else np.asarray(exog)
    508 if endog.dtype == object or exog is not None and exog.dtype == object:
--> 509     raise ValueError("Pandas data cast to numpy dtype of object. "
    510                      "Check input data with np.asarray(data).")
    511 return super()._convert_endog_exog(endog, exog)

ValueError: Pandas data cast to numpy dtype of object. Check input data with np.asarray(data).

The predictor and the response come from the same dataset. Equal number of rows, no missing values. The same error happens on a dataset with many more predictors, where after some variables pruning it stops fitting the model. I understand that maybe somewhere there could be an hidden intercept or a costant so the model goes awry See here, but that's more a conjecture than a informed guess. I'm no programmer. Any suggestion? Thanks in advance

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Are you sure that all your entries in the data frame are numerical? $\endgroup$
    – Igor F.
    Commented Oct 2 at 21:46
  • $\begingroup$ 'Pol Or' is ordered categorical, but labels were strings, i changed them and now it works. Thanks man $\endgroup$
    – GT87
    Commented Oct 3 at 7:51

1 Answer 1

Reset to default
0
$\begingroup$

Ordered model expects the predictors to be categories with integers as labels

Df['Pol Or'] had to be changed from this:

Categories (5, object): ['sinistra' < 'centro-sinistra' < 'centro' < 'centro-destra' < 'destra']

to this:

df['Or Pol'] = df['Or Pol'].cat.rename_categories({ 'sinistra': 1, 'centro-sinistra': 2, 'centro': 3, 'centro-destra': 4, 'destra': 5 }) Categories (5, int64): [1 < 2 < 3 < 4 < 5]

Now it works. Thank to Igor F. for the useful comment.

Improve this answer
$\endgroup$
6
  • 1
    $\begingroup$ 'Or Pol' is a ordered categorical explanatory variable. After converting to numbers, it is treated as a continuous numerical variable. Alternatively, you could use dummy coding to treat it as (unordered) categorical variable. $\endgroup$
    – Josef
    Commented Oct 3 at 15:49
  • $\begingroup$ Hi Josef, somewhere else I'm doing a backward selection and it gives me the same value error. All vars in that case are categorical, ordered or unordered. All casted with pd.Categorical. I'm wondering, looking at the documentation, if there is somewhere a constant I'm not aware of. maybe you are aware on how to spot implicit constants in the model? $\endgroup$
    – GT87
    Commented Oct 4 at 10:13
  • $\begingroup$ explanatory variables (exog) have to be numeric, i.e. float, unless you use the formula interface. Formula handling converts string or categorical variables to a numeric encoding, by default dummy representation. The numpy/pandas interface in models does not change the users design matrix exog and the user has to provide a valid numeric (float) exog. $\endgroup$
    – Josef
    Commented Oct 4 at 15:14
  • $\begingroup$ So what could be the problem there? $\endgroup$
    – GT87
    Commented Oct 5 at 19:52
  • $\begingroup$ how did you convert categorical to numeric? maybe dummy variable trap. Not enough information here to guess what your did. $\endgroup$
    – Josef
    Commented Oct 6 at 14:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.