1
$\begingroup$

I'm looking to construct a maximum entropy model using general iterative scaling.

In the process, I've come across the SharpEntropy project for .NET and I'm using that as a reference for the implementation I'm building.

That said, the simple example uses the following observations (all nominal values):

Day       Temp.       Moisture  Time of Day   Take Umbrella (outcome)
---       -----       --------  -----------   -----------------------
  1       Warm        Dry                     No_Umbrella
  2       Cold        Dry                     No_Umbrella
  3       Cold        Rainy                   Umbrella
  4       Cold        Dry                     Umbrella
  5       Warm        Dry                     No_Umbrella
  6       Cold        Dry       Early         Umbrella
  7       Cold        Rainy     Early         Umbrella
  8       Cold        Dry       Late          No_Umbrella
  9       Warm        Rainy     Late          No_Umbrella
 10       Warm        Dry       Late          No_Umbrella

Looking through the code, it seems that in order to generate the binary functions (predicates) for the maximum entropy model, it's taking the labels across all the attributes, no matter where the value is found.

Meaning, if Warm was in the Moisture column, then that would be tallied when figuring out the entropy associated with the binary function observation of warm.

This seems very incorrect to me. Granted, for the data above, it happens to work because all of the labels are unique across all of the attributes.

Let's add another feature to the set above, the temperature of the prior day:

Day   Prior Temp.  Temp.  Moisture  Time of Day  Take Umbrella (outcome)
---   -----------  -----  --------  -----------  -----------------------
  1   Warm         Warm   Dry                    No_Umbrella
  2   Cold         Cold   Dry                    No_Umbrella
  3   Cold         Cold   Rainy                  Umbrella
  4   Warm         Cold   Dry                    Umbrella
  5   Warm         Warm   Dry                    No_Umbrella
  6   Cold         Cold   Dry       Early        Umbrella
  7   Cold         Cold   Rainy     Early        Umbrella
  8   Warm         Cold   Dry       Late         No_Umbrella
  9   Cold         Warm   Rainy     Late         No_Umbrella
 10   Warm         Warm   Dry       Late         No_Umbrella

Now what SharpEntropy seems to do is generate the following binary function:

// I wanted to do this in LaTeX, but my LaTeX-fu failed me.
// This would be, f0, for example.
return (priorTemp == "Warm" || priorTemp == "Warm");

This seems wrong to me, and that you really want two binary functions, one for the prior day's temperature:

// f0
return priorTemp == "Warm";

And then one for the current day:

// f1
return temp == "Warm";

That said, is this the proper way to generate the binary functions, that each attribute on an observation is has it's own universe of values, even though they might semantically mean the same thing when considering a larger universe of values (say, temperature in general versus temperature on a given day)?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.