I have a data frame like this :

dictt = {'country_cd': ['SE', 'IL', 'PL'],'timezone':['Euro/Stockho','Asia/Jerusalem','Euro/Warsaw'] , 'datetime':['8/10/2017 19:15', '8/10/2017 2:15', '8/10/2016 10:15'],
         'x':[1, 73, 248], 'y':[0, 48, 144], 'z':[0, 5,30], 'm':[0,6,40], 'response':['ok', 'cacel', 'suspend'] }

And I want to do multi-class classification over this(I have nearly 450000 observations).

The accuracy I got so far is around 56% which is not good, so I am asking if you guys have any idea what can I do to improve the performance:

so far I have done this:

1.country_cd have 85 unique nominal data, so I used dummy version of them (categorical to numeric)

  1. time zone I split the data (continent/country) the continent is separated from the country. like the previous feature I again got the dummy of that feature

  2. datetime I split the year and month, and created two different feature(I could not see any important correlation between month and response).

4.The other 4 features that I named x y z m I dont know what they are stands for(imagine you can not get the business behind them). in these 4 features, the sample presented in the dictt above is (min, mean, max) of each feature(just for getting an idea of how the distribution of the numbers is). for the numerical features, I applied standardization in sklearn.

  1. null value is available in the `country_sd only 100 rows which I filled them with Mod.

and I did labelencoding in the response feature having 4 different values(0,1,2,3).

I tried different approaches: MLP, LogisticRegression, Knearestneighbor ... but all the accuracy is around 55%.

I was wondering if someone has any idea I can apply and improve the model. (overal with all pd.dummied I did over the categorical features I came up with 220 features).

  • 1
    $\begingroup$ What hyperparameters are you searching over? It could also just be the case that there is no strong relationship between your measured features and the outcome $\endgroup$ Oct 23 '19 at 4:53
  • $\begingroup$ @DemetriPananos Thank you so much for your response. Yea that might be the case that there is no strong relationship between response and the independent variables. I am looking for an idea in case I can improve this model without adding extra features. I am also interested in learning if I have missed something to apply in case of the steps of preprocessing on the data before feeding to the model. $\endgroup$
    – sariii
    Oct 23 '19 at 14:46

It seems that your data have panel form. There are two dimensions: Country and Time.

It has been my experience when running machine learning methods (like neural networks, random forests, boosted trees or SVM) on data with the time dimension that it helps to compress time into many summary features. Almost always. For example, you could calculate running minima, maxima, mean, volatility, strength of trend, strength of reversion and whatever else makes sense from the domain point of view. As much as I like random forests and such, they are not capable of figuring out such high-dimensional (spread in time) transformations on their own. Domain knowledge helps, typically.

You have much data. Sure, logistic regression can be tried. But I would not be surprised if a more complex and flexible machine learning method would work better in this situation. To avoid overfitting you can always employ something like cross-validaton and/or early stopping.

Hope I have partially answered your question. If I have misunderstood your problem, please let me know.

  • $\begingroup$ Thank you so much for your response. Actually there are four more features x y z ... which I do not know what they are. Actually I have to solve this without knowing the business behind the data. I was wondering if I am missing something in which can help to get better performance. $\endgroup$
    – sariii
    Oct 23 '19 at 14:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.