0
$\begingroup$

I'm trying to predict the primary crime type on a given location using the Chicago crime dataset.

Stripping out all the provided features to just:

  • Location Description Encoded (The location information like street, house, apartment - label encoded)
  • Beat (An int representing the police patrol route)

And only trying to predict from the types, label encoded:

  • Narcotics
  • Battery

Yields the highest score against the predictions of about 75% using a DecisionTreeClassifier.

What I did to try and improve on this score, was to include crime cluster information.

For each community area in Chicago (there are 77), I divided each into 10 clusters (using K-Means). Then, for each cluster - I got a percentage for each of my target feature crimes out of all the crimes in the cluster e.g.

Cluster has 100 crimes

BATTERY_CLUSTER_PERCENT = 0.25
THEFT_CLUSTER_PERCENT = 0.15
NARCOTICS_CLUSTER_PERCENT = 0.05
OTHER_CLUSTER_PERCENT = 0.55

And I assigned each of these values to every row for that cluster.

When I went back and tried my model with these features, the score appears to have actually dropped by a percent.

This is my model without the cluster information

enter image description here enter image description here enter image description here

And with the cluster information added in:

print(f'DecisionTreeClassifier Score: {model.score(X_test, y_test)}')
DecisionTreeClassifier Score: 0.747892534416884

enter image description here enter image description here enter image description here

What's interesting is that my F-1 score increased as well as my ROC.

Why might this happen? Is there anything I can do to improve my score or is this a totally wrong approach?

Improve this question
$\endgroup$
7
  • $\begingroup$ It sure looks like the situation where you’ve got the higher AUC also has the higher accuracy. Please explain why you think your model with higher accuracy has lower AUC. $\endgroup$
    – Dave
    Commented Apr 18, 2020 at 21:51
  • $\begingroup$ I'm confused why model.score(X_test, y_test) is outputting (ever so slightly) lower scores, I would have expected some improvement rather than consistently a tiny bit lower... $\endgroup$
    – TomSelleck
    Commented Apr 18, 2020 at 22:01
  • $\begingroup$ I don’t follow what you’re doing. Which numbers confuse you. $\endgroup$
    – Dave
    Commented Apr 18, 2020 at 22:02
  • $\begingroup$ See the values for DecisionTreeClassifier Score: 0.747892534416884? That's the output from model.score(X_test, y_test) - it's nearly the same when including or excluding the cluster information. Is there a reason why the model accuracy wouldn't have increased to 0.80 for example? $\endgroup$
    – TomSelleck
    Commented Apr 18, 2020 at 22:09
  • $\begingroup$ But then what’s the 65% that you mentioned? $\endgroup$
    – Dave
    Commented Apr 18, 2020 at 22:15

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.