1
$\begingroup$

Suppose I have about 1.6 million public filings to the FCC supporting or opposing net neutrality. It would be interesting to assign labels like 'pro' or 'against' and also maybe 'hand-written', 'form-letter', 'bot'. I'm willing to hand label a bunch.

Example:

I support Net Neutrality. I do not support the "Restoring Internet Freedom" proceeding. This proceeding, if it moves forward, endangers access to information and education for millions of people. Anybody that does not support Net Neutrality has lost my vote permanently.

Some entries are highly repetitive, many being form letters. Many contain predictable strings such as "preserve net neutrality" or "I support net neutrality". But, there might be some similar sets of words with opposite meanings, such as 'I support net neutrality; repealing it is a terrible idea.' vs. 'I support repealing net neutrality because it is a terrible idea'.

A 2-step workflow might take care of the easy cases first using simple string matching or regexes. Then, for the hand-written responses try to classify them into 'pro' or 'against'. Any hints on how to do this? Specific libraries or techniques? I'm working in Python, but can do R as well.

If anyone else wants to try their hand, the code here might save you a bit of time.

UPDATE: Getting off topic, but some nice analyses of this data set have been done by Jeffrey Fossett, Chris Sinchok, and Nathaniel Fruchter.

Improve this question
$\endgroup$
4
  • $\begingroup$ This sounds like a candidate for machine learning, classification techniques. $\endgroup$
    – Matthew Gunn
    May 17, 2017 at 7:00
  • $\begingroup$ @matthewgunn, serves me right for asking an overly general question, huh? Can you suggest a specific technique that would work for texts on the same topic using similar language but with opposite positions - "please support" vs. "don't support"?? $\endgroup$
    – cbare
    May 17, 2017 at 17:59
  • 1
    $\begingroup$ Disclaimer: I'm not a machine learning, natural language expert. You could use various two word combinations as a feature. The basic outline of what you want to do is: (1) Map each document to a vector (i.e. extract features) (2) Create a training set (3) Train your algorithm on the training set. Once you have (1) and (2) done, it's actually not that hard to try different classifiers: eg. support vector machine, random forest, naive bayes etc... $\endgroup$
    – Matthew Gunn
    May 17, 2017 at 18:25
  • 1
    $\begingroup$ This link might be useful. And I haven't used tensorflow yet myself, but it seems to be a commonly used tool in the natural language processing, machine learning context. $\endgroup$
    – Matthew Gunn
    May 17, 2017 at 18:26

1 Answer 1

Reset to default
2
$\begingroup$

So at its core this is a binary classification problem but there are many ways you can approach it. Two of which: (i'm ignoring data cleaning and other data preps)

  1. Come up with various features like word count, n-grams, tfidf scores or any other hand-made features. You then feed these into a classifier like xgboost, decision tree classifier (sklearn comes with a number of such classifers)

  2. Use a neural-net based approach. A super simple solution would be taking word2vec represenation of words, encoding them with long short term memories (lstm) and attaching a few dense layers on top. This should give you decent results with much less time invested in feature engineering.

the network architecture in keras would be as simple as: 

embedding_layer = Embedding(nb_words+1,
        300,
        weights=[embedding_matrix],
        input_length=MAX_SEQUENCE_LENGTH,
        trainable=False)



sequence_input = Input(shape=(MAX_SEQUENCE_LENGTH,), dtype='int32')
embedded_sequences = embedding_layer(sequence_input)

lstm_layer = LSTM(200, dropout=0.2, recurrent_dropout=0.2)
encoded_input = lstm_layer(embedded_sequences)

dense_layers= Dense(200, activation='relu')(encoded_input )
dense_layers= Dropout(0.2)(dense_layers)
dense_layers= BatchNormalization()(dense_layers)


preds = Dense(1, activation='sigmoid')(dense_layers)

model = Model(inputs=sequence_input, outputs=preds)
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.