I am trying to build an LSTM neural network to do sentiment analysis on twitter feeds. The dataset I use contains ~1.5M twitter feeds with either positive or negative sentiment (the tweets were ranked by the emojis they use). Additionally the tweets are made uppercase to simplify learning.
The reason I use LSTM is that I hope it will learn nuances between different words which are possibly misspelled etc., also it simplifies pre-/postprocessing of data when using the classifier.
I've been trying a few different sets of hyperparams but it seems to max out at 60% accuracy, which is not really what I hoped for. In the last iteration I added the third LSTM layer to the network but now training is so slow on my GTX970 GPU so I'm considering aborting since there is not even a hint of improvement even after an hour.
In which ways could I either improve the data, or redesign my network so that it still can understand weird spellings of words but still train at a decent rate? Other more traditional classifiers seems to be able to get 70% to 80% accuracy for the same dataset.
Here is the code
# Training parameters checkpoint_path = './checkpoint' best_checkpoint_path = './best_checkpoint' max_checkpoints = 10 tensorboard_dir = './tensorboard' # Import `data` from file filename = "training.1600000.processed.noemoticon.csv" columns = ['polarity','id','date','query','user','text'] import os, pandas as pd, numpy as np """ 0 - the polarity of the tweet (0 = negative, 2 = neutral, 4 = positive) 1 - the id of the tweet (2087) 2 - the date of the tweet (Sat May 16 23:58:44 UTC 2009) 3 - the query (lyx). If there is no query, then this value is NO_QUERY. 4 - the user that tweeted (robotickilldozr) 5 - the text of the tweet (Lyx is cool) """ df = pd.read_csv(filename, names=columns) df = df[['polarity','text']] # Filter out relevant columns df = df[df.polarity != 2] # Do not consider neutral polarity df = df.sample(frac=1) # Randomize order data = np.array(df.as_matrix()) # A matrix is enough # Preprocess the `data` from tflearn.data_utils import to_categorical, pad_sequences test_ratio = 0.1 n_train = int(len(df)*(1 - test_ratio)) n_test = int(len(df) - n_train) X_train = map(lambda string: map(lambda string2: ord(string2.upper()), string), data[:n_train,1]) X_train = np.array(pad_sequences(X_train, maxlen=140, value=0), dtype=int) X_train = np.reshape(X_train, (n_train, 140, 1)) Y_train = map(lambda x: 0 if x == 0 else 1, data[:n_train,0]) Y_train = to_categorical(Y_train, nb_classes=2) X_test = map(lambda string: map(lambda string2: ord(string2.upper()), string), data[n_train:,1]) X_test = np.array(pad_sequences(X_test, maxlen=140, value=0), dtype=int) X_test = np.reshape(X_test, (n_test, 140, 1)) Y_test = map(lambda x: 0 if x == 0 else 1, data[n_train:,0]) Y_test = to_categorical(Y_test, nb_classes=2) # Build model import tflearn net = tflearn.input_data([None, 140, 1]) net = tflearn.lstm(net, 128, return_seq=True, dropout=0.8) # Borde denna vara 128? net = tflearn.lstm(net, 64, return_seq=True, dropout=0.8) # <= ny net = tflearn.lstm(net, 32, dropout=0.8) # <= viktig net = tflearn.fully_connected(net, 16, activation='softmax') # <= viktig net = tflearn.fully_connected(net, 2, activation='softmax') net = tflearn.regression(net, optimizer='adam', learning_rate=0.01, loss='categorical_crossentropy') # Fit model model = tflearn.DNN(net, tensorboard_verbose=3, checkpoint_path=checkpoint_path, best_checkpoint_path=best_checkpoint_path, max_checkpoints=max_checkpoints, tensorboard_dir=tensorboard_dir) model.fit(X_train, Y_train, validation_set=(X_test, Y_test), show_metric=True, n_epoch=100, batch_size=1024)
And the training set is available here