Code

Our proposed solution combines Ordinal Hyperplane (OHPL) with Long Short Term Memory (LSTM Recurrent Neural Network) and has the potential to out-perform state-of-the-art alternatives in predicting multi-level ordinal classification.

The below code was developed by this group, with the utilization of Dr. Bob Vanderheyden and Dr. Ying Xie solution.

OHLP Code

Imports

Get the Data

Clean the Data

This step cleans up the data by removing special characters as it does not offer any meaningful context to the review. During this step we replaced commonly used word contractions with it’s meaningful counterpart.
Natural Language Processing also takes advantage of stop words ( common words which would appear to be of little value ) and stemming(replacing words with its root words).
For sentiment analysis purposes we will not be removing stop words since it removes word like ‘not’ and can potentially change the meaning of the review.

Review words in data set

This snippet of code is checking the review word count in our Amazon data set and providing a visual representation.

This snippet of code is reading the words counts and providing a word cloud based off the data to visually spot the most common words.

Split Training, Validation, and Testing

This step samples the original data set to create different versions of the X and Y data to train the model

GloVec

To map our list of words from reviews, we use, GloVe word embedding database to provide initial weights for the embedding layer. For the purpose of our project, we are using ‘glove.6B.50d.txt’ that contains 400,000 word vectors.

Tokenize

Tokenization is the process by which big quantity of text is divided into smaller parts called tokens. Converting input text to a stream of token, where each token is a separate word.

Embedd Words in Review

Converting Review Words to Numerical Array and Padding

Converts review words into an integer token and we have padded these sequences to get same length.

Build the Model

We are building a sequential model with embedding that uses 50 length vectors to represent each word using the GloVe embedding matrix as weights. The next layer is the LSTM (an artificial recurrent neural network (RNN) architecture) layer with 100 memory units (smart neurons).

Compile Model

The Model is compiled with OHPL (Ordinal Hyperplane Loss) function. OHPL enables deep learning techniques to be applied to the ordinal classification problem. By minimizing OHPL, a deep neural network learns to map data to an optimal space where the distance between points and their class centroids are minimized. As it minimizes the loss, it is able to maintain nontrivial ordinal relationship among classes. Existing work in the industry is limited to perform binary classification only.

Training the Model

The model is trained in batches of 15 over 50 epochs. As you can see the loss is reducing with each epoch. By the end of 50th epoch loss reduced from 4.0 to .52
Once trained the model is evaluated and scored against training data and prediction as made against test data.

Evaluate the Model

Once trained the model is evaluated and scored against training data, prediction as made against test data.

Mean Absolute Error is calculated to evaluate performance of the model

Confusion Matrix is then created to show the performance of classification matrix

Page updated

Google Sites

Report abuse