Amal Khan

Predicting book's literary book success based off specific literature features.

PHASE II

Exploratory Data Analysis:

1. Import and combine json files: goodreads_books_json, goodreads_genres_json, and goodreads_authors_json.

2. Report essential findings, trends, and patterns:

• Goodreads books dataset contains detailed specific information on each book: titles of the book, author's name, books' edition information, ID's of books in the series if the book is part of a series, publishing company, publishing year and date, number of pages of the books, average rating of the booking, text reviews count, description of the book, and language.

• Extracted fiction and non fiction classifications of books, and extracted information on if the books were part of a series or not.

• We also have information on all the author's books, the authors average rating, and authors ratings counts. (to measure author's popularity)

• Genre information: what categories each book is classified under- non-fiction or fiction.

• Series information: what books are part of a series and which books aren't.

• Book's average rating is not based off books rating count which changes the true the rating of that book

• author's average rating is also not based of authors rating count, which changes the author's true rating

3. Trends found:

• Book's part of a series appear to have higher ratings - (Figure 1)

• Top rated authors are authors who have written book series or have higher historical author popularity- (Figure 2)

• Authors who have written the most books- (Figure 3)

• Fiction books tend to be more successful - (Figure 4)

• Most books are published in October- (Figure 5)

• Positive linear relationships between book's log weighted rating and book's ratings count (Figure 6): the more people rated the book, the more popular and successful the book was.

• Positive linear relationships between book's log weighted rating and book's reviews count (Figure 7): the more people reviewed the book, the more popular and successful the book was.

Bonus features to explore:

d. Does author's historical success and popularity contribute to the author's next books' success?

e. Does author's productivity (in terms of how many books they've written, and how many pages each books is) contribute to the books' success?

PHASE III

Application of Machine Learning Algorithms, Final Results, and Conclusions

Final Accuracy Test results

Methodology

Machine Learning Algorithms selection:

After conducting EDA and looking into which feature variables play a significant role in predicting a book's literary success, I decided to run three different machine learning algorithms to determine which machine learning algorithm would provide the highest accuracy result. Utilizing a classification approach, I chose to run a Decision Tree Classifier, Support Vector Machine, and K-Nearest Neighbor algorithms.

Each machine learning algorithm presents its' own individual advantages. Decision Tree classification are useful for supervised learning , they are easy to interpret, and don't require feature scaling (Galarnyk, "Understanding Decision Trees for Classification in Python" ). Classification Machine Learning algorithms are beneficial when datasets contain multiple classification groups. K-nearest neighbor is nonparametric and doesn't make assumptions based off the data distributions, and it mainly classifies objects by feature similarity (Gahukar, "Classification Algorithms in Machine Learning...").

I divided the classifications of the target variable by five classes: bad, average, above average, good, excellent. My categorical columns were feature variables: book series or not classification and fiction and nonfiction classification. In my initial trial, Decision Tree Classifier proved to provide the highest accuracy results.

Results/Conclusions:

After trial and error of refining my feature variables, vectorizing and transforming variables into numerical values, the Support Vector Machine model provided the highest accuracy result of 70.58%. The feature variables utilized in the machine learning model that provided the highest results were: book's title, book's description, book's ratings counts, books' reviews counts, number of pages per book, book's fiction or nonfiction classification, book's series or not classification, total number of books authors had written. These features play a significant role in predicting how successful a book will be.

Support Vector Machines have the advantage of being able to function effectively in high dimensional spaces and "uses a subset of training points in the decision function so it's memory efficient" (Gahukar, "Classification Algorithms in Machine Learning..."). After my initial trials of running SVM, I tried Hyperparameter tuning to allow the algorithm define the parameters that govern the entire model prior to training the model and thus increase the accuracy result. I tried to limit the number of hyperparameter optimization techniques. Choosing effective hyperparameters allows "efficient search of parameters in space, and ease to manage a large set of experiments for hyperparameter tuning" (Prabhu, "Understanding Hyperparameter techniques and Optimization Techniques").

Limitations:

Throughout the development and ongoing process of my project there were a number of limitations and obstacles that I faced. In the Exploratory Data Analysis portion of the project, I spent a lot of time cleaning the data and extracting the features I wanted to use in the machine learning models. Each file I utilized from USCD took me a significant amount of time to clean out nested lists and dictionaries. For example, in the Goodreads genre file, I had to decide if I wanted to focus on extracting book's genres or fiction/nonfiction classifications, and most books were categorized by a combination of genre and nonfiction/fiction classification, such as "historical fiction" or "young adult fiction". In addition it took time to clean out the nested dictionaries and extract only the fiction and non fiction classifications. Similarly with the book series classification. Initially i wanted to utilize Goodreads book series dataset information but it didn't prove to be very useful so I chose only chose to classify by Goodreads books dataset's "book series column"; if there was a book_id included in that list then the book was classified as part of book series. Another limitation I faced was that because there was a lot of detailed book information and features, and there was 2.3M books in the dataset, there was only so much data I could read in and process in my notebook without it crashing (similarly with running in my machine learning algorithms, there was only so much data I could read in).

In addition to the limited dataset, although it has a lot of various specific information about the books, it didn't have the feature information I wanted to explore for and research. I had to create the feature variables from the dataset provided (log_weighted_rating, series or not, fiction or non Fiction).

In terms of running the machine learning algorithms, I could only use a certain amount of data and even then it took a significant amount of time to split the datasets into training and test sets, to fit the models, and to score the results. As mentioned, I tried Hyperparameter tuning to help improve the accuracy results, but similarly it took too long to run. I also tried limiting the number of classes to 3 instead of 5 when scoring the ratings, but that also didn't improve the accuracy result. Eventually, after limiting the number of feature variables to limit the number of parameters being run, which in turn decreased running time, I was able to improve the accuracy results on all three algorithms.

Further Research:

There is a significant amount of research needed to predict what features attribute to a book's success. It would be essential and crucial to explore the revenue financial aspect of book success: what books prove to be the most profitable? One study, analyzed and tried predicting book's sales prior to publication (Wang, Yucesoy, Varol, "Success in books: predicting book's sales before publication"). More data needs to be collection on book's sales information to be able to conduct further research.

Book's representation in the media would also be interesting to explore and conduct further research on. How many times are people searching for books online, or how much are potential books advertised online to encourage readers to buy them. In addition, it would be interesting to see what books turned into movies and how successful the movies were; historically books that have movies or TV shows made based off of them tend to be successful and popular amongst readers/public (for example Game of Thrones is a popular book series that turned into an even more successful TV series).

As I mentioned, I had considered to explore author's historical popularity and literary success and how it could affect future publications by the same authors. What kinds of books attributed to the author's success? Or what books should authors write in order to be more successful in the future?

Lastly, another topic of interest to see what attributes to a book's success is time and seasonality. Are there certain times of the year that books are more successful? What kinds of books are best published in what times of the year? Are there certain times of the year that books are most profitable? Publishers and authors can use this information to know when to publish books to maximize success and profit, but in order to utilize this information we would need more historical information and data on book's publication times.