AI to human interaction is an emerging area of focus for AI technologies. Chat bots such as Alexa and Siri are becoming more and more prevalent in our daily lives. Human to computer interaction represents a shift from command line data transfer to a more natural spoken transfer of data from humans to computers. Modern applications of “chat” technology still lack key elements such as context and intonation.
The goal of this project is to build upon the 2019 Alexa Prize Socialbot Grand Challenge dataset. The data set was developed for teams competing in the challenge which is still ongoing. Our team aims to gain a deeper understanding of Natural Language processing (NLP) techniques and explore the applications of Machine Learning (ML) with the Alexa Topical Chat dataset.The human to human dataset consists of over 210,000 utterances (>4.1M words) which makes it the largest publicly available social conversation knowledge dataset. The data was gathered as part of an Amazon funded crowd worker project through Amazon Mechanical Turk services. Workers are commonly referred to as “Turkers”. In the dataset, a pair of turkers were asked to have a naturally coherent and engaging conversation based upon given reading sets. These reading sets were constructed from sources such as Reddit, Wikipedia, and the Washington Post. Various fun facts and articles were selected, cleaned, and compiled to form the basis for the conversational reading sets for the pair of Turkers ranging in topics such as Fashion, Politics, Books, Sports, General Entertainment, Music, Science and Technology, and Movies!
Each conversation required a minimum of 20 “turns” and between which each Turker rated the conversation with sentiment tags such as Angry, Disgusted, Fearful, Sad, Happy, Surprised, and Curious. At the end of the conversation each participant rated the overall quality of the conversation.
The extracted data from .json file format is converted into a tabular form, and loaded into pandas data frame to facilitate application of NLP along with Naïve Bayes and Random Forest Classifier were implemented to predict message sentiments and turkers’ turn rating using the conversations among the two bots with an accuracy of 44% and 50% respectively.