This Data Hackathon was hosted by UT's MLDS. It is a yearly hackathon where UT students receive a prompt and a data set, and we choose how we want to approach/analyze the problem and the type of analysis/result we want to create.
This was the first data hackathon I had been to, and I was expecting it to be similar to a regular hackathon, but it actually felt quite different. The prompt, the approach, and the presentations were all drastically different than what I had experienced before. For this Hackathon, we received a dataset of Amazon products as well as various reviews and other information about the products. Using this data set, we had to determine if the reviews written about the products were AI generated by LLMs, or if it was a genuine review written by a real human.
I was in a group with three others (and jokingly Jeff Bezos), and we looked at the prompt from a variety of perspectives. We showcased the way we went about addressing the prompt in six different approaches. Approach One: Looking at the statistic that was given to use. As a guideline, we were told in the data that if we needed we could assume that if one of the variables in the dataset for each record (# of Helpful votes) was 30% or less, it corresponded to fraudulent reviewers. We analyzed this and found this not to be true. Approach Two: We believed that if we found a fake reviewer id, then all of the reviews created by that id would be fake, but this was actually found not to be that useful as AI-generated reviews had on average 2 reviews in comparison to 8 form Human reviewers, which was the opposite of what we expected. Approach Three: Compare the correlation between Reviewer Names having Product Category Keywords. Often times reviews for a product that was fake would have a reviewer name that was correlated to the product name, and we found that roughly 30-40% of results show that product names are correlated to fake names. Fourth Approach: We wanted to look at the correlation between product IDs and whether they were AI-generated, for example, one type of product might have a significant amount of fake reviews. We found that Olay Regenerist Night Recovery Moist Treatment was the product with the most fraudulent reviews, and a lot of skin-care/self-care products seem to have the most fraudulent reviews, so this is something that we kept in mind. Fifth Approach: We believed there would be a significant difference between the length of responses from AI-generated reviews and Human-generated reviews, and we were right. AI-generated reviews were almost half the length of human reviews, and this is something else we kept in mind for our final approach. Sixth/Final Approach: We wanted to create a machine learning model to accurately predict whether a review was AI-generated. We ended up with a weighted average of 98% for our model accuracy, indicating that the model performed incredibly well.
Throughout this project, we worked with Python, SQL, R, Tableau, and Microsoft Access. A comment that we got from the judges that stuck with me is that we had one of the best and most concise explanations for our workflow and an incredible presentation to back it up. I learned a lot from having the opportunity to talk to various professionals in the field, made mistakes and grew, and overall had a lot of fun working with new people.