GENAI-POWERED CHATBOT HELPING USERS FIND INFORMATION AND PLACE ORDERS ONLINE

OVERVIEW:

ABOUT COMPANY: 

Waters Corporation (B2B eComm and SaaS) unlocks the potential of science by providing the tools, technology, and insights that enable scientific breakthroughs and allow us to advance human health and well-being. It is a leading provider of analytical instruments, software, and services, launched the Waters AI Advisor (GenAI-powered Chatbot) as part of its efforts to integrate artificial intelligence into its suite of laboratory solutions. The AI Advisor is designed to enhance data analysis and decision-making processes within analytical laboratories, particularly those in industries like pharmaceuticals, biotechnology, and chemicals.

MY ROLE:

Led end-to-end UX Research for the GenAI-powered Chatbot product (0-1 Phase), Carried out secondary research, remotely moderated usability sessions with the target users, Collaborated with UX Designers, Content Designers, Product Owners, Product Managers, etc, analyzed findings, prepared the final report and delivered design recommendations.

PROJECT BACKGROUND:

AI is becoming increasingly important in many scientific fields. Waters Corporation, by offering a dedicated AI tool, positions itself as a leader in the intersection of analytical science and advanced technology, which can give the company a competitive edge in an increasingly data-driven industry. To accomplish this, the business at Waters decided to release a GenAI-powered Chatbot tool - Waters AI Advisor that aims to assist scientists and researchers by providing real-time, data-driven insights that help them make more informed decisions. This could improve research outcomes, facilitate troubleshooting in lab processes and even offer predictive recommendations based on previous results. Through thorough secondary research and impactful usability testing of the tool before release, our team could successfully identify the pain points and expectations of the users from the tool and how we can provide a successful user experience when it comes to using GenAI-Powered Chatbot to find relevant information and buy products online.

OBJECTIVE:

• The aim was to evaluate the user-friendliness and effectiveness of the Waters AI Advisor while also measuring its impact on helping users achieve their desired outcomes.

RESEARCH PROCESS:

1 - Secondary Research 

• I conducted stakeholders interview involving the Product Owner, Product Manager, Subject Matter Experts, UX Designer and others working on the product and had them talk about the background of the project, covering facts, opinions, guesses, etc. I further determined our target participants and defined research goals and objectives.

• I did an extensive competitive analysis to gain insights into what we have/have not to offer that is being offered/not offered by our competitors in the GenAI space. I also tapped into best practices of designing conversational UX including necessary/expected features, compliance features, UI standards, etc along with the latest trends in AI naming.

• I found that although our competitors are leveraging AI/ML integration into their current processes, we are the pioneers in our industry to release a GenAI-Powered Chatbot kind of tool for our users.

2 - Remote Moderated Usability Sessions

• After having all the information about our competitors, I moved forward to prepare a usability testing script, prepared a recruitment screener and recruited potential participants using the UserInterviews platform and conducted 12 usability sessions on Microsoft Teams to understand how users interact with our AI tool and identify the pain points and expectations of the users from the tool. At the end of every task, I asked a Single Ease Question that allowed me to compare which parts of the tool interface were perceived as most problematic.

• I also looked for participants behaviors (click paths and common actions) and thoughts to gauge need gaps, ensuring how we can fulfill them by providing specific features or functionality. I also gave them the opportunity to try their hands on their tool as they liked which in turn helped me to tap into their daily work goals and requirements.

• During each session, I asked users to share their screens to tap into their interaction with the tool and to see how they compared our tool with Microsoft Copilot and similar tools available in the current market. 

• I observed the user's screen throughout, it gave me more clarity about the exact moments of struggle for the user while interacting with our AI tool. At the end, I asked them to fill out a short questionnaire (System Usability Scale) and presented them with a Net Promoter Score question in order to gather substantial quantitative data.

3 - Analysis and Reporting

• I collated all the data in the excel sheet using the voice-in-voice typing tool (for data entry) and analyzed the data to identify the behavior patterns of users, the major pain points, the overall expectations of participants from the tool, the redefining tool along with business growth opportunities.

• On the basis of my analysis, I then moved forward to prepare an engaging final report in which design recommendations were discussed and the team was debriefed over a Microsoft Team call focussed on how we can refine and improve the user's experience on our tool not just for buying but also finding the desired information.

KEY INSIGHTS:

• The progress/loading bar was not visible when the user asked any more questions after the first prompt, so users felt whether the tool stopped working.

• Users felt a lot of steps were involved in order to have the necessary information and expected to find information about the assigned sales representative using the tool.

• Users clicked on the disclaimer and the tool gave an error. Also, the answer seemed complete, but the loading/progress bar was still running, which gave the user the notion that it was still generating an answer.

• Users felt the output was very wordy and could be simplified and the tool needs to be trained to be more conversational and could ask back clarifying questions.

RECOMMENDATIONS:

• Answers need to be polished so that users have more confidence in them and reference links/images need to be made available.

• Do not cluster too much information together and provide answers in the most concise way possible and have a bullet point break wherever necessary.

• The progress/loading bar needs to be made visible with each prompt/question and aim for a minimal number of steps to provide an answer if the user asks any follow-up questions.  

• Show proprietary pricing as per the contract with their companies, as some users have a budget and personalized question recommendations based on their previous searches.

RESEARCH IMPACT:

• My research insights directly went into product development and the product is now Live and being used by users globally (39-point NPS increase, 63% Conversion rate, 81.25 SUS Score, 3000+ active users – All in the first iteration).

• Product strategy pivot backed by research insights.

• ​UX Research​ became a strategic partner of the product teams and it also championed collaboration between research and other functions that are not traditionally impacted by research (Engineering and Data Science).

• Through collaboration, designers were able to quickly relay feedback from research and incorporate it into the prototype - Agile.

KEY LEARNINGS FOR ME:

• Be ready to witness the unthinkable while working on developing GenAI products. (Tool gave very unusual answers during the testing) 

• Conducting Usability Testing on GenAI products (Conducted testing for the first time in GenAI space)

• Creating a pipeline of potential users (A few users didn't show up on the day of the session)

• Stay updated with things happening in the AI world (It helps in generating new ideas)