Adventure Biofeedback

Sponsors

The founders for our project are Brian Cohn, an NSF Graduate Fellow in USC's Computer Science PhD program, and Christopher Laine, a Research Assistant Professor at the USC Chan Division of Occupational Science and Occupational Therapy. They started Adventure Biofeedback with the goal to make therapy easier for people with speech disabilities and their speech language pathologists (SLPs). To fulfill this goal, Brian and Chris looked to develop a web application for SLPs that would serve as a clinician dashboard and homework builder for their patients. The homework builder would allow SLPs to easily and effectively create focused homeworks for patient therapy and the dashboard would store information about patient's and their progress.

Brian and Chris also introduced us to Kathy, an actual SLP who had been working with them to develop Adventure Biofeedback. Kathy was a key stakeholder to our project as she would be an actual end user for our product. As a result, our development process largely revolved around Kathy and what she thought would be beneficial for the project.

Read more about Brian and Chris' story with Adventure Biofeedback at their website: https://www.adventurebiofeedback.com/

Development Process

Beginning Development

Our team came to this project with a strong coding foundation. Five out of the six of our team members are Software Engineering majors with our remaining member being an Informatics major still having strong coding experience. Despite this, only a few of our team members were familiar with web development and the beginning of our development process was filled with a lot of knowledge transfers to try and give everyone at least some base with web development.

Focusing on the project itself, Brian and Chris already had a strong foundation built for Adventure Biofeedback that they had been testing with Kathy. This was a very basic version where Brian and Chris would use VanillaJS to manually create homeworks for Kathy to test with patients. When our team came on board to this project, one of the first major design decisions we made was to upgrade the project to use ReactJS.

Utilizing React

ReactJS offered a much more developer friendly environment which would allow us to more easily create the intuitive, user friendly environment Brian and Chris wanted for Adventure Biofeedback. Within the React framework we used NextJS, a production grade framework built on top of NodeJS. We also primarily used MaterialUI, a React UI library that allowed us to make robust, customizable components for our project.

Sprint Cycles

Following the theme of the class, we developed our project using Agile methods. Development on our project was characterized by two week sprints with about four sprints in both the Fall and Winter quarters. Each sprint was characterized by a weekly professor meeting, 2-3 weekly stand ups within our team, and a weekly meeting with our sponsors and Kathy. Throughout every sprint we also constantly updated our Trello board with what was getting done and what needed to get done.

Most of our actual development got done during the weekly stand ups within our team. From there, we would take what we developed to show to our sponsors and Kathy in a weekly meeting. These weekly meetings quickly became one of the most valuable parts of our development process as we were able to meet with an actual user of the product in Kathy. These meetings created a constant feedback loop for our project, helping us to follow the steps of Agile development and make effective progress throughout our project.

Our team also found lots of value in the professor meetings, where we would essentially show Professor Bietz and Yang, our class TA, what we showed to Brian and Kathy. In these meetings we would receive different types of feedback, sometimes less focused on the details project itself but on our team and how we were going about our project at a larger scale. These meetings also helped our team to stay focused on following Agile development at the larger scale, helping ensure that we were working effectively as a team.

Communication with our Sponsors

Fortunately for our team, we had very involved sponsors who were readily available to help us with anything related to the project and Kathy, an actual SLP, who provided valuable feedback throughout the entire project. Meeting with our sponsors was difficult in the beginning of the project as we were only using email to communicate. We quickly realized it would be much more effective to have a Discord server to communicate with our sponsors. Both of our sponsors were very willing to join and with a more effective method of communication we were able to really get into the flow of development for our project. Throughout development, all major decisions came from communication with our sponsors and Kathy. As Kathy is an actual SLP who would use the product, we were always able to go to her for what she wanted to see in the project. From there our team would communicate with Brian on what we thought was the best way to implement Kathy's wants for the project.

Quarter 1

The first quarter of development on our project was characterized by familiarizing ourselves with the project, creating our development environment, and developing a functioning clinician dashboard. As mentioned earlier, once we became familiar with the progress Brian and Chris already had on the project our team made the decision to convert the project to a React project. The members of our team who were familiar with web development knew a lot about the React framework and thought that it would suit the needs of the project very well. Essentially this meant starting over on the project, but since our sponsors wanted an intuitive UI for both the clinician dashboard and homework builder out team thought it would be best to start from the ground up. This was a major design decision, but one that ultimately worked out for the best as it helped our team to better understand what our sponsors wanted and begin the project in a way that made sense to us.

To begin, we created our React environment for the project and a PostgreSQL database backend hosted by Amazon Web Services along with a Prisma Object Relational Mapper to query data from our backend using NextJS API routes. With a backend in place, we made us of Auth0, a third party authentication service, to allow clinicians to make accounts and log in without us having to worry about any authentication or security issues. These features were a major upgrade from what Brian and Chris had in their existing prototype, so while they were not directly planned by our sponsors out teams decision to start form the ground up was immediately bringing value to the project. With a way for clinicians to log in and have information stored in a database we then looked to create the clinician dashboard so that they could create patients and store information about them.

Making the clinician dashboard was relatively simple, as mentioned earlier we used MaterialUI to create simple React components that we were able to populate data with using our backend; more details and a view of the clinician dashboard can be seen in the next section. An established clinician dashboard was also not included in Brian and Chris' initial prototype, but we all agreed it would be beneficial to the project for clinicians to have an overview of all their patients and homeworks. With an established, functioning clinician dashboard, our team was able to deploy our site for clinicians and our team to play with, and we shifted focus to the homework builder, the most important part of the project. In the first quarter, our team laid out a lot of ground work for the project as a whole through creating a functioning backend, clinician dashboard, and deployed website. Our team felt this added lots of value to the project for the sake of our development and having a strong background environment to create the homework builder and for clinicians like Kathy to immediately play around with.

Quarter 2

The second quarter of development was focused entirely on creating the homework builder and implementing recording functionality into the homeworks. The vision for the homework builder was a Google Forms like builder in which clinicians would be able to select preset exercise templates, fill out the template, and add it to the homework page. This would allow clinicians to create customizable homeworks for specific patients or focused on specific speech therapy tasks that could apply to lots of patients. Essentially, the overall goal of the homework builder was to make it as customizable as possible so that clinicians would be able to create whatever they wanted. Each homework would also have recording functionality attached so that clinicians would be able to go into their dashboard and listen to patient recordings from homeworks. This would allow clinicians to ensure patients are doing their homeworks and also let them hear how patients are doing with their exercises.

It was simple for us to make example homeworks like Brian and Chris had been doing, but our team struggled with how to make customizable components that could be added to create a homework page. Fortunately, Brian was able to help us with the design for this and we ended up having the homework builder create a JSON of homework components as clinicians added exercises. This design fit our needs perfectly as we were able to make unique homework components in an inheritance like structure that would allow for different types of exercises to easily be added and all be stored in the same JSON block. By representing homeworks as a JSON block, we were also able to easily store them in our database to be rendered as needed by clinicians or patients. Once a homework was assigned to a patient, a unique url would be created using the clinician and patient's ID so that the patient could view and complete the homework, with information being saved specifically to that patient. With the ability to create homeworks, we now moved to implementing recording functionality into the exercises so that clinicians would be able to hear patients perform the homeworks and evaluate their progress.

The way Brian and Chris had implemented this functionality before was to record in the background and when an exercise checkbox is clicked, capture the last ten seconds of audio and have it processed and posted to our database. While Brian and Chris had working code for this feature, our team struggled to make use of it and thought it would be easier to use React libraries for recording and processing the audio. The MediaRecorderAPI and FFMPEG libraries allowed us to perform all the same tasks Brian and Chris and once implemented we had homeworks that now capture patient audio as they work through the homework. In the second quarter, our team was able to successfully implement the major features required for this project, the ability for clinicians to create customizable projects and capture patient audio as they work on their homework. With all of these features completed, our team is able to deliver a minimum viable prototype for Kathy and other clinicians to make use of.

Challenges

• Everyone on the team not having a strong foundation in web development

  • We often felt we knew how features should be implemented, but didn't have the React foundation to easily write the code

    • Ex: homework builder

  • Knowledge transfers, working in groups as a team, and having sponsors with a strong technical background helped us address this

• Understanding what our sponsors wanted and where to start based on what they already had

  • Building our own version from the ground up helped us address this as we were able to make use of Agile to rapid prototype to gain a better understanding of the project

• Agile development is fast and demands constant attention

  • We always planned to meet multiple times throughout the week, but other classes and responsibilities made this difficult

  • As the project went on we gained a better understanding of how much work would be required to implement different features

What Worked Well

• Working in sub groups

  • Dividing work and splitting into sub groups allowed us to work a lot more efficiently, especially with our limited React knowledge

• Communicating with our sponsors through Discord

  • Using Discord allowed us to communicate much more effectively than email as we could ping Brian if necessary, send messages quicker, and even hop in a voice channel if needed

• Rapid prototyping

  • Rapid prototyping worked very well for us as we had an SLP, Kathy, to provide us with feedback weekly

  • This feedback cycle drove our development forward and helped us to progress in an efficient manner

Documentation

Here we will include pictures of the website and explain what they do and if possible why we chose to do it that way as opposed to another way

Evaluation

Very early into when we started the project, Brian introduced us to Kathy, a speech language pathologist, who was able to attend our weekly meetings with him. In each of these meetings we would show whatever progress we made or talk about what we planned to do and Kathy would respond and tell us what changes could be made. Eventually, when we were able to develop the website enough that it could be used by an SLP to make homework, we had Kathy do a hands on test with it and give us feedback on what she thought of it. After that we made a walkthrough video and instructions and sent it to 4 other SLPs who then tested the website and gave us their feedback. From these user tests we were able to find out a lot of valuable information. Some of the feedback was of how some things didn't feel intuitive enough and we also found out what parts of the website weren't clear for the SLPs. We also got to see what features they thought were nice and that they appreciated seeing. From these user tests, we were able to go back and improve or update our implementation.

Final Product and Deliverables

https://abfhw.netlify.app/

Account to Test:

Email -

Password -

• This is the deployed version of the webpage where clinicians are able to login and assign homeworks to their patients.

Figma Wireframe and Prototype

• Figma wireframe and Prototype is the vision of the application and what we anticipated the webpage would look like.

Sprint Reports

• Sprint Reports are all the reports of the past two quarters where we explain what we did in each two week sprint.

Ethics Review

• The Ethics Review is an analysis on potential ethical risk zones within the Adventure Biofeedback project

Requirement Documentation

• Requirement documentation includes all details of the application from technical aspects to non technical aspects so that it is easier for someone to go back and review if they are lost and or confused