Evaluation

Learning Goal

The Art Book’s main target audience is college students in NYC. The learning goal for them is to be able to apply the knowledge about arts with real-world contexts after using the app, thus enhancing their experiences while visiting museums in New York City.

Since we put all these functions in an app, we would primarily rely on the backend data to evaluate learners’ learning performances and to see if they achieve the learning goal we set for them. We would use the data to see if learners have completed watching videos/audios/texts under each topic, how many times and how much time they’ve spent exploring the recommendation of museums nearby. In this way, we can see if they are able to transfer the knowledge they’ve learned in the app into real-world settings. If necessary, we would include quizzes at the end of each topic to assess if learners are managing their knowledge about art. We can also evaluate the transfer of knowledge through the punch card feature under community. The increase in the number of punch times conveys the message that users are actively inputting the knowledge into themselves and are able to transfer it in real-world contexts.

Another method is to use questionnaires to compare learners’ past and present behaviors. For example, we would ask users how often would they go to museums before/after using the app, and do they think that their museum experiences are becoming more meaningful with prior knowledge about art. If there shows a prominent difference between their behaviors, this means our app’s learning goal is being achieved.

Usability of the app

To evaluate the usability of the app, we will conduct user testing during the design process. The purpose of this is to gather information on how potential users think and navigate the app, so we can improve the app based on their reactions and feedback before it launches. To prepare for the testing, we will also establish success metrics - what does it look like if the user can accomplish the task? During the user testing, the evaluating method we will incorporate is mainly behavioral, in which we will observe the user as he/she is using the app. Notes will be taken along the process to indicate any frustrations, hesitations, or any other reactions.

Another evaluating method that will be helpful is self-report. We will conduct surveys and user interviews to find out users’ satisfaction with the app, getting feedback on their user experiences.

We believe the usability of the app should not be evaluated as a one-time thing, instead, we think it should be evaluated regularly even after the launch of the product. By doing this, designers will be able to get feedback from users and continue to work on improving the app.

Quality of the implementation

In order to evaluate the success of the implementation process, we can conduct surveys consisting of open-ended questions to collect users', investors' and museum partners' feedback. Questions may include 1) whether the initial tutorials are clear and helpful for users to understand the functions of the app 2) whether there are difficulties and confusion on certain features that the app tutorial videos aren't covered 3) whether the app helps the users to achieve their goals and if they would recommend it to others 4) Finally, other comments and suggestions that users would love to give. These survey questions could be automatically sent to users through the app after the program is launched for a certain period of time.