Configuring Knowledge and Knower:  Evaluating Educational Usability

For auditory learners, these videos provide an audio version of the text.

"…the usability evaluation stage is an effective method by which a software development team can establish the positive and negative aspects of its prototype releases, and make the required changes before the system is delivered to the target users"  (Issa & Isaias, 2015, p. 29).

“…the design and production of a new entity…amounts to a process of configuring its user, where 'configuring' includes defining the identity of putative users, and setting constraints upon their likely future actions” (Woolgar, 1990).

Usability is a complex term which refers to the evaluation of the relationships between the user, the system (technology), and the task or goal to be accomplished. The relationship between the user and the system can be evaluated by examining mechanisms such as learnability, flexibility, robustness, efficiency, memorability, errors, and satisfaction (Issa and Isaias, 20015, p. 33). The success or failure of the relationship can also be assessed through an evaluation of utility, (whether the goal or task is successfully accomplished). 

For auditory learners, these videos provide an audio version of the text.

For auditory learners, these videos provide an audio version of the text.

While this definition is useful for evaluating Human Computer Interaction (HCI) from a marketing perspective, its usefulness for educational purposes is limited.  For instance, while Issa and Isaias' (2015) discussion of usability focuses on the relationship between the user and technology, a second relationship needs to be examined in the case of education.  The main emphasis in determining educational usability would need to include the relationship between the user and knowledge, in addition to the relationship between the system and knowledge. Thus, educational usability might be defined as the degree to which technology enhances or facilitates the relationship between the user and his/her knowledge attainment (See Figure 1). The assessment of the relationship between the user and the system is in alignment with Issa and Isaias’ (2015) discussion of usability.  For the relationship between the user and knowledge, I am proposing an evaluation that is guided by Bloom’s taxonomy.  The evaluation of how knowledge is mediated through technology would include assessing the accuracy, intelligibility and consistency in how knowledge is represented.

While this approach to educational usability has some merit for evaluating educational tools, Woolgar (1990) cautions us to critically consider the bi-directionality of these relationships.  Thus, we must consider not only how users access knowledge though systems, but also how systems configure the user.  The Cambridge Dictionary defines configuring as “to arrange or make changes to a computer system, a piece of computer equipment or software, etc. to make it able to do a particular task or work in a particular way.” While the focus here is on how the machine is configured, for Woolgar (1990), it is the user who is being made to do a particular task or work in a particular way. In one example, Woolgar (1990) discusses the artificiality of the user trials (p. 85-6). 

For auditory learners, these videos provide an audio version of the text.

For auditory learners, these videos provide an audio version of the text.

Although the trial was meant to uncover how a user would interact with the machine, the observers interjected and intervened during their observations, sometimes configuring (or reminding) the participant to align their actions with the expectations of what a ‘user’ would do.  In another example, Woolgar (1990) illustrates how the focus of usability trials shifted. What was initially evaluated was the users’ ability to interpret instructions in order to connect a printer to the computer.  However, when it was discovered that the computer could not be connected because of an incorrect plug, the evaluation shifted to the machine, which was then framed as deviant.  Here we can see how usability trials, although framed as tests of how users configure systems, instead become about how users are constrained and configured by the systems. 

The differences between Woolgar’s (1990) discussion of usability and Issa and Isaias’ (2015) are stark.  While the first takes a post-structuralist approach, the second takes a much more pragmatic approach.  Issa and Isaias (2015) see usability studies as effective because they believe that there are identifiable users who can be understood to represent the end-user, and that these users (or at least the users’ feedback) can be used to configure the machine.  However, Woolgar (1990) argues that the user must be understood as being configured and constrained by the machine.  While from a business or marketing perspective, Issa and Isaias’ (2015) approach may have some value, from an educational perspective, Woolgar’s approach suggests a more nuanced approach, which must take into account how technology limits what a learner can know and how they can know it. This is an important reminder that the tools we use, while inanimate, are imbued with the assumptions and intentions of humans who created them.  As such, they shape not only the relationship between the machine and the human, but between humans and knowledge as well. 

For auditory learners, these videos provide an audio version of the text.

This image shows a Venn diagram of three overlapping circles which represent the importance of three considerations: User, System, and Knowledge. To the right are suggesting how the intersections might be addressed. The intersection of User and System should include a focus on Learnability, flexibility, robustness, efficiency, memorability, errors, and satisfaction. The intersection between user and knowledge is addressed through the key terms “remember, understand, apply, analyst, evaluate and create. The intersection between knowledge and system is represented as evaluating accuracy, intelligibility, and consistency.

Figure 1: Evaluating Educational Usability (with descriptive text option)

References

Armstrong, P. (2010). Bloom’s Taxonomy. Vanderbilt University Center for Teaching. Retrieved [September 11, 2022] from https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/.

Cambridge Dictionary. (2022). Configuring. In Cambridge Dictionary.  Retrieved [September 11, 2022] from https://dictionary.cambridge.org/dictionary/english/configuring

Issa, T., & Isaias, P. (2015) Usability and human computer interaction (HCI).Links to an external site. In Sustainable Design (pp. 19-35). Springer.

Woolgar, S. (1990). Configuring the user: The case of usability trialsLinks to an external site.. The Sociological Review, 38(1, Suppl.), S58-S99.