Overview:
In February 2018, following conversations with a number of potential development partners, the team undertook a deeper discovery process with the following firms (in order of preference):
Nona Creative
Sea Monster
Playlogix
The contract was awarded to Sea Monster for the following key reasons:
Delivery could be achieved within budget
Impressive discovery process
Experience in game design and development.
Positive recommendations from trustworthy sources
Experienced and dynamic CTO
The initial build involved honing the business requirements and the design and ideation for the game world. This was achieved through 5 x 2 week sprints.
Key Resources:
Overview:
The 2019 build consisted of 9 x 1 week sprints focused on the build, and 3 x 1 week sprints focused on User Acceptance Testing and associated bug fixes.
Indium Software, a specialist testing firm, were commissioned by Sea Monster to run thorough testing on the platform across a wide range of device types. The Wavumbuzi team also conducted testing sessions with ten schools across South Africa during class time.
Key Resources:
Overview:
The challenge system development cycle leading up to the Wavumbuzi Rwanda 2021 iteration spanned from May 2020 - March 2021, following the Kenya 2020 Challenge and the subsequent insourcing and onboarding process between January 2020 and April 2020. The development cycle was broken down into several sprints (each focusing on a specific functional area of the system.
Transition to In-House Development Team
An in-house development team was established in Rwanda following a multi-stage organization and project onboarding process. Realm was contracted on a retainer basis to complete technical onboarding. Detailed technical documentation was also captured as part of the onboarding process, which is important for efficient system maintenance and preserving institutional knowledge.
Minutes: Realm Handover Workshop
Efficient Multi-Instance Code Base Configuration
The underlying code base inherited from Sea Monster was not configured for the efficient deployment of multi-country and multi-year iterations, which prohibited efficient scalability. As such, the following adjustments were implemented:
Several steps in the deployment process which previously required manual reconfiguration for each new instance were automated.
Several fixed (hard-coded) elements in the code base were recoded to form “white label” variables, which could be easily reconfigured for new countries and new iterations.
Ionic (the underlying framework for the code) was upgraded to improve security and offer access to the latest version of the framework (required for updated support and features)
Removing lameco bundle & tables
Immediate returns were seen with the seamless deployment of the Rwanda 2021 iteration.
Improved Teacher and Learner UX
Insights from the Kenya 2020 iteration of the challenge indicated that changes to the learner and teacher registration experiences would result in improved conversions rates and more accurate tracking of user engagement. As such, the following adjustments were implemented:
Learner Portal
Portal was configured to handle pre-registration (previously handled by third party forms)
Tutorials on first entry were added to automate learner onboarding
New popup notification triggers were added to improve user flow and story
The collection point for secondary data was relocated to limit progression hurdles
Additional automated emails/SMSs were added to prompt learner engagement
Teacher Portal
All of the above changes to the learner portal were also applied to the teacher portal
In addition, new functionality was also built to facilitate virtual teacher training
Key System Integrations
Without integrations between the challenge platform, selected CRM system, reporting system and key communication tools, significant manual work was required to align data across the respective systems, increasing the risk of data contamination. To reduce this risk, and increase the efficiency of data control, the following integrations were configured:
Improved Peer Review System
Following the Kenya 2020 iteration, system moderations indicated that the challenge assessment system had been compromised. Investigations revealed a pattern of undesirable behaviour by a handful of teachers, which also had a knock-on effect to student behaviour in the latter parts of the challenge. As a result, a complete overhaul of the assessment system was identified as a priority for the development cycle leading up to the Rwanda 2021 challenge.
Full Peer Review Scoping
Summary of Key Changes
Redefine user permissions to remove the ‘trusted status’ from teachers to award final marks, as well as adding the ability to suspend users who are displaying clear patterns of undesirable behaviour.
Redefine the user hierarchy to (i) include ‘moderator’ roles, which allowed for ongoing moderation throughout the gameplay period; and (ii) allow system administrators to overrule score changes resulting from undesirable behaviour.
Introduce a user ‘credibility weighting’, so that the assessment system can automatically adjust a user's influence on the system when detecting undesirable behaviour patterns.
Introduce the display of warnings to users to deter undesirable behaviour as it is detected by the assessment system.
The addition of several parameters with the configuration of the assessment system to better manage user interaction (e.g. maximum daily limit of reviews allowed for a user).
The addition of live system dashboards to enable the Wavumbuzi team to track and monitor key indicators linked to reviews.
Include stronger messaging across all communications (e.g. teacher training videos, knowledge base, FAQs, etc.) regarding the active monitoring of undesirable behaviour.
Empowering Rekgo to adopt a systems management role where she could more actively oversee and coordinate live moderations and audits relating to the assessment system.
Random Control Trial Configuration
To support the administration of a Random Control Trial (RCT) for the Rwanda 2021 Challenge, the system required several upgrades to existing functionality, including; (i) the building of advanced survey logic to be able to host the RCT measurement instrument; (ii) the addition of suspension functionality at a school level, to segment control and treatment group users; and (iii) the construction of custom SQL (database) queries and data export functions to provide the data required for RCT analysis
Scoping
Technical Scoping Workshop: 05-07 April 2021
Credibility Weighting Upgrade
Restructuring
Comp, Quest, Challenge
SVG Graphics
Multiple Choice and Multiple Select Questions
Sprint Planning
HANDOVER
Kanban Board (JIRA)
The Kanban board was cleaned up a/o 08 October 2021. There are some items awaiting Peace and Ruti's input, but otherwise it is fully up to date.
The Wavumbuzi Project was exported from the XELT Atlassian account by Peace Alliance on 13 October 2021 and transferred to a new AGGP Atlassian account.
The new AGGP Atlassian account can be accessed using the below links:
Items Selected for Development
This is the suggested course of action for 5 October to 5 November as Kat (interim Product Owner) is onboarded and begins her research/scoping exercise:
First 9 tickets relate to the most higher priority bugs (up to WT-134). Additional bugs further down, but less of a priority.
Next, documented in WT-1118, WT-1119, WT-1120, WT-1121 looks at the set down and setup of Kenya and Rwanda respectively.
Then, I suggest focus shifts to system handover items (including shift to AGGP Atlassian account)
Thereafter (WT-310 through to WT-378) I suggest a focus on system maintenance, which should require no design input.
From WT-1046 through to WT-1016, I suggest attention then shifts back to lower priority bugs.
Finally, from WT-1047 through to WT-1123, I suggest moving on to scoping the items we discussed during Rwanda week (documented below)
A key consideration here is that with the desire to limit any major new feature development before Kat has had a chance to scope longer-term changes for the system, it is likely that Kenya 4 next year will be very similar in format to Rwanda 2 this year, with only time for minor changes likely to be possible in the new year before Kenya 4 (following the completion of Kat's scoping on 10 December).
JIRA Backlog
All other ideas, thoughts, inspirations, lessons and musings sit in the JIRA backlog in broad feature sections (Epics), which should be considered in the next scoping/sprint planning exercise for future development priorities.
Key Constraints
Here is a list of key constraints to consider for the 6 months between Nov 2021 to May 2022 to guide thinking on priority setting:
30 Sep 2021: Rwanda 2 Registration Live
17 Oct 2021: Kenya 3 Gameplay Ends
25 Oct 2021: Rwanda 2 Gameplay Live
05 Dec 2021: Rwanda 2 Gameplay Ends
Mid Mar 2022: Kenya 4 Registration Live
End Apr 2022: Kenya 4 Challenge Live
Key Improvement Areas
Below is a list of features/improvements identified during the Rwanda Summit which is recommended for consideration in the scoping of future development priorities. These all sit in JIRA as tickets for scoping.
WT-1105 = Shifting to a single sign-on system (registration only required once and data carried across challenges)
WT-1106 = Reconfiguration of badges in alignment with new quest structure
WT-1107 = Front-end configurations to handle multiple competitions (ability to have year-round content as well as 6-week challenge)
WT-1122 = Shifting to a single iteration (country-agnostic) system (with ability to contextualise for country challenge iterations)
WT-1108 = Upgrade of current servers to handle growing user base
WT-885 = Reviewing and upgrading report functionality on super admin and teacher portal (to ensure accurate, live data is displayed)
WT-218 = Setting up a country agnostic demo (which can be shared with external stakeholders)
WT-1123 = Amend API to shift from Google Data Studio to Power BI (TBC by M&E team)
WT-952 = Automation of regular system audits to identify bad behaviour (currently done manually)
WT-14 = Improvement of the school points formula
WT-984 = Expand the scope of the current GTM API (to improve operational reporting capabilities)
WT-1111 = Expand the scope of the current Hubspot API (to improve operational reporting/tracking capabilities)
TRANSITION
Onboarding (Interim Product Owner)
An interim Product Owner (Katarina Scholtz) was contracted between 05 September 2021 to 10 December 2021 to conduct stakeholder research, review the current product capabilities and compile a product vision for 2022 and beyond. The first month of the contract was dedicated to onboarding. To support this process, several resources and key pieces of documentation (all of which can be found within this playbook) were shared as reading material. Several meetings were also scheduled with the outgoing Product Owner (David Jenkins, XELT) to answer questions, problem solve and share insights.
Recruitment (Permanent Product Owner)
The recruitment process for a new permanent Product Owner spanned across 15 November 2021 and 15 December 2021. The outgoing Product Owner (David Jenkins, XELT) prepared several interview questions and scenario simulations to assess the short list of applicants. Following a round of interviews and scenario simulations, psychometrics were conducted ahead of a final interview, before a candidate was appointed. The process outcomes were documented in this Candidate Tracking Sheet.