Front End Analysis

Learning Need

Mobile should be an integral part of your learning and information infrastructure / architecture. It can be viewed as a replacement for other learning options, but is usually considered as a supplement or reinforcement for learning, or as a platform for providing performance support.

Five Moments of Learning Need

A review of the Five Moments of Learning Need could help to form a learning strategy and determine where mobile learning may be appropriate. These were created by Dr. Conrad Gottfredson (from Gottfredson and Mosher (2011))
  1. When learning for the first time
  2. When wanting to learn more
  3. When trying to remember
  4. When things change
  5. When something goes wrong
Which moments are most appropriate for delivery on a mobile device for your learners? What materials would be most appropriate at their moments of need?

Learning Categories

You may also want to think about your mobile learning project in terms of three basic categories presented in Quinn (2011):
 
1. Learning augmentation
  • Motivational examples - presented before and after a formal course to reinforce the need to learn the material
  • Extending learning processes
    • Reconceptualization - providing new concept representations
    • Recontextualization - new contexts of application as examples
    • Reapplication - more practice
  • Connecting with feedback
  • Supporting learner preferences – presenting material in the medium, time, format, etc. preferred by the learner
  • Contextual opportunities – adding value by tailoring learning to specific locations or times
2. Performance augmentation
  • Media capability – user generated content (using camera, etc.)
  • Data and processing ability – calculators, decision trees, etc.
  • Communication – ways to connect with other learners
3. Learnlet
  • Microcourses (just a few minutes in length)
    • Single feature of a product
    • Single aspect of a service
    • Single step of a larger procedure
  • Five screens
    • Intro
    • Concept
    • Example
    • Practice with feedback
    • Summary
For microcourses, Quinn suggests digital comic strips as examples or counterexamples, combined with simple concept introductions and limited practice.
 
Another type of learning augmentation could be metacognitive awareness, as described in the Rethinking Learning section of this handbook.
 
Performance augmentation can take two forms, according to Rossett and Schafer (2007): planners and sidekicks. Sidekicks are described as follows (adapted from information on p.117):
  • Called upon DURING a challenge
  • Provide information, reminders, directions, and warnings right when needed
  • Can be tightly or loosely integrated with a task
  • Help us avoid mistakes and costly errors
  • Are sometimes just one element of a blended solution to a performance problem
  • Offer advice and help performers comply with business rules or best practices
  • Increase the speed or accuracy of tasks
  • May be temporary or permanent supports, depending on whether the info is learned through application or not
  • Can be delivered very simply
  • Can help performers complete a task without prior training
Planners are described as follows (adapted from information on p.97):
  • Called on just BEFORE or AFTER a challenge
  • When referred to before performance, they provide info that helps clarify what is to be done or what will happen
  • When used after performance, they provide info to reflect on success and plan for future adjustments or improvements
  • Help us reflect on our performance with intent to improve
  • Can make complex tasks much simpler and help user perform smarter
  • Delivery methods and content presentation can encourage collaboration and influence attitudes
  • Can be tailored for users and provide personalized information to encourage and guide performance
  • Make information accessible to decision makers.
If you are developing performance augmentation for the mobile platform, use the principles of performance-centered design. Design interfaces and navigation around the way that mobile performers do their work, not based on the inherent organization of the domain of knowledge. See Craig Marion’s article on PCS  for more details.

Analysis Questions

A determination of your users and their needs and interests should be one of the first steps. The steps in mobile learning needs analysis are similar to those you would follow in any other project, with some specific considerations for mobile learning. Below is a representation of some of the analysis questions you would ask for a mobile learning project.

Objectives

  • Why do you want to implement a mobile learning project
  • What unique result do you want to achieve that requires a mobile learning solution?
  • How will you know when you have achieved your learning goals?
  • Do you have a working description of the project including the vision, benefits, and cost estimates?

Technology Issues

  • To what degree are mobile devices in use by the targeted audience? 
  • Are there training needs that are not being met for the mobile audience? 
  • How technologically sophisticated are the mobile devices in use by the target audience? 
  • Are there sufficient resources?
  • How technologically savvy are the users with their devices? 
  • What internal mobile competencies does your organization have?
  • Are there policies that may need to be changed or added? 
  • Is there sufficient support, knowledge, and experience for a mobile solution from IT and content owners?

Organizational Issues

  • Are there other mobile learning related efforts that you can collaborate or learn from? 
  • Are mobile devices owned and issued by your organization and who would administer them? 
  • What future developments for mobile learning are on the horizon within your organization?

Design Strategies

  • Is the proposed mobile learning project for formal learning programs, informal learning or performance support?
  • What kind of user interface will be used?
  • Is content already available? If not, who will produce the content and do they have experience with mobile learning specific considerations for content? 
  • What content creation tools will be used?
  • How will you maintain and update content?
  • Who owns the rights to content?
  • Does the content currently exist in another (i.e., non-mobile optimized) form? What will be necessary to adapt it?
  • Have you considered how user interface design decisions can impact learning outcomes and user experience? 
  • Will mobile device-specific interactions be used? (e.g., touchscreen)

Implementation

  • Who is going to produce the content?
  • What is your timeline?
  • Will users need offline access? (i.e., will they have connectivity at all times)
  • What content product standards will be employed?
  • What devices will be used/supported?
  • What actions or activities need to be tracked?
  • What will the user interface look like?
  • Where is the data going to be stored?
  • How will availability of the product be marketed?
  • Who is going to support the users?
  • Have content production guidelines been developed; for both technical implementation and consistent learning experience purposes? 
  • Which, if any learning actions or activities will be tracked? How? (e.g., xAPI and LRS) 
  • Consider the work context of the mobile learning in very specific terms. For example, will the user need both hands to operate equipment? How will the mobile device fit into this workflow? (Elearning Guild, 2013)

Audience and Stakeholders

  • Who are your target audience and stakeholders? 
  • What will your stakeholders need to be successful? 
  • What does your target audience need to be successful? 
  • Is there a person in your organization’s management that will advocate for the mobile initiative?

Technical

  • Do you have testing devices and debugging tools? 
  • Where and how will mobile learning content be stored? 
  • How will content be distributed in a way that is suitable for mobile learning? 
  • Who will provide the data service for mobile delivery?

Evaluation

  • How will you evaluate your effectiveness?
  • How will you assess learning?
  • What reports will be generated and for whom?
  • Do you have a plan for testing and evaluation?

Challenges

  • Are there special software, infrastructure, device, etc. requirements to build the mobile solution? 
  • What will it cost and how will it be funded? 
  • Who will pay for and own the equipment, the network and service charges, and system maintenance? 
  • How will this new mobile learning initiative be communicated/marketed? 
  • Is there a sustainability plan in place to support possible scalability, growth, development, expansion, updates, funding, etc.

Opportunities

  • Are there other departments or groups within your organization that could benefit from your mobile learning initiative? 
  • What current content can be re-purposed for mobile distribution?

Security

  • Are there security considerations that need to be taken into account for content, devices, network access, etc.? 
  • Will training on securing techniques and features be available?

Support

  • Does your organization need to support users with devices or mobile apps, e.g., for those without personal devices or funding for purchase of applications? 
  • What user support will you provide for device and applications? 
  • Will users need training on the use of the devices?

Frequent Technical Concerns

Concerns to be addressed during the analysis phase include:
  • Ambient physical environment - The screens on many devices are hard to see in bright sunlight; the devices are susceptible to damage from getting wet; and they have an operating range of temperatures that may be exceeded by environmental conditions.
  • Battery life - This can vary greatly depending upon use and connections. There are considerations to conserve battery life in development as well as optional charging options until battery life improves.
  • Connectivity - See Connectivity & Bandwidth in Design Considerations for details. Connectivity can be mitigated to some degree by pre-loading the device (with static documents, for instance).
  • Data & service charges - These costs vary greatly among carriers and can rise significantly during  international travel.
  • Device ownership - Will devices be furnished to the users?
  • Media  compatibility - Flash is not supported on iOS devices, for example.
  • Safety – There may be safety concerns, depending on the context of use of the mobile content. You would not want your content design to be predicated on interaction with a mobile device while the user is preoccupied using a dangerous piece of equipment.
  • Screen size - See Displays in Design Considerations for details.
  • Security - This relates to the issues of losing a mobile device that stores personal data, “rubbernecking” of people in the vicinity of a user, and transmission of personal data across wireless networks. There are often industry-specific rules or government regulations driving the latter (for example, Privacy Act of 2005). Depending upon the sensitivity of content, this may be the most difficult challenge. There are available solutions, but trainers need to work with their information systems team for the best solution. “Remote wipe” capability is one aspect of security that may need to be considered (i.e., if the device is lost, the data can be deleted through a remote command). 
  • Standards – SCORM and other standards are predicated on traditional elearning, which usually doesn’t work well on the mobile platform. ADL is currently researching and developing a mobile version of SCORM that will integrate the tracking and assessment features of the Experience API. Additional non-learning standards may also be worth considering such as S1000D and DITA (i.e., for technical manuals), and W3C accessibility.
  • Technology changes - The global mobile industry is now the most “vibrant” and “fastest growing industry” on the planet. Expect improvements and changes.
Variables include:
  • Network - Performance varies with connection speed. The needs of the disconnected user may also need to be addressed.
  • Mobile device capabilities – The designed capabilities of mobile apps are driven by the opportunities and limitations of the devices themselves. This includes basic features such as processing power and memory, as well as “add ons” like cameras.
  • Carrier - Not all carriers support all devices or versions of platforms. Global deployments need to consider carrier options.
  • Device(s) supported - New devices continue to become available with varying capabilities and sizes. Unless your organization is furnishing all devices and regular updates, this can become difficult to support (unless standards are implemented).
  • Platform(s) supported - Deloitte estimates the cost of developing for two OSs is 160 percent of the cost of developing for one.

ADL conducted a survey in early 2011 identified the following concerns and issues within DoD organizations:

  • Personal Accountability - Measures ensure individuals are responsible for their actions.
  • Organizational Policy - Internal policy that provides guidance for content access and device usage.
  • Ownership/Life Cycle Management - Ownership and maintenance of content.
  • Assessment & Testing - Testing capabilities and offerings on mobile devices.
  • Data at Rest & Records Management - Determination if one’s data is no longer valid and if/when it should be expunged from system.
  • Connectivity & Bandwidth - Ability to access the internet or other network connection and the speed of that connection. 
  • User experience/Usability - Providing an optimal experience in terms of navigation and other user interactions.
The results of this survey are in the report Mobile Learning: The Current Landscape in the DoD

Justifying Mobile Learning Initiatives with ROI

As with any learning project, it is important to substantiate the expected return on investment. This needs to be estimated in the planning stages, to gain institutional support for your initiative, and measured after deployment to establish the ongoing importance of your mobile learning program.

When trying to gain institutional support, and in general throughout the process of design, development, and implementation, make sure you involve all potential stakeholders. As part of your sales pitch to them, you may want to point out ways that users in the target audience, and they, the stakeholders, are already using mobile devices in an ad hoc way to do their jobs better. And the stakeholders may understand the value of your mobile learning initiative better if they are already familiar with the target smartphone or tablet platform (i.e., they own and use it themselves). (Elearning Guild, 2013)

Keep in mind that you may need to justify your initiative in a different way than other learning initiatives, since mobile learning is such a different paradigm (for example, getting away from courses and using more performance support). Mobile learning may drive different outcomes that may not be so easily measured in the ways that your organization is used to measuring training outcomes. (Elearning Guild, 2013)

Pangarkar (Elearning Guild, 2013) uses the RADAR concept shown in this diagram to express strategies for maximizing buy in for a mobile learning initiative:


There are particular unique opportunities and problems in doing ROI for a mobile learning project. Here are some pointers, summarized from Woodill and Udell (2011):
  • Use both quantitative and qualitative measures
  • Calculate benefits for the life of the project (instead of on an annual basis, as is usual for training projects), since on the mobile platform, content can reinforced at any time
  • Include financially intangible benefits in any documentation, such as increased job satisfaction, improved quality of interaction between a company and its employees, etc.
See the paper above for more details on the above, and a model of how to conduct an ROI study for a mobile learning project.
You should carefully consider which parts of mobile learning initiatives should be considered capital investments and which should be considered operational expenses. This can make a difference in ROI figures, since capital investments are amortized over a long period instead of as a single cash outlay (as in operational expenses). (Elearning Guild, 2013)
Comments