3. Recommendations

Learned Lessons & Challenges

Monitoring and evaluation of development programs can contribute to learning between organizations and the people they aim to serve. Strong M&E strategies are also crucial for work within organizations and beneficial to the overall field of social change. Mobile technology has the potential to increase learning on all three of these levels, since it is more participatory, useful, and cost-effective than current methods of data gathering. As more organizations begin to use mobile technology for learning, their experiences should be documented and shared in order to understand how mobile technology can be used most effectively to improve M&E of all types of social change programs.

Using cell phones to conduct surveys can decrease costs of this kind of data collection. Data collected through mobile phones or PDAs can be sent directly to a central database, reducing time and money spent on data entry. Use of mobiles will also lower other administrative costs, including the price of telephone calls or postal charges incurred through other methods of data collection.

SPECIFIC ISSUES

Questionnaires* It is necessary to have specialized questionnaires for women, men and children, depending on their literacy rate and areas of responsibility.

* Questionnaires must have a section for open-ended comments and observations, in case there are thoughts/observations that are not captured by the questionnaire.

Analysis of the information

* In addition to analyzing the data collected, experts must interpret the data and translate results into clear reports.

* It is extremely difficult to evaluate projects in conflict-ridden areas, especially when fieldworkers are unable to work in a routine and organized way.

* No development project takes place in a vacuum; there are always outside social, political, economic and cultural influences.

IT

* The lack of network coverage in certain rural areas means that data capturers must to go to areas with network coverage in order to send their collected forms.

* There have to be specialized technologies to make it easy to use the mobile under these circumstances.

* There is the need to create a backup plan in instances of battery failure.

* In the near future, the EMIT application will be compatible with all Java enabled cell-phones, meaning that most facilitators will be able to use their personal cellphones, minimizing the cost of providing handsets and the management thereof.

Training

* There is a great need for broad-based IT skills as training is still required at most of the partner NGOs. Training has become a core part of the product offering.

* It is necessary to have special training to use the technology at local, central and ministerial level.

Policies and Security

* There are challenges involved in the provision of cellphones. Clear policies for cellphone usage are necessary to try to reduce loss.

* There is a big concern about security of the information.

Starting a new project on M&E using mobile? Things to keep in mind

Mobile data collection and reporting projects are abundant now that mobile use for development is taking off. Mobile data collection tools are often used internally in an organization, customized to fit with existing organizational processes.

Three main points to get your process started:

• Collect the right information: This is critically important, whether conducting a one-off study or developing a mobile solution for routine and ongoing data collection and reporting.

• Start from scratch with a system design process: This is necessary regardless of whether the mobile solution is meant to either replace an existing paper-based process or constitute an entirely new business process.

  • Involve all stakeholders, including those who will collect the data, those who will use or analyse it, and those who will manage the process.

Questions to think about before starting:

• • What data needs to be collected? Does the current system (if any) meet requirements? Is redundant data being collected, or is there something important missing? If data was to be available for analysis in a much shorter time-frame with the new mobile-based system, would it be useful to collect something that is not currently useful? Whether designing or redesigning data collection requirements, it may be helpful to create a paper form that represents the new data set. This can be a shared artifact agreed on by all stakeholders, as well as feeding into the system development process.

  • How will the system fit best with the work flow of data captures? Is it possible for data input to be done directly on the phone, or will it still be captured on paper first? For example, if a nurse is expected to capture the data while talking to a patient, would it be considered distracting and/or inappropriate for a phone to be used? Or, if fieldworkers are expected to work in potentially unsafe areas, would conspicuous phone use make them vulnerable to crime or surveillance in any way? In other words, be very clear about by whom and how the data will be collected and transmitted.

  • How will the data be analyzed? It is important to consider the data requirements of external systems that will be used in analysis. If everyone at the central coordinating office uses Microsoft Excel, consider a solution that allows data to be exported to Excel unless you can convince everyone to switch. If data is to be analyzed and responded to in real-time, consider setting up an auto-response system, or having the system staffed at appropriate times by a human responder. How feasible this is really depends on the volume of data coming in, but it's an important point to get right. If the person sending in data is expecting a response and doesn't receive one, they will very quickly give up on the system.

  • How will the data collection process be managed? Regardless of the technology used, successful data collection requires management. Responsibility for tasks such as initial training, provision of phones and airtime, ongoing management of data capturers and resolution of system problems should be decided upfront. This includes the financial responsibility for the system in the long term, something which many pilot studies fail to consider. To this end, it is important to know what the running costs of the system are likely to be in the long term, including replacement phones and monthly airtime bills. It may well be that costs exceeds the cost of an existing paper-based system, so all returns on investment (time savings and speed of transmission, increased accuracy of data, etc) need to be taken into consideration to justify the costs of a mobile data collection system.

So many technologies to choose:

Mobile data collection systems typically have several components that communicate for data collection, transmission, storage and retrieval.

For each component and communication channel, there may be several technology options, appropriate for different situations. While many data collection systems are built from existing commercial or open source components, or even come packaged as an end-to-end solution, it's important to understand the options and limitations imposed by the technical system design. It is important to take a deep look at the following components:

• • The data collection client interface, which the user interacts with to accomplish data collection and transmission

  • The data transfer method, which dictates how the information input on the phone is transmitted to a central server for storage and retrieval.

  • Server-side components to receive and store the data, and allow users to display and manage the database.

The graphic below shows how these three components relate to each other: