Over the last 10 years, our research team at the Human Rights Center has applied empirical methods of research to capture the opinions and attitudes of individuals in countries affected by war and mass violence. The research methods have included population-based surveys, qualitative studies, focus groups, and ethnography.

 

In northern Uganda we conducted a paper-based survey in 2005 and a PDA-assisted survey in 2007. Both surveys were equivalent in scope (sample size, geographic coverage, content, number of surveyors) and only differed in the data collection method. Comparing both experiences, there are a number advantages and a few challenges of PDA-assisted data collection, which are outlined below.

PDA-assisted data collection is cost effective: The 2005 and 2007 surveys were conducted with roughly the same budget. The specific costs of a paper survey include: paper purchase, printing, shipping of surveys, and data entry. On a survey of 2,000-3,000 people these costs are upwards of US$10,000, which is equivalent to the purchase of at least 20 PDAs. With PDAs, the specific costs included the PDAs themselves, the cost of programming, and the cost of one additional week of training for the teams. In addition, because PDAs can be reused in subsequent surveys, cost per survey decreases after the initial investment.

PDA-assisted data collection saves time: With paper-based surveys, implementation of the data collection is relatively rapid once the questionnaire is developed. However, data-entry is a time consuming process that can take several weeks. This results in a long delay between the end of the data collection and the analysis. With digital data collection, once the questionnaire is finalized, it has to be programmed, formatted and tested to be used on the PDAs. However, data are available on a near real-time basis, resulting in a very short delay between data collection and publication of the results. One of the objectives of KoBo is to reduce the time needed to program the surveys on PDAs, therefore allowing for a rapid "questionnaire development - data collection - data analysis - data reporting" cycle. 

PDA-assisted data collection results in cleaner data: There are a range of reasons why the database from PDA-assisted survey is cleaner compared to that of a paper-based survey: (1) PDA allow to set rules on what values can be entered. For example, unrealistic values can trigger an error message (e.g. if the interviewer tries to enter an age of 567 years, this would be out of range); (2) PDA allow to set rules on response options, for example allowing or not multiple responses; (3) PDA facilitates the navigation: interviewer are less likely to skip a question by mistake because the PDA automatically navigates to the next question. Where skipping patterns are used (e.g. some questions are only asked based on previous answers) PDA automatically navigate to the appropriate questions. With paper-based surveys, this is a frequent source of errors; (4) With PDA, digital data are collected, while paper-based surveys require data entry. Data entry, even carefully implemented (e.g. double data entry) leaves room for errors; (5) surveyors reported that PDA facilitated the interaction with respondents because they were less cumbersome to use and could be held at eye level, facilitating eye contact with the respondents.

PDA-assisted data collection provides better quality control: The PDA we used in northern Uganda automatically captured beginning and end time, as well as GPS coordinates for each interview. That information was used to assess the length of each interview. Surveys that took abnormally short or long time to fill in could be singled out and checked for validity. We could also assess whether the time between interviews was consistent with the steps described in the protocol (e.g. time needed to identify the next respondent and obtain consent). Similarly, the GPS coordinates allowed for comparison between the locations assigned to the interviewer and the actual place where the interview took place. It allowed to check movement between interviews, ensuring that interviewers indeed reached a different location.

PDA can be used for digital data collection by surveyors with no or little computer experience: Using PDAs for data collection is similar to using a cellphone, navigating menus and sending messages. It does not require experience with more complex handling of a computer. As cellphones are increasingly popular, it is easy to recruit interviewers that are comfortable with PDAs. In fact the use of PDAs was an incentive for the surveyors in northern Uganda as they saw it as an opportunity to learn a new tool. to ensure the proper handling of the PDA, an additional week of training was planed for the survey. In the end, we experienced no major technological problems during data collection. furthermore, KoBo is designed to make the data collection process even more transparent to the interviewer by changing the PDA interface so that it is solely dedicated to the survey data collection.

Power supply, security concerns and technical challenges still exist: One of the main challenge in situation of complex emergencies is to provide adequate energy sources for the PDA. In northern Uganda we tested some solar battery chargers, external battery packs powered with AA batteries, and portable generator. We are further testing external solar chargers, especially those developed by SOLIO which have the advantages of having high capacity internal batteries, so they can be charged up during the day and used at night to charge the PDA. They can also be charged from the grid were electiricity is available. Another concern in war-affected areas is the security of individuals and the risk to see PDAs stolen or confiscated by armed groups. We are working on ways to fully encrypt the data so that they cannot be access by third parties, as well as ways to synchronize the PDAs with a remote database so that data are not lost. Finally, the pilot projects we conducted so far require that each survey be programmed manually. While the solution works well, it is time consuming and not very flexible. One of the objective of KoBo is to make that process much simpler.