Final Report: Independent Evaluation of the Ushahidi Haiti Project

posted Apr 12, 2011, 7:26 AM by Adam Papendieck   [ updated Apr 20, 2011, 1:46 PM ]

The Ushahidi Haiti Project (UHP) and the Independent Evaluation Team are happy to announce the release of the Independent Evaluation of the Ushahidi Haiti Project Final Report.  

Many thanks to the diverse set of individuals and organizations who made the UHP happen, and for their invaluable contributions to this evaluation process.  It is our hope that the work put into creating this document will be found useful in the continued development of crisis mapping approaches and help strengthen the positive impact of their application worldwide.

With the release of the final evaluation report, the evaluation team would like to share a few additional thoughts on the importance of the Ushahidi Haiti Project Activity. Information systems used by the humanitarian response community have rarely been evaluated.  The ten year evaluation in 2006 of Relief Web is a notable exception and had prompted some of the Relief Web functionality that we all enjoy today.  Needs Assessment has received some attention, but there are no other well evaluated examples in the humanitarian realm.  None-the-less, one of the most common recommendations of almost any evaluation of humanitarian response is a call for improved participation of beneficiary populations and better information systems. Both the UHP team and the evaluators knew we had a unique opportunity with this evaluation to contribute some important learning in these areas to the humanitarian response community.

Being able to document that the humanitarian spirit is alive and well with young people today was perhaps one of the most rewarding aspects of the evaluation.  As worrying as the proclamation by Jan Egeland that “humanitarian action is under attack” in OCHA’s recent Stay and Deliver document must be, we think that UHP and other projects like it continue to show that Humanitarians will continue to be more dynamic and innovative than those that would attempt to pervert or repress perhaps the most noble human motivation -- to help those in need.  In our mind, UHPs ability to connect volunteers, those who are suffering and those that want to help will be the project’s lasting relevance.

Volunteer efforts are at the core of any emergency response.  Whether it is family members or neighbors sheltering someone that has lost their home or students collecting money for an international organization, it is voluntary action that makes the difference in emergency, especially during the first seventy two hours, which is the time often required to launch a vigorous formal humanitarian response.  UHP showed us how the principals and technology behind social media and rapid information integration can quickly connect volunteers.  Rapid—this is important because it allows for rapid action by local people and smaller voluntary groups in the immediate aftermath of the emergency.  

The evaluation re-affirmed on-going problems in evidence-based humanitarian action. The disconnect between information, needs, and decision making persists to thwart effective humanitarian response. Much can and should be done to highlight scrutiny of the relationship between needs, information and action in differing humanitarian stakeholder groups. Most significantly, crowd sourcing through social media is an emergent and highly potent instrument for humanitarian information and action. We are on the frontiers of applying this tool effectively for population well-being. 

Of interest to evaluators --the data sources that the evaluation team used were not your grandfather’s document review, key informant interviews or probability surveys.  Automatic time stamps are added to Skype chats, sms messages, email and web postings, and this is an invaluable tool for evaluators when trying to triangulate information.    On the other hand, there are significant challenges of analyzing so many small and often disparate threads of conversations.  Reinforced and adaptive models of program logic and improved techniques for intelligent analysis of these data, including multi-media data will become increasingly important for evaluators of this generation.

Crowd sourcing is proving itself globally as a powerful information/social movement integration that is transforming the world. Georeferenced and aggregated crowd sourced information is transformational to disaster management, especially catastrophy management. We were honored to be part of the analytical team that might foster learning and adaptation. We favor the application of developmental evaluation as an evaluation strategy, in contrast to traditional evaluation frameworks.


The UHP Independent Evaluation Team

Nathan Morrow 
Nancy Mock 
Adam Papendieck 
Nicholas Kocmich

New Survey for Volunteers

posted Feb 15, 2011, 11:25 AM by Adam Papendieck

If you were a volunteer for Ushahidi Haiti or Mission 4636 we would very much appreciate your participation in our short online survey.  We feel your input is extremely valuable and are hoping in particular to collect and document cases where the project had an evident impact on quake-affected Haitians. 

Our online survey is 32 questions and 10 minutes long.  Please take it here: 

Executive Summary of Preliminary Findings

posted Jan 12, 2011, 11:18 AM by Adam Papendieck   [ updated Jan 13, 2011, 2:28 PM ]

We are happy to report that the Executive Summary of Preliminary Findings from the Independent Evaluation of the Ushahidi Haiti Project (UHP) has been released for your review.  See also the related post by UHP Evaluation Manager Chrissy Martin over at CrisisMappers.

The UHP/evaluation team acknowledges the dedication of the Haitian workforce that made Mission 4636-Ushahidi Haiti possible.  They worked under the same conditions as the international aid community, but with the added burdens of their personal losses. Our thoughts remain with them and their families today.

This summary highlights the multiple method evaluation of the Ushahidi Haiti implementation.  The evaluation assessment techniques included semi-structured interviews, internet surveys and document review/analysis of chat records, blogs and the database of mapped incidents.

The UHP was implemented amidst a network of interrelated information initiatives.  Preliminary findings in the initial phases of this evaluation have suggested that the scope of the study be expanded to include more work in the domain of the Mission 4636 activity as the Diaspora mobilized and leveraged by the 4636-UHP interaction has probably played a significant role in the networked response.  In order that the experiences and impact of this key group be appropriately assessed, a revised survey instrument has been developed is currently under institutional review. The final report will be delivered in Feb 2010. In the meantime, Ushahidi Haiti and Mission 4636 volunteers, we are still seeking your input in a revised survey!

A few key points:

The evaluation concludes that UHP was remarkably relevant to key stakeholders in this large scale catastrophe as the only source of dynamically mapped situational information during the early days of the crisis. The UHP was a dramatic demonstration of effective networking among social media communities and affected communities.

The information UHP provided was used by important stakeholders in the response for situational awareness and--to a more limited extent—tactical and operational decisions. The US Department of State, US Marines, US Coast Guard and a number of small non-profit organizations and private citizens were users of the UHP data stream.

The extent of information use was limited by a number of factors including stakeholder awareness of UHP, information content and classification of UHP map entries, familiarity with crisis informatics tools, accuracy of information and attitudes of some stakeholders toward social media.

While the efficiency of this effort was enormous--as evidenced by the fact that thousands of messages were processed by a large scale global volunteer effort--the evaluation identified specific areas for improvement related to soliciting and processing messages, capacity development and general systems improvements.

UHP Independent Evaluation Team

Nancy Mock

Nathan Morrow

Adam Papendieck

1-3 of 3