Information Retrieval from Microblogs during Disasters (IRMiDis)

Track Description
Microblogging sites like Twitter are increasingly being used for aiding relief operations during disaster events. In such situations, identifying actionable information like needs and availabilities of various types of resources is critical for effective coordination of post disaster relief operations. However, such critical information is usually submerged within a lot of conversational content, such as sympathy for the victims of the disaster. Hence, automated IR techniques are needed to find and process such information.In this track, we focus on two types of tweets:

(1) Need-tweets: Tweets which inform about the need or requirement of some specific resource such as food, water, medical aid, shelter, mobile or Internet connectivity, etc. Note that tweets which do not directly specify the need, but point to scarcity or non-availability of some resources (i.e., a covert expression of the need) are also included in this category. For instance, the tweet “Mobile phones not working” is considered as a need-tweet, since it informs about the need for mobile connectivity.
(2) Availability-tweets: Tweets which inform about the availability of some specific resources. This class includes both tweets which inform about potential availability, such as resources being transported or dispatched to the disaster-struck area, as well as tweets informing about the actual availability in the disaster-struck area, such as food being distributed, etc.

Note that a particular tweet may be both a need-tweet and an availability-tweet  if it informs about the need of some specific resource, as well as the availability of some other resource.

In this track, a large set of microblogs (tweets) posted during the Nepal earthquake in 2015 will be made available. Table 1 shows some examples of need-tweets and availability-tweets from this dataset. Note that the dataset contains tweets not only in English but also in local languages such as Hindi and Nepali, and also code-mixed tweets, as shown in Table 1.

Table 1: Examples of need-tweets and matching availability-tweets, posted during the 2015 Nepal earthquake

Mobile phones are not working, no electricity, no water in #Thamel,#Nepal. #earthquake #NepalQuakeReliefPlease contact for drinking free service water specially for Earthquake Victim. Sanjay Limbu [mobile num] 
 RT @ArtOfLivingNow: Contribute Food, water, medicines, shelters, sanitation and basic relief materials needed for #NepalEarthquakerelief Tons of food, medicines, clothes & blankets reached to Nepal via RSS help still under progress.#EarthQuakeAgain #NepalEarthquake
 @skyasesh @YouthForBlood they are in search of blood donors for the people who are injured in earthquake If blood required than please contact representative of Nepal Voluntary Blood Donors Society [url]
 Nepalis, r w/o water & electricity. Water is essential to be supplied to the affected people in Nepal भरतीय रेलवे ने 1 लाख पानी क बोतले भेजी है । धन्यबाद @sureshpprabhu जी #NepalEarthquake
 भूकम्प पीड़त को खाना-टट चाहिए [url]can anyone we know pick the 2000 second hand tents from Sunauli and distribute it to the people in need in Nepal? #NepalQuake 

The track will have two sub-tasks, as described below.

Sub-task 1: Identifying need-tweets and availability-tweets

Here the participants need to develop automatic methodologies for identifying need-tweets and availability-tweets. This is mainly a search problem, where relevant microblogs have to be retrieved. However, apart from search, the problem of identifying need-tweets and availability-tweets can also be viewed as a pattern matching problem, or a classification problem (e.g., where tweets are classified into three classes- need-tweets, availability-tweets, and others).

Sub-task 2: Matching need-tweets and availability-tweets

An availability-tweet is said to match a need-tweet, if the availability-tweet informs about the availability of at least one resource whose need is indicated in the need-tweet. Table 1 shows some examples of need-tweets and matching availability-tweets. 

In this sub-task, the participants are required to develop methodologies for matching need-tweets with appropriate availability-tweets. Note that an availability-tweet is considered to match a need-tweet even if there is a partial match of the resources, e.g. if the need-tweet mentions about multiple resources and the availability-tweet inform the availability of a subset of these resources. Also, note that a need-tweet and a matching availability-tweet can be in different languages; either or both might be code-switched as well.

Participants may take part in sub-task 1 only or in both the tasks. However, to participate in sub-task 2, it is mandatory to participate in sub-task 1. 

There can be at most 4 participants in a team.

* The target for the participants is to identify all need-tweets and availability-tweets and match them with high precision as well as high recall.

* The runs that retrieve and match non-English and code-mixed tweets will get additional credit during the evaluation process.