Information Retrieval from Microblogs during Disasters (IRMiDis)
Track Description
Microblogging sites like Twitter are increasingly being used for aiding relief operations during various mass emergencies.  A lot of critical situational information is posted on microblogging sites during disaster events. However, messages posted on microblogging sites often contain rumors and overstated facts. In such situations, identification of factual or fact-checkable tweets, i.e., tweets that report some relevant and verifiable fact (other than sympathy or prayer) is extremely important for effective coordination of post-disaster relief operations. Additionally, cross verification of such critical information is a practical necessity to ensure the trustworthiness. Hence, automated IR techniques are needed to identify, process and verify the credibility of information from multiple sources. The track will have two sub-tasks, as described below.

Sub-task 1: Identifying factual or fact-checkable tweets
Here the participants need to develop automatic methodologies for identifying fact-checkable tweets. This task can be viewed as a binary classification problem, where tweets are classified into two classes – fact-checkable tweets and non-fact-checkable tweets. However, apart from classification, the problem of identifying fact-checkable tweets can also be viewed as an IR problem where the goal of the participants would be to produce a ranked list of tweets based on their worthiness for fact checking. Additionally, It can also be posed as a pattern matching problem.
 Following are some examples of fact-checkable tweets and non-fact-checkable tweets from the dataset that consists of about 50,000 tweets posted during the 2015 Nepal earthquake .

Example of fact-checkable tweets
  • ibnlive:Nepal earthquake: Tribhuvan International Airport bans landing of big aircraft [url]
  • #Nepal #Earthquake day four. Slowly in the capital valley Internet and electricity beeing restored . A relief for at least some ones
  • @mashable some pictures from Norvic Hospital *A Class Hospital of nepal* Patients have been put on parking lot.
  • @ Refugees: UNHCR rushes plastic sheeting and solar-powered lamps to Nepal earthquake survivors [url]
  • @ siromanid: Many temples in UNESCO world heritage site Bhaktapur Durbar Square have been reduced 2 debris after recent earthquake [url]
  • @SamitLive: Nepal has requested for Drinking water. @RailMinIndia has decided to send 1 Lak liter of Rail Neer over night.

Examples of non-fact-checkable tweets
  • Students of Himalayan Komang Hostel are praying for all beings who lost their life after earthquake!!! Please do...[url]
  • We humans need to come up with a strong solution to create earthquake proof zone's.
  • really sad to hear about d earthquake. praying for all the ppl who suffered & lost their loved ones. hope they get all the h…
  • @Gurmeetramrahim Msg helps earthquake victims
  • Nepal earthquake Students light candles offer prayers for victims: Students in Amritsar led a candle light vig...
  • I am so deeaking scared omg i dont even know what should i tweet.. This could possibly be my last tweet if the earthquake  doesnt stop 
Sub-task 2:  Identification of supporting news articles for fact-checkable tweets
 A fact-checkable tweet is said to be supported/verified by a news article if the same fact is reported by both the media. The dataset will contain around 6,000 news articles related to the 2015 Nepal earthquake. In this sub-task, the participants are required to develop methodologies for matching fact-checkable tweets with appropriate news articles. Table 2 shows some examples of fact-checkable tweets and extracts from news articles that support the fact reported in the tweet. For each fact-checkable tweet, participants should report - (i) the supporting news article id, and (ii) the particular sentence in the news article, which supports the fact-checkable tweet. It should be noted that many of the fact-checkable tweets might not have supporting news articles in the dataset.

Table 1: Examples of fact-checkable tweets and corresponding news article that verifies/supports the fact, posted during the 2015 Nepal earthquake

Fact-checkable tweet  Extract from supporting news article
 #Nepal #Earthquake day four. Slowly in the capital valley Internet and electricity being restored . A relief for at least some ones Four days after the deadly quake, more shops opened here and traffic returned to Kathmandu's roads. Authorities also restored electricity while telephones began to function in more areas
 ibnlive: Nepal earthquake: Tribhuvan International Airport bans landing of big aircraft [url] Furthermore, the Tribhuvan International Airport in Katmandu, Nepal has been closed for all commercial flights. Only flights carrying relief are allowed into its runways.



Note: 

*Participants may take part in sub-task 1 only or in both the tasks. However, to participate in sub-task 2, it is mandatory to participate in sub-task 1. 
There can be at most 4 participants in a team.