Lab Overview


CLEF eHealth Lab Sessions Programme [Tuesday 6th September, Evora, Portugal]

Session 1: [Session Chair - Joao Palotti]

13.30-13.50: Workshop Introduction (Hanna Suominen)

13.50-14.30: Invited Keynote 

Medical Information Retrieval Evaluation: Overview and Challenges (Allan Hanbury, TU Wien, Austria)

This talk will present a categorisation scheme for medical search tasks, and discuss the types of evaluation that can be done (computational, interactive, observational). An overview of some current evaluation initiatives in the medical domain will be presented. Finally, the challenges of each type of evaluation will be discussed, as well as potential approaches to overcome the challenges.

14.30-14.50: Task 1: Handover Information Extraction (Hanna Suominen)

14:50-15.10: Task 2: Multilingual Information Extraction (Aurélie Neveol)

15:10-15.30: Task 3: Patient-Centred Information Retrieval (Joao Palotti)

15:30-16:15: Poster Session (Task 1, 2 and 3 participating teams)

Session 2: [Session Chair - Aude Robert]

16.15-16.35: The Replication Track at CLEF eHealth Task 2 (Aurélie Neveol)

16.35-16.54: Team TUC-MI's participation in Task 1 (Christina Lohr) [15min + 4min QA]

16.54-17.13: Team Erasum's participation in Task 2 (Jan Kors) [15min + 4min QA]

17.13-17.32: Team CUNI's participation in Task 3 (Shadi Saleh) [15min + 4min QA]

17.32-17.51: Team InfoLab's participation in Task 3 (Carla Lopes) [15min + 4min QA]

17.51-18.15: Wrap-up and Open Discussion (Aurélie Neveol)


Problem: Laypeople find eHealth documents to be difficult to understand and also clinicians have problems in understanding the jargon of other professional groups even though laws and policies emphasise the need to document care in a comprehensive manner and provide further information on health conditions to help their understanding. A simple example from a US discharge document is “AP: 72 yo f w/ ESRD on HD, CAD, HTN, asthma p/w significant hyperkalemia & associated arrythmias.” However, authors of both care documents and consumer leaflets are overloaded with information and face many challenges in the timely and efficient generation, processing and sharing of such information. One example here is clinical handover between nurses, where verbal handover and note taking can lead to loss of information. Coupled with this the use of the Web as source of health-related information is a widespread phenomenon. Search engines are commonly used as a means to access health information available online, however the reliability, quality and suitability of the information for the target audience varies greatly. On top of this individuals’ abilities to express their information needs, and indeed their expression styles, vary greatly. Previous research has shown that exposing people with no or scarce medical knowledge to complex medical language may lead to erroneous self-diagnosis and self-treatment and that access to medical information on the Web can lead to the escalation of concerns about common symptoms (e.g., cyberchondria). Research has also shown that current commercial search engines are yet far from being effective in answering such queries. Further research needs to be carried out in order to provide solutions to these problems. CLEF eHealth aims at providing datasets and gathering researchers working on related topics.


Usage scenario: is to ease and support patients, their next-of-kins and clinical staff in understanding, accessing and authoring eHealth information in a multilingual setting. eHealth documents are much easier to understand after expanding shorthand, correcting the misspellings, normalising all health conditions to standardised terminology, and linking the words to a patient-centric search on the Internet. This would result in “Description of the patient's active problem: 72 year old female with dependence on hemodialysis, coronary heart disease, hypertensive disease, and asthma who is currently presenting with the problem of significant hyperkalemia and associated arrhythmias” with the highlighted words linked to their definitions in Consumer Health Vocabulary and other patient-friendly sources. Further, providing the required eHealth information in response to our target user groups information needs in a timely manner, where this information is reliable, accurate and available in a multilingual setting is crucial. In addition, auto converting a verbal nursing handover to text and then highlighting important information within the transcription for the next nurse would aid care documentation and release nurses time to, for example, discuss these resources and provide further information for a longer time with the patients.

This year, CLEF eHealth organises 3 tasks: 

Task 1: Handover information extraction
(New challenge)

Task 2:
Multilingual Information extraction
New: clinical text dataset, causes of death extraction from French death reports)

Task 3:
Patient-centred information retrieval
New: web crawl, queries, evaluation criteria).