Essential Tips for Data Extraction

1. Highlight the study PDF – this may not be possible if you are viewing the study on a browser. Downloading the PDF to your computer will in most cases allow you to highlight it.

• Highlight key text as soon as you read it – this can help you keep track of all key information, especially if it relates to a different tab than the one you're currently coding

• Use highlights to develop more accurate intervention/outcome descriptions – you have to be careful when copying large chunks of text into the description boxes as doing so may pick up a lot of unnecessary information. Highlighting the most important information throughout the paper will help you to be more discerning and develop more accurate descriptions.

Examples of Good and Bad Descriptions

2. Begin each study extraction in the Findings tab – coding intervention first will centre you on the most important details of the study. This will help you:

• Confirm a study is DEP eligible – while all studies assigned for DEX have been screened, some ineligible studies may get through. Closely considering the intervention first will help you confirm the study is includable for DEP.

• Code Sector, Themes and DAC Codes with more ease - it's important to remember that sector, theme and DAC codes are always coded on the INTERVENTION. Once you're explicitly clear on what the intervention is, the intervention code you choose can often help direct you to the most appropriate sector.

After you've extracted the Findings tab, start from the Publication Info tab – jumping between tabs too often could result in missing fields, it's therefore advised that you begin coding consecutively from the Publication Info tab once you've finished the findings to reduce your chances of missing anything.

3.  Utilise paragraph indents to quickly identify key areas of text – reading the first line of each paragraph can help orientate you within a paper, helping you to locate the most crucial text. It's advisable to do this in the 1) introduction 2) context/background and 3) methods sections. (Note:  in general, literature reviews are less likely to have any specific details on the intervention in a study – however pay attention to any headings within the literature review that could relate to background/development of the intervention)

4. Review ALL tables (including in the Appendices) – anything that's evaluated with an IE method should be extracted. Ensuring to review all tables will help you identify:

• ALL outcomes – while some authors make is explicitly clear which outcomes they're investigating within the text, other authors may investigate a numerous list of outcomes which they may not write about in much detail. Ensuring to always search through each outcome table will help you capture everything.

• Equity analysis (subgroup and heterogeneity) – authors should make it clear within the text whether they explore any subgroups or heterogeneity - however it's often easily determined from the outcome tables whether authors have split the sample or encoporated any interaction terms to capture heterogeneity. 

How to Read Outcome Tables

Tips – 3ie Admin Panel Tabs

Attention to detail is crucial for all fields on the DEP –  however this is particularly important for the Publication Info Tab as all of the information here will be displayed on our public site

• Studies must be in title case – in many instances you will have to manually adjust titles yourself. A title case converter may be useful to you (https://titlecaseconverter.com/)

• Authors names should be in full wherever possible – sometimes searching through the dropdown list will give you a more complete name for an author (particularly if the name in your DEP record has single initials). [Note: it's always worth doing a google-check to ensure the full name of the author is affiliated to the same institutions as those in your paper]

• Check that publication URLs link to appropriate sites for members of the public who may have no institutional access

1. Ensure the URL doesn't take you to a login page like EBSCO – it should help a member of the public locate the publishing site of the paper.

2. You should code the "Open access" field from this appropriate URL. If you find a free version of the paper on a site such as "research gate", please do not use this URL.

• If you have institutional access you will need to be cautious and try to search on a separate browser to determine whether a study is Open Access. If you're unsure, please contact your coordinator.

Sectors, Themes and DAC Codes should be coded on the INTERVENTION

• If you've started your extraction by coding the intervention of the study – you can use the Interventions Taxonomy to help you identify the likely sector the intervention belongs to (simply look at the sector assigned to that intervention code and read the sector description to confirm you agree)

• Be mindful of studies that investigate the impact of an intervention on outcomes from a different sector for example:

1. [Write example out for study 15466 - study looks at impact of reservoir on indigenous community. The sector should be WASH not social protection. Although there are some intuitive links with social protection, this reservoir was not implemented purely for the benefit of the indigenous community. It's important to scale back to the core intervention itself to ensure it's coded correctly]

•  Unless the sector is clear, it's advised that you first search the program/policy/key term for the intervention in the search bar on the top-right of this website. This will show you all the DEP resources that this program/policy/key term appears in. Remember that finding a key term in a certain sector code does not necessarily mean it is applicable to your intervention - always ensure to read the full definition and included/excluded activities. Note that the "Excluded activities" sections can be particularly helpful in clearing up tricky cases. The taxonomies are also hyperlinked to ease your navigation of them

• Don't forget to review the dropdown for 'Other Topics' – it can be easy to forget this field as many studies will be "Not applicable" so it's always good to double-check the list

• Coding equity:

1. For lower-level codes such as subgroup and heterogeneity analysis, a clear point of reference will be the tables i.e., is the sample split (then you must code as a subgroup analysis)? Are there any interaction terms (then you must code as a heterogeneity analysis)?

2. For higher-level equity codes, you'll usually need to determine this from the rest of the text. The study should make it clear if there was a particular focus on an equity dimension – use the first sentence of each paragraph or search key equity terms to help you quickly filter through the paper

• For tricky cases – please clearly describe the intervention and your confusion in the FAQ

Dataset name/location
For the Transparency Tab, we're interested in whether a study discloses information. Is there sufficient information to go out and find this dataset (perhaps it wouldn’t be immediately available to a member of the public – but is there sufficient detail that someone with interest and resources could locate the dataset used?)

• Check the Data section

• Search the term "data" throughout the paper

• Check whether the tables provide sources

• Check the references

• Double-check the correct country is listed – but usually this tab will not require any input from you

Unit of Observation

Many different countries have different ways of referring to are areas within a country – this can sometime make this field difficult to code. It may help by considering that the list in the protocol increases in size/magnitude.

Project names (or policy names)

• These should be sufficiently unique to the program/policy – uniquely identifiable – not a general term for a type of project or policy

• A program name will usually have capital letters but beware of common intervention names such as Community Driven Development (CDD) or Carbon Emission (CE) trading policy that describe a widely studies intervention mechanism but are not unique to the particular intervention being evaluated. 

• Make sure to write the name in full (any abbreviations added in brackets)

Agency names/Agency types

• Before selecting an Agency name, check if it exists in the Common Agencies tab of the IE and SR protocols. If it does exist there, and you use the exact name from that tab, the agency type will be auto-populated for you (saving you precious time).

• We aim to code Agencies at the level of the highest organisational level (e.g. Country government or Multilateral institution) except in certain specified cases where we want to track the contribution of different agencies (UN, US government, Chinese Government). We aim to periodically update sub-agencies in the 'Common agencies' tab of the protocol (see screenshot to the right) for those common agencies that are causing confusion for our coders.

• When coding the Funding/Implementing Agencies tab on the admin panel, you may have noticed you can no longer create a new tag. The option to create a new tag was removed as we have recently done some clean up on the agency names fields

• DEP Team: If you cannot find an agency name in the list that you're coding, please go to the "Agency Names" sheet in the DEP DEX Tracker. Please enter your name, the study ID, the type of agency and the name of the agency in the labelled columns. Once you have provided the detailed on the "Agency Names" sheet, you do not need to take any further action.

• Other projects: Please follow instructions from your project coordinator/manager.

In rare cases you may find that the code does not fit into any codes from the DEP Intervention/Outcome taxonomy. There are options in the intervention and outcome dropdown list named 'Other intervention' and 'Other outcome', respectively. When you choose that option, these are the rules you MUST follow:

DEP Team:

•Log it in the DEP DEX Tracker in the ‘Other Category Suggestions’ tab. Where you MUST mention intervention/outcome, sector, broad group, definition and disaggregated intervention/outcome. There may be cases where it is difficult to specify this. You can always ask the DEP Coordinator for help in such cases.

Other projects:

•Follow the instructions of your project manager/coordinator.