Examples of Good and Bad Descriptions

Intervention descriptions should clearly describe the details on the intervention or outcome to someone who has not read the paper.

Example 1:
Intervention code: Government tax policy for financial relief

Bad description:

Gave tax incentives

• This is far too short and nondescript. 

• What sort of tax incentives did the government give? You have to use this field to describe the specific details of the intervention/program/policy.

• This information is important for researchers who are using our DEP data. They want to be able to tell quickly whether an intervention is relevant for their purposes.

• This description also has no page numbers (likely as the text was not referred to at all).

A better description might be:

The intervention in this study involved government tax incentives, the authors use the Chinese VAT reform from 2004 to 2009 as a natural experiment and investigate it's impact on  export product quality.

“Under a new type of VAT, firms were allowed to deduct the purchase of fixed assets from the tax base, thus the investment costs are significantly lowered. 4 The reform was implemented in several stages… For the pilot industries in the three provinces, all the costs concerning the purchase, transportation, and installation of fixed investment in equipment would be deductible from the sales of final goods in calculating a firm's VAT liability.6

The second stage of the VAT reform took effect in July 2007, in which the pilot sites were expanded to eight industries,7 which included mining and electric power in 26 cities of six provinces of Central China (i.e., Shanxi, Anhui, Jiangxi, Henan, Hubei, and Hunan). In July 2008, the VAT reform was implemented in five cities of Inner Mongolia and 51 counties suffering from the Wenchuan earthquake. In January 2009, the VAT reform became eligible in all industries nationwide as a stimulus in face of the global financial crisis. Therefore, in the final stage of VAT reform, all firms were allowed to deduct the costs of fixed assets from the tax base.” (Page: 6)

Example 2:
Intervention code: Public safety laws in transportation

Bad description:

"Our study adopted a quasi -experimental design, particularly a single group interrupted time series analysis.  We analysed secondary data on the number of RTCs and

RTFs recorded annually, between 2006 and 2020 from the Road Transport and Safety Agency (RTSA). The study population included all the RTCs reported and recorded

throughout the country in the study period. We also conducted robustness checks to ensure that the data met the assumptions of ITSA." (Page: 2)

• The intervention description box should not be used to describe the evaluation approach used by the authors – we have a methods tab precisely to capture authors methods.

• We don't want to know what the authors did during their evaluation – we want to know WHAT THEY'RE INVESTIGATING.

A better description might be:

To curb the increasing number of crashes at night, Zambia instituted a ban on night travel for all public service vehicles in November 2016.

"According to the statutory instrument, night travel was defined as using a public service vehicle between 21:00 and 05:00. Night travel can thus be viewed as travelling at times when somebody would reasonably be assumed to be sleeping or resting such that their judgement on the road may be impaired compounded by poor visibility

during these hours."(Page: 2)

Example 3:
Intervention code: Agrobiodiversity | Ecosystem conservation

Bad description:
The intervention consisted of a workshop for Brazilian landowners to inform them about public PAs and show them how ESs offered by native regions have a favorable impact on rural activities.

"We predict that after participation in the workshop, landowners: i. will increase their perception of a public PA; ii. will perceive more ESs associated with native areas protected on their properties; iii. will better perceive the ESs related to agricultural activities; iv. will better perceive the ESs presented at the workshop; v. will perceive more ESs."

• We are not interested in the hypothesis or predications of a study. 

• What did the workshops involve? Use this field to detail the specific aspects of the program/policy.

A better description might be:

The intervention consisted of a workshop for Brazilian landowners to inform them about public PAs and show them how ESs offered by native regions have a favorable impact on rural activities.

“The workshop was entitled ‘Ecosystem Services on Rural Properties: Environmental and Economic Gains’ and hosted at the SNF headquarters on September 30, 2017. About 100 people participated, among them 25 landowners who were interviewed in our pretest, as well as their relatives, a number of under- graduate students and teachers from Silvânia… The following issues were presented: i. information about the SNF (history, activities engaged in, its role in biodiver- sity conservation); ii. the Cerrado biome and its role in water provision; iii. the PAs on private properties; iv. ESs (concept and types); v. ESs benefiting rural prop- erties (pollination, pest control, water purification, and climate regulation). The last item focused on animal welfare, mainly in cattle raising. We based the entire workshop on information extracted from scientific papers and academic reports.” (Page: 5)

Example 4:
Intervention code: Labour market services and intermediation

Bad description:

Hero program description:

The Hero program was developed in Argentina by Mesurado et al. in 2019 [8] and targets adolescents aged 12 to 15 years. Its objective is to promote prosociality through five socioemotional variables to make the effects of the intervention more lasting. To our knowledge, this is the first virtual intervention program aimed at promoting prosocial behavior and the socioemotional variables linked to it [8, 9]. Several advantages have emerged from virtual modality programs. First, they are highly structured intervention programs. Second, self-administered programs could improve adolescents’ personal reflection and self-interest in topics. Third, guided virtual modality programs are more cost-effective than face-to-face intervention programs. Fourth, online programs are easily spread. Hero is a self-administered program accessed from a Web page. To enter the program, adolescents must generate a username and choose an avatar, that is, they must select a virtual identity that represents them within the application. The program is guided by a Sensei who accompanies the adolescent throughout the application and presents the activities to be carried out through text and audio instruction, using the Spanish language with a neutral Latin American accent. The program is structured in five linked modules for promoting empathy (understanding someone else’s emotional state) [10], gratitude (being aware of having received some type of personal benefit from someone) [11], positive emotions (emotional experiences in which pleasure or well-being predominate) [12], forgiveness (“the process by which thoughts, emotions, and negative behaviors toward an offender are transformed into thoughts, emotions and more positive prosocial thoughts”) [13], and finally, prosociality (voluntary behavior aimed at helping other person, such as strangers, friends, family members) [14]. According to Mesurado and colleagues (2019), the variables empathy, gratitude, positive emotions, and forgiveness were chosen to be part of the program because they can be taught and because there is empirical evidence showing their predictive effects on prosociality [8, 9]. The program is presented as an adventure that consists of a trip to five islands. The adolescent participates in the different phases in a sequential and predetermined way by visiting the island of empathy, the island of gratitude, the island of positive emotions, the island of forgiveness, and the island of prosocial behavior. The stay in each of the islands coincides with an intervention session, which lasts approximately 30 and 40 minutes. When the adolescent arrives on an island, he or she watches one episode of a psychoeducational video that deals with the behavior to be stimulated and then performs a series of activities. The videos narrate different conflictive situations in the daily lives of four adolescents (two females and two males). The conflict is resolved in each episode by exercising the socioemotional variable corresponding to the visited island. The videos offer a brief explanation of this variable and conclude with three brief suggestions from Sensei with exercises to stimulate it. The two or three activities

that the adolescents then carry out are different on each island; some are playful, while others are reflective, relaxing, etc. (details of the activities for each island can be found in the papers by Mesurado and colleagues [8, 9] (Page: 2).

• This is far too long and copies in a lot of unnecessary text. 

• Intervention descriptions should be a balance between sufficient and precise detail. We want to capture the important information – we are not interested with the

• Use multiple quotes or ellipses (...) to avoid unnecessary detail in descriptions.

A better description might be:

Hero program description:

The Hero program was developed in Argentina by Mesurado et al. in 2019 [8] and targets adolescents aged 12 to 15 years. Its objective is to promote prosociality through five socioemotional variables to make the effects of the intervention more lasting... Hero is a self-administered program accessed from a Web page. To enter the program, adolescents must generate a username and choose an avatar, that is, they must select a virtual identity that represents them within the application. The program is guided by a Sensei who accompanies the adolescent throughout the application and presents the activities to be carried out through text and audio instruction, using the Spanish language with a neutral Latin American accent. The program is structured in five linked modules for promoting empathy (understanding someone else’s emotional state) [10], gratitude (being aware of having received some type of personal benefit from someone) [11], positive emotions (emotional experiences in which pleasure or well-being predominate) [12], forgiveness (“the process by which thoughts, emotions, and negative behaviors toward an offender are transformed into thoughts, emotions and more positive prosocial thoughts”) [13], and finally, prosociality (voluntary behavior aimed at helping other person, such as strangers, friends, family members) [14].... The program is presented as an adventure that consists of a trip to five islands. The adolescent participates in the different phases in a sequential and predetermined way by visiting the island of empathy, the island of gratitude, the island of positive emotions, the island of forgiveness, and the island of prosocial behavior. The stay in each of the islands coincides with an intervention session, which lasts approximately 30 and 40 minutes. When the adolescent arrives on an island, he or she watches one episode of a psychoeducational video that deals with the behavior to be stimulated and then performs a series of activities. The videos narrate different conflictive situations in the daily lives of four adolescents (two females and two males). The conflict is resolved in each episode by exercising the socioemotional variable corresponding to the visited island. The videos offer a brief explanation of this variable and conclude with three brief suggestions from Sensei with exercises to stimulate it. The two or three activities that the adolescents then carry out are different on each island; some are playful, while others are reflective, relaxing, etc. (details of the activities for each island can be found in the papers

by Mesurado and colleagues [8, 9] (Page: 2).

Outcome descriptions should clearly describe the outcomes of the study and how they are measured. You should NOT write the exact/final results for each of the outcomes.

Example 1:
Outcome code: Greenhouse gas emissions

Bad description:

We explored a range of diferent specifcations to examine the impact of carbon emission rights policies on carbon emissions (Table: 3)

-       Notice that this quote is very vague and uninformative. It tells us that ‘a range of different specifications’ were used – however, it doesn’t tell us what these specifications were

-       HOW were the carbon emissions measured?

o   We want to make outcomes as clear as possible to a researcher who has not read the paper before – it might be helpful for a researcher to know how exactly carbon emissions were measured.

-       Also, notice the spelling mistakes. Attention to detail is important so please utilise the admin panel’s spelling function.

A better description might be:

Carbon emissions: Authors use an accounting method “to calculate the carbon dioxide emission inventory of 30 provinces and cities in China from 1997 to 2015, including 17 fossil fuels in 47 sectors and 9 industrial processes.” (Page: 2584 and Tables: 3, 5)

-       Notice how I write the name of the outcome (as the authors present in in their outcome tables “carbon emissions:”), I then immediately follow with information on how the outcome was measured.

-       Once you’ve identified an outcome, search the name of that outcome within the text (CTRL + F) – this will help you to identify how the authors are describing/defining that outcome

-       Finally, I provide a page number for the definition quotation, and I reference ALL outcome tables that outcome appears in

Example 2:
Outcome code: Index of economic indicators

Bad description:

the impact of the carbon emissions trading system on the economy? In

order to study the impact of the policy on the economy (Table 4)

-       This is a very poor description and was likely accidental/rushed. The sentences do not make any sense in describing what ‘index of economic indicators’ the authors might have measured. A researcher who has not read the paper would have no chance at understanding what this outcome is.

-       Also notice the poor punctuation e.g., no capital letter to begin and a question mark where there is no question

A better outcome description might be:

Economic output: Authors “select industrial output values as a proxy variable for economic output, measured by total output value of industries above designated size” (Page: 2584 and Tables: 4, 5)

-       Again, notice how I write the name of the outcome (as the authors present in in their outcome tables “economic output:”), I then immediately follow with information on how the outcome was measured.

-       Again, I provide a page number for the definition quotation, and I reference ALL outcome tables that outcome appears in

Example 3:
Outcome code: Index of argicultural knowledge

Bad description

(Page.7)
-The workshop did not promote any change in correct naming of the SNF (24.0% positive change in the experimental group and 13.6% in the control group.

(Page.7)
-The workshop did not affect the experimental group's perception of the number of ESs associated with private PAs (interaction time and intervention.

(Page.10)
-The educational intervention also did not change land owners’ perception of ESs provided by private PAs in improving production of crops (20.0% of the experimental group showed positive change, com-pared with 16.3% in the control group.

(page.10)
-The workshop had an effect on land owners’ broad perception of ESs.

-       We should NOT be discussing the RESULTS of a study in the outcome description box i.e., we do not want to capture whether the intervention had a positive of negative impact on the outcome. We just want to capture what the outcome is.

-       A researcher cannot get a clear idea from these descriptions what exactly is being measured by the authors

-       A researcher may also not know what SNF, PAs and ESs are referring to. It is always useful to put the full terms in brackets.

-       Keep thinking to yourself – if a researcher had not read this paper before, would this outcome be clear to them? Could they tell what the authors were measuring?

-       Also note there is no reference to the outcome tables

A better description might be

·      Perception of SNF [Silvânia National Forest]: reported activities developed in the SNF (quantitative number of activities) (Page: 6 and Table: 3)

·      Reported advantages in having a property near to the SNF (quantitative number of advantages) (Page: 6 and Table: 3)

·      Number of ESs [Ecosystem services] related to the increase in crop production (Page: 6 and Table: 4)

·      Number of ESs related to the increase in beef/milk production (Page: 6 and Table: 4)

·      Number of ESs related to the increase in orchard production (Page: 6 and Table: 4)

-       Notice how much more informative this description is. From the previous description there was no mention of the quantitative number of activities developed in the SNF, there was no mention of the quantitative number of advantages of having property close to the SNF.

-       There was also no mention that the authors observed outcome associated with increased beef/milk production or orchard production.

-       In this new description, a researcher can see PRECISELY what farmers were asked and what was observed during this intervention.

-       Also notice how I have added the full terms for SNF, PAs and ESs in square brackets to further assist any researchers using our DEP data. We want all of our data and descriptions to be as clear as possible!

Example 4:
Outcome code: Patents

Bad description

Table 4. Smart city policies with urban innovation
Control variables: covering financial development (FIN), trade openness (TRADE), foreign direct investment (FDI), industrial structure (TS) and S&T human capital (HC).

-       We are not interested in the control variables of a study, especially not in our outcome descriptions

-       A researcher might be asking themselves, HOW are the authors measuring smart city policies with urban innovation? It’s our job to search through the paper and find out how an outcome is measured.

A better description might be

Urban innovation: the number of newly granted patents in a given year of the city (Page: 7 and Table: 4)

-       Notice that the first description tells us nothing about patents and nothing about how the outcome was MEASURED. This could all be very helpful information for a researcher.

-       Some outcome descriptions will be short, it depends on how complex the outcome or method or measurement is