4. Quality Control

This chapter describes the procedures for assessing the quality of the  maps temporal series, the publication of this information and other information necessary for the transparency of the initiatives and the dialogue with external agents to be carried out by the MapBiomas teams.

This section describes when the accuracy analysis is required, teams’ independence, what estimates should be reported, issues related to publication of validation points and sample-based area estimation.

• The first collection of a temporal series of maps is the beginning of a learning process. Therefore, the quality of the map can be assessed by comparison with other existing maps only;

• From the second collection onwards, it is important to plan an accuracy assessment based on independent validation points, not used in algorithm training;

• It is recommended that the team responsible for validating the map is independent of the one carrying out the mapping. However, interaction between teams is crucial to define criteria, characterize classes, create interpretation keys and train interpreters;

• The number of validation points and sample design chosen must be based on the relevant literature and explained in as much detail as possible in a specific document for this purpose;

• The accuracy analysis must report the values ​​of global accuracy, allocation disagreement and quantity disagreement, following the scientific literature, for each legend level, as well as the omission and commission errors of each mapped class, considering all years of mapping;

• The accuracy analysis can be developed in stages, but it is important to plan the inclusion of the accuracy of rare classes, accuracy of deforestation and regeneration for secondary vegetation and of other initiatives (water, fire, etc.), when applicable;

• Validation points and all results of accuracy analyzes must be publicly available on the respective territories' websites;

• Validation points must also be published as collections. When new validation points are added or when they are revised, they must be published as a new collection. For revisions, an evaluation by the classification teams and indication of inconsistent points (points, classes, criteria, specific regions) and necessary corrections are recommended. When updating data, a revision is recommended every year in cases of incompatibility with the previous collection.

• Validation points must be used to report sample-based area estimation adjusted based on omission and commission errors, especially in scientific publications.

This section describes how to internally evaluate the collections. 

• After the publication of each collection and during the production of a new classification, additional analyzes must be carried out to internally evaluate the data, in accordance with the Spatio-temporal Assessment Guide for Collections;

• Suggested analyzes include:

  • comparison with other available maps and data;

  • assessment of the impact of different processing steps (classification, integration and filters) on data throughout the time series;

  • comparison with the two previous collections to understand and justify possible differences;

  • analysis of pixel trajectories to understand landscape dynamics and identify possible inconsistencies in class changes;

  • identification of problems and challenges in mapping that still exist, as well as innovations planned for the next collection;

  • spatio-temporal analysis of accuracy;

• If a serious error is found after the collection publication, it is recommended to publish a new version of the collection (e.g. Brazil 7.1 collection), with special attention to communication material and transparency of improvements (updating informative notes, highlights documents, website, infographics, etc.), as described in the Scope chapter.

This section describes the content of the informative notes and when to update them.

• Informative notes (see example in this link) must be published in each module of the platform to describe the respective data, informing the publication date, the period of the historical series, and noting that the data is in constant development;

• Information notes must always be kept up to date, including when products and modules are updated without changing the collection;

• It is important to include the contact email " contato@mapbiomas.org "  to receive criticisms and suggestions. These emails will be directed to focal points in the teams for each territory, module or theme;

• Links to the page where the ATDBs are made available (with DOI of the general ATBD, if any) and to the document on the accuracy analysis must be included in the informative note. Furthermore, it is essential to inform the license of use and the method of citing the data according to the model:

MapBiomas data is public, open and free under a Creative Commons CC-BY-SA license and with reference to the source, observing the following format : "MapBiomas Project [Territory Name] - Collection [version] of Maps [Territory Name] [Theme Name], accessed on [date] through the link: [LINK]"

• If there is a scientific reference to the data, this must be cited in the informative note, including the DOI, when available.

This section describes the Scientific Advisory Committee functions, meetings frequency, records of recommendations, and meetings with experts and stakeholders.

• According to the network structure of each territory, as the mapping is consolidated throughout the collections, a scientific assessment committee (Scientific Advisory Committee - SAC) with well-known experts from the country/region may help with improvements to new collections and future steps;

• It is recommended to maintain a minimum annual meeting frequency with the SAC, close to the launch of each new collection;

• It is recommended to keep a record of documents with responses to SAC recommendations and make them available to network members;

• It is also important to hold meetings with experts and stakeholders in the country to present the product and gather impressions about improvements and demands for new information, also ensuring that the data is better understood and used.