Embedded Files
The Data Dictionary is essential for supporting the FAIRification of your data and the relevance of your project after delivery. The main objective is to provide a relatively standardised, human-readable description of the data utilised in the project, alongside their relationships and lineages throughout the analytical process. Noting this down provides the contextual metadata which can be critical for a customer, future colleague, or even yourself to understand what you've done with the data, and how you did it.
Data dictionaries can take many forms, and the gold standard would likely be integrated programmatically and alongside a fully interoperable data ontology. For a practical implementation in the DCMW, we have distilled the key elements into an accessible Excel workbook, which you can use to track and link a variety of data items in a reasonably standardised way.
General Usage
General Usage
The data dictionary is split across a number of tabs to help you organise your metadata. Navigate between these tabs and populate them with content as you progress with your project.
Each worksheet contains a table which you can use to record key metadata which is relevant to the project.
Nomenclature
Nomenclature
The Nomenclature table is included to track common language terms, their acronyms and their meaning in the context of the project. It is worth noting that some customers use different terminology to describe the same concepts, and new terms or acronyms may be generated during project scoping and delivery.
Nomenclature table
Each table in the data dictionary contains a legend (upper right) to clarify the meaning of each column.
Table legend
Assets
Assets
The assets table is provided to track all the key assets used to generate (or consume) data in the project. This might be a machine tool, robot, measurement system, bespoke experimental test rig…any physical or virtual object which creates the data you will analyse in the project. Entries in the first five columns are used to auto-fill the Reference name and Reference code for each asset, which can be linked in the subsequent tables.
Assets table
Datasets
Datasets
The datasets table provides a top-level dictionary location to track all the datasets which have been generated or utilised throughout your project. It is up to you to define what constitutes a dataset in the context of your project, it could be independent files stored locally or alternatively could be a specific time period and selection of metrics stored in the Factory+ historian. The key objective in this table is to give clear details on the relevant datasets and ensure that they can be both found and understood following close of the project.
Datasets table
The Datasets table gives a high-level summary of the utilised datasets, and should be completed in parallel with the data, feature and model reporting process integrated into the DCMW. This level of detail (don't worry, the template is designed to reduce the amount of effort required on your part!) should help to clarify the data used in the project, deepen the exploration and ensure the research is accessible and reusable following project close.
The maturity level of the dataset is designated as Bronze, Silver, or Gold, in accordance with the Medallion Architecture. See the link below for an explanation of this and what it means for the DCMW.
Variables (Bronze, Silver, Gold)
Variables (Bronze, Silver, Gold)
Ideally, you might want to assign metadata for your variables directly on the data source, if this is an option with the data management tool you are using - it is possible to define metadata tags per column for tables in Unity Catalog, for example; utilising such a mechanism would likely be the most effective way to ensure your data adheres to the FAIR principles. Not all projects at this point in time will necessarily have access to this though, so to address this gap, the final three tabs contain tables to manually track metadata of individual variables within your datasets.
The tables are split into the three maturity levels to reflect the layers defined in the Medallion Architecture. Keeping a metadata record of key variables, their attributes and relationships with one-another can massively streamline the process of accessing and extracting value from data, especially for someone who hasn't already spent a lot of time studying it.
Some columns are validated based on previous table entries (Asset, Dataset and Report) and some are auto-filled with entries in these tables (e.g. Storage location/Unique string and Reference code).
Variables (Bronze) table
Variables (Silver) table
Variables (Gold) table
Page updated
Report abuse