Implementation
Automating the delivery
Metadata management, code generation, and DevOps are introduced by Jeff Jones, who brings invaluable experience to the table.
Combined with the implementation patterns created by Amal Leyla Qasim, the complete solution can now be delivered.
For the creation and management of the data platform, design metadata is everything.
The data lineage, the physical model, the data logistics processes - these are all governed by a relatively small amount of details that can describe how data is meant to be used.
Design metadata, further referred to as just ‘metadata’, is a succinct and technology-agnostic collection of conventions, definitions and instructions about which data elements need to be mapped, where they are mapped to, and how they need to be used.
This metadata can be seen as the DNA of the data solution, because it holds the essential instruction to fully (re)build the data solution into its physical form. Using strands of metadata, parts or all of the data solution can be completely regenerated.
In practical terms, metadata is the ingredient that goes into DevOps workflows, infrastructure, physical models, and the creation of data logistics processes. Together, these allow for the data solution to be automatically generated, tested, and deployed.