As organisations become increasingly reliant on technology, it is imperative the data generated and consumed by internal systems flows freely across system boundaries and is managed proactively at the enterprise level. This post provides a high-level coverage of Master Data Management a topic relevant to organisations of any size that integrate data between enterprise systems.
Master Data Management (or MDM) addresses the data integrity issues related to master data (e.g. Customer) being generated and consumed by multiple systems across the enterprise, each applying a different set of policies and a proprietary structure to the data. Without MDM processes in place, it is likely that each system will have an isolated view on the same data; this situation could negatively impact on the customer relationship where demographic updates aren’t reconciled across systems or duplication exists, reduce the accuracy of management information or introduce regulatory and compliance issues. As such the primary objective for MDM is to deliver a single version of the truth for master data records that spans the enterprise.
The bullet-points below provide a high-level categorisation of the data held by organisations. Note, there are further categories such as analytical and hierarchical that aren’t directly relevant to this context.
Enterprise data categories;
- Master : The common entities that support core business processes across the enterprise. Customer, Supplier and Product are typical examples.
- Reference – Static reference data such as countries, regions, categorisations.
- Transactional – Records that represent a specific transaction with a master data record, i.e. Customer Order, Sales Opportunity etc.
- Unstructured – Emails, files etc.
Taking the definition provided above, it is clear that master data is a highly valuable data asset and should be proactively managed as such. The term management in this context can often be interpreted to imply an after-the-fact management of data once an integration solution is established, i.e. a process issue. This is a common oversight; instead a clear master data design should be developed during the logical integration design phase – the two concerns are inextricably entwined. Ideally the master data design should be defined at the enterprise level and be subject to change management; new data integrations should adhere to the global scheme not introduce point solutions.
In considering the master data design, a clear specification is required that encompasses all data sources per-entity and ownership of data at both the entity and attribute level. The design should also clarify system specific handling of non-owned data (how are records categorised, are records read-only?), identifier mappings and system behaviour in respect to record creation, modification and deletion operations. Designation of an authoritative system of record at the entity level is often impractical, instead ownership may be applied at the category level. For example, ecommerce customers may be mastered in the ecommerce system while in-store customers are mastered in the ERP system. It is therefore the case that master data designs range in complexity from simple entity-level ownership, through to highly complex attribute-level schemes with horizontal record partitioning. A keep-it-simple, tiered approach can help in maintaining focus and avoiding the perception that MDM is simply too difficult and time consuming to get right. In all but the most trivial cases, the day-one solution should focus on the essential elements only, with the broader picture emerging incrementally over a series of iterations. This will inevitably require some degree of compromise from the business stakeholders involved.
The following points outline the main benefits to implementing a proactive approach to master data management.
- Accurate Common Data
- Operational Insight / Decision Making
- Data Validation
- Versioning & Auditing
- Hierarchy Management
This is the primary objective for MDM; master records should be de-duplicated, accurate and consistent at all points of consumption across the enterprise. How this goal is achieved varies greatly across implementations, but the fundamental principle should never change. High quality master data provides the foundation for operational efficiency and effective relationships with external parties.
With an MDM solution in place it should be possible to correlate all touch-points with an external party (e.g. Customer, Supplier) against the master record regardless of which system records the relationship or related transactions. This consolidation of enterprise knowledge, often referred to as a Single Customer View provides the basis for highly informed decision making, but is contingent on the mapping of master records across system boundaries.
A key function for an MDM solution is the enrichment of the master records through validation against external data sources. For example, address data or bank details may be validated against a 3rd party source.
An MDM solution should support version management for master records to provide an auditable history of data attribute changes over time.
In addition to entity and attribute management capabilities, the MDM solution should provide flexible hierarchy management features that allow data to visualised (and aggregated) across hierarchies that exist within any enterprise system.
The benefits of MDM can be significant in terms of operational efficiency and insight, there are however a number of challenges to be mitigated in delivery of this business utility.
- Data Governance
- Master data management must be viewed as part system automation and part data governance process. The process element will typically require dedicated, specialist resource to manage data stewarding tasks such as data quality auditing, candidate record match validation and general oversight and management of the master data domain.
- User Experience
- Single instance
- Multiple instance – reference
- Multiple instance – merge
Master data management projects are often be viewed as highly complex, enterprise-level, strategic projects. Many organisations will have such a project on the roadmap as a future item, gaining business support can be challenging when faced with pressure to deliver tactical solutions. For this reason, tactical projects can be tempted to ignore master data considerations. MDM projects that do make it to the initiation stage must adopt an incremental delivery style and present a programme of work; where small evolutionary steps systematically take the organisation toward the desired future state. Again there can be a challenge here in terms of clarity of vision; the complexity and cost can simply look disproportionate to the return.
Master data management can require specialist tools to perform tasks such as record normalisation, matching and merge. Such tools can be expensive in respect to both procurement cost and the time required to mitigate the learning curve.
Master data management processes may introduce constraints in regard to which systems are able to apply data modifications. In cases where the systems in question can’t reflect such a constraint and allow edits to be made that are subsequently overwritten, there can be a negative impact on user experience. Wherever possible such a scenario should be avoided – records and attributes should be read-only where modifications are not permitted.
MDM Implementation Models
Master data management solutions can vary greatly in terms of the tools, techniques and practices applied. In high-level terms however there are three primary implementation models to consider.
The master data records exist in one location only and are referenced in real-time by consuming systems (SOA for example). This model is contingent on the provision of an efficient, real-time data interface (API etc.). Consuming systems may cache master records for a short period.
Master records are held by consuming systems in read-only form. Master record modifications are made directly to the master system exclusively, with updates synchronised to the consuming system periodically – or on-demand. In such a model end-users may require direct access to the master system, or may request updates offline.
Master records may be modified by consuming systems; updated records are then subject to record matching and merge in the master system to determine the correct master record relationships. The reconciled master record state is then synchronised back to the consuming system. With this model, all record modifications invoke a full re-evaluation of the record – this can be an expensive operation at scale.