No matter where organizations store their data quality—whether in data lakes, warehouses or cloud platforms—managing that data is always a challenge. Organizations recognize the power of enterprise-wide data governance to better manage data, mitigate risk and maximize data value, but the importance of integrated data quality is often overlooked.
Declining Data Quality
As businesses continue to create and ingest data at exponential rates, data quality is increasingly at risk. By the time companies recognize declining data quality, reputational, regulatory and operational damage has already been done. Consequently, business users no longer trust either data or analytic results. Poor data leads to inaccurate decisions, errors and wasted time and money.
Data Quality Process
The integrity of data is paramount to successful data-driven initiatives. Accurate, consistent and trustworthy data is used to produce valuable business insights, mitigate regulatory risk, achieve operational efficiency and increase organizational value. Bad data system wide and across the data supply chain can only derail business initiatives and poses significant operational, financial and reputation liabilities.
To effectively confront data integrity challenges, companies need to not only establish a comprehensive data governance framework, but also prioritize data quality.
Integrated Data Quality and Data Governance
There are obvious synergies between data governance and data quality. Data governance is an effort to officially align people, processes and technology to empower businesses to utilize and take advantage of data. On the other hand, data quality is a calculation of data’s accuracy, completeness, consistency, trustworthiness and usability. Combined, they ensure that all data users can quickly find, understand, use and trust data to enhance decision-making, achieve business objectives and drive ROI.
An enterprise data governance program with an integrated data integrity strategy provides end-to-end data quality from data creation through consumption. Data integrity builds user trust among IT and business users who are using data as a basis for critical decisions. Comprehensive quality rules between sources and systems assure that data integrity is maintained throughout the data supply chain.
As the business and technology landscape rapidly change due to increasingly burdensome regulatory requirements and explosive growth in transactional data, companies require a streamlined and scalable approach to data quality.
Implementing a Streamlined Approach to Data Quality
Success with data requires a well-designed and sustainable data governance program that features integrated, comprehensive data integrity initiatives. There are five critical steps that organizations must follow to incorporate data quality into their data governance initiative, including:
- Discover: This step is about identifying critical information flows from data provisioning systems, including external source systems, along with their data lineage, to develop metric baselines. These metric baselines may be established by profiling the data to get a sense of what the data looks like. Once the baselines are established, source and target system owners must work together to establish data integrity criteria and data quality measurement metrics for mission-critical data elements.
- Define: Using the information gathered in the discovery phase, organizations must assess data quality risk by identifying and assessing critical data quality issues, pain points and dangers. Once the risks are evaluated and prioritized, companies must determine which data quality rules to include within a robust data governance framework.
- Design: In this phase, organizations must design information rules and exception management processes to address the risks uncovered during the “define” phase. Businesses should use automated rules to avoid sampling errors and increase efficiency. Rules are designed to either capture data in real-time or batch depending on the process requirements.
- Deploy: Companies must identify the most severe risks, determine which rules or actions to deploy and prioritize accordingly. Considerations for phased rollouts or support structures are decided and implemented during this phase.
- Monitor: During the monitor phase, business owners utilize enterprise reporting to provide visibility and monitor the data quality indicators established during “discovery.” With clarity into the data quality process, companies can identify opportunities for improvements by analyzing the micro trends in the data integrity indicators.
Data quality improvement requires a continuous operation involving people, processes and technology that works in collaboration to turn data into meaningful actionable business intelligence. Data quality-powered data governance is key to making the most of an enterprise data quality program and cultivating success with data.