Today’s organisations are creating and storing more data than ever before – data which often holds enormous business value. As volumes increase, however, the challenge of discovering and managing that data grows exponentially, particularly when compliance and data governance are so essential.
Discovering personal information within a vast data landscape is a challenge we see many clients struggle with regardless of their size or industry. With data privacy laws like South Africa’s Protection of Personal Information Act, there is increasing awareness of the individual’s (or juristic person’s) rights with regards to their personal information including the right to access whatever personal information is being processed. Unless organisations are able to discover all locations where personal information is stored, responding to these so-called Data Subject Access Requests is going to be extremely challenging.
Navigating this data security tightrope has become such a pain point for enterprises that many would rather lock their data away than risk introducing governance or compliance shortfalls through its improper usage.
Now, thanks to Microsoft, there’s a much better solution available: Microsoft Azure Purview.
What is Azure Purview?
According to Microsoft, Azure Purview is a “unified data governance service that helps you manage and govern your on-premises, multi-cloud and software-as-a-service (SaaS) data.”
Essentially, Purview enables organisations to automatically discover and map their data landscapes across a variety of sources, classify sensitive information and trace data lineage reliably from a single, centrally managed platform.
This makes it a far simpler task to ensure the entire data landscape is not only secure and compliant, but also searchable and accessible to the right users, informing intelligent, well-substantiated business decisions.
How does Azure Purview work?
Azure Purview has three main components: a data map, a data catalogue and data insights.
1. Data Map
Azure Purview’s data map is responsible for scanning the organisation’s data estate to extract a wide range of metadata across a growing list of supported sources. These currently include:
- Azure’s native data and storage services
- Database like Hive Metastore, Orcale, SQL Server and Teradata
- Azure’s native data and storage services
- Databases like Hive Metastore, Oracle, SQL Server and Teradata
- PowerBI
- SAP ECC and SAP S4HANA
- Amazon S3
Extracted information includes technical and operational metadata, lineage and classifications, all of which are used to populate a comprehensive data map.
Scans operate serverlessly, on a pay-as-you-use basis, and do not move or otherwise influence the source data in any way.
In reality, many clients we encounter do not have a single location for their often business-critical data. Many clients have journeyed to the cloud in a phased approach leaving vast volumes of data on premises. Alternatively, some clients have elected to retain data on premises for regulatory or security purposes. In addition, specialised systems such as Oracle and SAP often leverage their own data repositories (either on premises or in the cloud) with the result that clients’ data landscapes are varied and expansive.
Technology that enables the organisation to search and manage data across these varying repositories while not requiring the data to actually be moved into a single environment is a critical part of discovering the data you have without impacting the security or integrity of the entire data landscape.
2. Data Catalogue
To make it easier for organisations to understand the data they’re holding and where it is stored, Azure Purview indexes all the metadata discovered during data mapping and compiles it into a comprehensive Data Catalogue.
This becomes a central platform (protected by role-based access control) from which users can search and browse the organisation’s entire data estate to surface trusted information, identify its sensitivity level, and understand its lineage and business context.
Since the digitisation of information, and because of the ease with which data is created, users have contributed to an unfettered expansion of organisation’s data estate which, as a consequence, has become harder to manage.
With regulatory requirements and the need for good data governance, organisations can no longer turn a blind eye to this data spread.
Understanding the landscape is necessarily the first step to managing it. Being able to use data analytics tools like Power BI to analyse the metadata of the information allows organisations to extract invaluable intelligence regarding the data and to manage the data from an informed standpoint.
3. Data Insights
Azure Purview doesn’t just provide centralised access to a data estate. It also delivers insights into the data that enable businesses to better their compliance and security stances and extract more value from their business information.
This includes interactive data lineage and data sensitivity visualisations that make it easy to pinpoint potential security flaws and/or compliance issues, as well as nifty tools like a curated enterprise-grade glossary of business terms that eliminates the need for data dictionaries.
Good to know
Azure Purview is a new release that is undergoing constant improvement. We’re particularly excited for the governance capabilities coming down the pipeline that promise dramatic simplification of the process of applying blanket governance policies across multiple sources.
As powerful as Purview’s governance capabilities promise to be, however, we do need to stress the importance of a holistic approach to enterprise security, governance and compliance. Technology can only go so far without the right people and processes in place.
Before diving into using the technology, it is essential that organisations understand what they seek to achieve from the technology and formulate a sound strategy for its use based on their specific business requirements and regulatory obligations.
To find out where you stand on the security and governance spectrum and get advice on leveraging technology like Azure Purview to enhance your overall posture, get in touch with a Cloud Essentials expert today.