The organisation’s core principles are to be managed and maintained through good data practices and strengthened through an iterative data lifecycle. Organisations should be committed to delivering a robust lifecycle approach to data, which links right back to their vision and objectives. The data lifecycle allows organisations to gain the trust in their data through the right oversight throughout its life. This will allow organisations to optimise our data’s usefulness, improving the information while minimising the potential for error. Finally archiving or disposing the data at the end of its useful life, will ensure we are compliant with applicable legislation and will reduce the consumption of valuable IT resources.
The data lifecycle will become an important process within organisations Data Management programme especially with the current data explosion from Big Data and the on-going development of the Internet of Things (IoT) that will become prevalent within any industry over the coming years.
The approach to data is iterative and cyclical, whereby it will need to continue and grow with the data, each cycle brings a new dimension and insight. In practice, it is not uncommon for a data stage needing to go backwards to the previous stage, typically this is completed through the Governance processes or realignment of business requirements, which essentially act as a review process and request for further definition and clarity. In effect, the data lifecycle can have mini-cycles within a few stages before moving on to the following stage.
Stage 1 -Specify data
One of the key principles of good data management lies in specifying what data we should keep and to what standard, to conduct our day to day operations of the business.
What ?
- What data do we seek to collect ?
- What format is the data required in ?
- Can we quantify the extent of the required data ?
Why ?
- Is there a regulatory/statutory or legislative driver?
- What is the customer benefit?
- How does it link to our performance commitments ?
How ?
- Are we clear on the methodology for data collection ?
- Has a similar exercise been done in past ?
When collecting new data, whether they are collected via automation or through human input, we endeavour to ensure we collect the appropriate metadata and information properties which are needed.
Metadata is ‘data about data’, its characteristics that allow effective and efficient referencing. A data object’s metadata can be thought of as a ‘fact file’ about it. This may include meaning, relationships to other data, origin, usage, format etc. Metadata is fundamental to successful data governance: it acts as a reference point to promote common understanding, describing what it means, specifying any relevant standards and clarifying how it is used within the organisation.
The above process is already embedded as part of our “privacy by design” approach to data protection, where we privilege the need to protect our personal data with the utmost care and privacy, particularly when delivering new digital solutions likely to yield data and information as input or output.
Stage 2 -Collection & storage of data
Data collection occurs early in the lifecycle based on the specified data in stage one. This data could be a raw data, file, image, or in document form. The information is typically entered or uploaded into an application and accessible to certain roles within the organisational hierarchy on whatever devices offer an access point to the proprietary system.
At this stage, it is important for those conducting data entry to ensure that the data they input or receive is as accurate as possible, this will limit the data quality issues downstream.
As part of a solutions delivery process, the data storage mechanism and its associated security, availability and recoverability is designed. Typically, this is implemented through a traditional database management system onto physical disk, although larger distributed data management solutions are changing this pattern. However, this design process provides valuable knowledge and accountability as part of our governance process ensuring continued business operations through strong architectural processes.
Stage 3 -Assess and improve/enrich data
Data quality is an essential element of any data strategy as it demonstrates the effectiveness of the data strategy and underpins the trust in our data inputs and outputs.
The “assess and improve” stage supports data quality by providing the “checks and balances” to our data, ensuring what we’ve specified in stage 1 matches what we’ve collected in stage 2. This stage provides the assurance that our data aligns to our requirements and governance controls, providing a measurable view of our data.
Any identified gaps in accuracy or completeness will be addressed, as this critical to the analysis phase. For instance, burst records maybe missing information on pipe characteristics and thus may need to be inferred. This stage provides the assurance that our data aligns to our requirements and governance controls, providing a measurable view of our data.
It is a necessity that we trust our data, and only though continually data quality validation can we gain the measures by which we have confidence on our data. Without this trust, the next stages of the data lifecycle cannot be trusted and as such, our data strategy would be ineffective.
As noted, this stage uses the information generated from stages 1 & 2 to validate our data, however, it also looks to provide the mechanism and guidelines to correct and continuously improve our data. Where data quality issues are found this should be highlighted back through the Requirements and Governance process and readdressed through stages 1 or 2.
Stage 4 -Analyse data
The Analyse stage supports the use of data to generate information and provide insight. Typically categorised under the Business Intelligence (BI) banner, this includes the reporting, dashboarding and analytical processing of our data into information and knowledge. Simply put, how we use our data to be informed.
Underpinning our analyse data stage is our hereditary need to optimise our “totex” management of the asset portfolio, whilst delivering great customer service. The techniques and services used during this stage of the data lifecycle have an inherent impact on our efficiency as a business from an Opex and Capex perspective.
We use a multitude of tools and data analysis techniques to manage information within our respective areas of the business. Much of this is generated thanks to common statistical techniques and modelling tools which provide the best level of information management possible.
However, we also rely heavily of ad-hoc analysis of structured data, through business intelligence platform services for reporting and analytics dashboarding. These tools enable us to explore the data in more detail utilising descriptive statistics techniques (mean, mode, median, and frequencies) to understand current trends, with more qualitative data analysis, particularly when examining unwanted contacts from customers.
Stage 5 -Share data
This stage of the data lifecycle relates to making data and information derived from the previous stages available to our people, partners and customers in a secure and robust manner.
One of our goals is to support our regulators view which aims to:
- Develop a collaborative network to share best practice and develop ideas;
- Champion innovative uses of data within your organsiation/industry;
- Use data to improve customer service and support to those who need it most.
- Improve operational resilience;
- Be more transparent, which in turn drives accountability and legitimacy building trust;
- Develop materials and support to help sell the benefits of data sharing internally.
To get the most value from our data and drive different behaviours, it is key to keep our people informed. Through the application of technology, we will be able to share our information to support our operational teams, so they can work as efficiently as possible in a safe working environment.
Stage 6 -Archive and Destroy
Stage 6 of the data lifecycle supports our responsibility to ensure our data and information is managed and destroyed in accordance with legislations and customer expectations.
We adopt a robust data archiving and destruction policy which ensures that retired devices and media have their contents securely removed, destroyed, or overwritten and physical documents are destroyed appropriate to their security classification