Achieving Data Lineage in 2023

With data being widely considered to be the lifeblood pumping through the beating heart of the modern financial world, there is a sore need for an increase in the transparency and accuracy of digital information.

Being able to trace the history of all data used by a firm is critical to establishing a trusted chain of information which can be checked at any time and by any stakeholder in the process. With digital information being used daily to drive business critical decision-making processes, it is more important than ever to be able to rely on the accuracy and heritage of that data.

We call this process data lineage and it’s going to be increasingly important for your organization to achieve it if you want to remain a key player in the digitally transformed and data powered financial industry of the future.

Data Lineage Benefits

"In modern business intelligence projects, understanding the flow of data from the data source to its destination can be a challenge,” says Microsoft. "The challenge is even bigger if you have built advanced analytical projects spanning multiple data sources, artifacts, and dependencies. Questions like "What happens if I change this data?" or "Why isn't this report up to date?" can be hard to answer. They may require a team of experts or deep investigation to understand. Data lineage helps you answer these questions.”

Not only will establishing data lineage help your organization answer these questions, but it will also empower you to achieve many other benefits as well.

We all know how increasingly draconian the regulatory environment is becoming in both the world of finance and data. Ensuring your firm remains compliant in this landscape requires you to be confident in how data is flowing through your systems, its origin, and its history. With an established data lineage strategy, you don’t need to worry about any skeletons popping out of the closet and falling your firm foul of GDPR, CCPA, BCBS239, and more.

"It is not easy to map data traceability for regulatory purposes,” says data intelligence cloud experts, Collibra. "It can take countless hours, and if it is wrong, it can result in fines and penalties. Data lineage supports the Risk Management and Data Governance teams by tracking how data flows through various systems from source to destination and enabling risk management to view the audit trail for all data transformations.”

Blockchain

No discussion of data lineage in 2023 would be complete without touching on the subject of blockchain.

The decentralized ledger technology is very well suited to enabling stakeholders within a value chain to trace the history of a data set. Blockchain is lauded in some circles for providing a secure and immutable record of digital transactions which can be accessed by any party in the process to validate and confirm the lineage of that information.

"There is no guesswork needed to understand the route that the data took to get to where it is today,” reports Chief Data Officer, Ari Cohen. "Everyone can see it and there’s no need for a central authority. Because every transaction on a blockchain needs to follow a strict protocol for a transaction to occur, we know that every data-point is accurate. The process is cast-iron. On top of that, the data is immutable. That simply means that you can’t erase or change the history. The protocol in place doesn’t allow it.”

Of course, blockchain isn’t the only method of achieving data lineage and is likely not even the best option available.

(Image source: techtarget.com)

As you can see from the above chart, there are several alternatives to blockchain technology which actually beat it when it comes to most of the desirable factors organizations are seeking when seeking technology suitable for data lineage. Whilst blockchain might be good when it comes to reliability and data integrity (undeniably a desirable quality for this kind of enterprise), it doesn’t outperform any of the alternatives apart from traditional cloud storage and lags in almost every other area.

"Despite the promise, blockchain adoption beyond proof of concept has been slow,” reports TechTarget. "One big issue has been the relatively slow performance of early blockchain technologies. The first blockchain networks were limited to a few transactions per second and could take up to an hour to guarantee the authenticity of transactions. Several alternatives to blockchain that provide better performance have emerged. Businesses may also want to consider them to reduce costs, simplify development and reduce integration challenges.”

Final Thoughts

Establishing data lineage is going to become increasingly important for digitally transformed organizations in the future. Increasing the reliability and trustworthiness of data will help firms better serve customers, make business critical decisions, and meet the needs of the regulatory landscape.

Data lineage is sure to be a hot topic at FIMA 2023, being held in April at the Westin Copley Place, Boston MA.

Download the agenda today for more information and insights.