Get the new Gartner® Market Guide

Visualize Snowflake Horizon and enhance its impact

Snowflake Horizon is Snowflake's built-in governance solution that unifies compliance, security, privacy, interoperability, and access capabilities enabling customers to easily govern data, apps, and more across clouds, teams, partners, and customers.

Data lineage: The cornerstone of modern data governance

Explore the various aspects of data lineage, and understand its crucial role as a strategic ally for your organization.

Achieving Basel III compliance: A 3-Step action plan

Basel III is changing – are you prepared? In 3 easy steps, Solidatus enables banks to meet Basel III simply, and efficiently.

Solidatus Joins Forces with Artefact to Expand Consulting Partner Ecosystem

Solidatus partners with Artefact to enhance the value of data management practices and provide data-driven solutions to customers. 

HSBC & Solidatus Win at Banking Tech Awards 2023 from FinTech Futures

We're excited to announce a significant win at the Banking Tech Awards 2023 from FinTech Futures for Best Use of Tech in Business Lending

amazon web services solidatus partnership square

Solidatus joins the AWS Marketplace to bring connected governance to AWS customers

Solidatus, leading data lineage provider, is now on AWS Marketplace. AWS users can use Solidatus for effective governance and risk management by bridging cloud, on-premises, and hybrid systems on one pane of glass.

10 May 23

Douze points for new way to visualize Eurovision data

Visualize Snowflake Horizon and enhance its impact

Data lineage: The cornerstone of modern data governance

Solidatus Joins Forces with Artefact to Expand Consulting Partner Ecosystem

Come closer, come closer and listen.

The beat of my heart keeps on missing.

“Listen to what,” you might reasonably ask, “and, more importantly,” you might go on, “what on earth has your heart got to do with it? Isn’t this a blog about data?”

Well, in a world-first – and, we imagine, much to the joy of 1969-era Lulu – we at Solidatus have linked the Eurovision Song Contest to the realm of data governance and regulatory compliance.

Boom bang-a-bang!

Reality check

“That’s great but so what?”

It’s a fair question. The answer is that this genuinely fascinating work doesn’t just reveal otherwise hard-to-find insights into Eurovision across the ages; the upcoming song contest has provided us with an excuse to develop some models that beautifully illustrate the importance of lineage and how powerful it can be to properly map and visualize data, or rather metadata, whether for business or pleasure.

So back to Eurovision. And in a move that we’re praying won’t equally alienate its two very different target audiences – Eurovision fans and data professionals – we’ve taken a range of rich and granular datasets stretching back to Eurovision’s founding contest in 1956 and fed them into a versatile piece of software that’s more usually used by people working in complex multinational banks and other big businesses.

In this blog post, we’ll:

Explain how these models work;
Provide access to them so you can dig around in them yourself;
Highlight some of our sample findings; and
Show the parallels between this work and more typical use cases of this software, such as data governance and regulatory compliance.

But first, and to whet your appetite, here are some sample Eurovision findings, upon which we expand later:

Cyclic voting graphics reveal patterns of pairs of countries each awarding the other top marks – and stats that will be familiar on Greece and Cyprus’s history in this practice.
In the longest period of consistent voting methodology (1975 to 2015), the highest-scoring winner scored more than three times the points of the lowest-scoring winner, revealing what is arguably the most successful Eurovision song of all time – see who it is below.
The UK’s belief that it’s getting progressively worse is exaggerated, and we have the analysis to prove it.

But how did we get there? Let’s first take a look at what a Solidatus model is.

A note on Solidatus models

Solidatus models aren’t databases and they don’t store data, or at least not the primary data you’d find in a typical database, with row upon row of similar information on names, addresses, dates of birth etc on thousands of similar records. Rather, they display metadata – data about data – through visualizations that enable users to see how data and systems relate to each other, and how data flows between them, its journey and how it impacts with other data.

In the case of Solidatus, we can meaningfully and with justification describe this metadata as ‘active metadata’, a concept you can read about in our blog post, From data to metadata to active metadata. You can read about other concepts in this field in Key concepts in data governance and managemeny: an A to not-quite-Z guide.

But lest we stray off topic, let’s take a quick look at what a model looks like.

Below is a section of a typical model, not dissimilar to those we’ve used for our Eurovision research. This one, though, was built for a more typical business use case. (We cater for many solutions and sub-solutions.)

A Solidatus model comprises:

Layers – these are effectively columns. User-defined, they provide a way of grouping together things that belong in the same broad category, whether by time, sequential ‘position’ in a chain of systems or however the user sees fit; in our three Eurovision models, we layer by year, country and placement (see below).
Objects – these are what you might describe as a primary record. In a normal Solidatus use case, they might represent a database or perhaps a table in a more complex database. Users set their own hierarchy.
Attributes – these are more granular details of objects.
Transitions – these are a special type of metadata, shown here as arrowed lines between attributes in different layers, but they’re also recorded in a model’s ‘Relationships’ panel. Typically, they show the flow of data between systems, but they can be used to describe any relationship.
Properties – these allow you to dig into the weeds of attributes, with information being added or views through the ‘Inspector’ tab on the right-hand side of the screen, or on an object or attribute.
Relationships – also available through the ‘Inspector’ tab, these show relationship information (transitions) in a tabular form.

When viewing a Solidatus model, bear in mind:

All of the rich info that can be found in the aforementioned ‘Inspector’ panel, which can be exported into CSV files.
A range of views are available on the left-hand side, based on what the model builder has built.
The show trace view and ‘target’ icon will help you isolate flow-related information on specific object and attributes.
As with our Eurovision models outlined below, skilled model builders might have set up display rules – here, for example we have gold, silver and bronze boxes around entries that were placed first, second and third, and small arrows in country entries that show whether they did better or worse the following year

And finally, a note on how this information is actually brought into Solidatus in the first place:

This is the clever bit. Models can be built manually, and there’s usually manual intervention. But we also have a series of connectors that can automate much of the process.

In the case of this project and alongside some other data repositories, we drew a lot of the info from Wikidata, the central storage area for the structured data of Wikimedia and all the many records on Eurovision it holds. And the connector we built for wider Wikipedia-related metadata ingestion and used for this project is for a query language called SPARQL.

Because here’s the thing: Solidatus doesn’t deal in data that’s unavailable to you if, crucially, you know where to look; rather, it elevates it into a more visually digestible environment, where it can be interrogated in a meaningful context.

But enough of the sales pitch. Let’s dive into the models!

Our three models

By year
This model shows layers arranged by year:

In this view, we’ve scrolled to the right of the screen so that the years 2020, 2021 and 2022 can be seen. But there’s more to the left, going back to 1956, and more below – in each layer, the countries are arranged in descending order of votes garnered.

Here, we’ve clicked on Ukraine in 2022, which shows a transition line from its position the previous year (and the one before that and so on), and changes the focus of the ‘Inspector’ on the right. This panel shows info such as:

The artist’s name (Kalush);
The song name (Stefania);
The language it was sung in (Ukrainian); and
Its genre(s) (contemporary folk music and, if you can believe it, hip hop).

If a user clicks on ‘Views’ on the left-hand side of the screen, you can also isolate data along the lines shown below, as built by our data modeller.

Cyclic voting, based on who gave whom the top score of douze points, is an interesting one to explore, as illustrated by the transition lines here for the years 2013, 2014 and 2015:

So, when you dig into the model, you can see, for example, that in 2013, Sweden’s judges awarded their two top scores to Denmark and Norway, whereas their entry received only one top score, which they got from Norway’s judges.

By country
This model shows layers arranged by country:

In this view, we’ve scrolled to the left, where we can see the first three countries by alphabetical order – Albania, Andorra and Armenia. By layer, each object is arranged by year, so Albania, for example, first took part in 2004.

At random, we’ve clicked on its record for 2008, the card itself showing that it came 17^th that year. The transitions lines pointing into it show which countries (to the right of the model) gave it points. And in the ‘Inspector’, the usual info is available along the lines of the bullets listed in the ‘by year’ model.

‘Zemrën e lamë peng’ was its entry’s catchy title, for example.

By placement
This model shows layers arranged by placement from first to tenth:

Here, we’ve clicked on Sweden’s winning entry in 2015, the transition lines showing whom Sweden gave its votes to and who voted for its winning song, Heroes, sung in English by Måns Zelmerlöw.

And the opportunities for data analysis just go on.

The point of this exercise is to illustrate the richness and granularity of this easily visualizable data, something that of course has more practical applications in the world of big business, rather than to home in on any particular stats. Nonetheless, we feel compelled to highlight a handful of key findings, which you can supplement with your own digging around.

Some sample findings

By choosing the ‘Cyclic Voting (top points)’ view in the ‘by year’ model, we can see that pairs of countries all gave each other 12 points:

The UK and Switzerland, and Italy and Ireland in 1976, the first year this phenomenon arose;
Cyprus and Greece in 1986, 1987, 1994, 1997, 1998, 2002, 2003, 2004, 2005, 2010, 2012, 2017 and 2019; and
Many others we won’t list but you can find yourself, which might raise an eyebrow, given the geopolitical landscape now and in the past.

Cyclic voting in 1976, 1978 and 1979, extracted from a wider model

By exporting the ‘Inspector’ info from the ‘1st’ layer in the ‘by placement’ model, we can see that:

These are the most successful languages in terms of winning song: English (with 33 wins), Dutch, Hebrew and Italian (all with 3 a piece), German, Norwegian, Serbo-Croat, Spanish, Swedish and Ukrainian (all with 2), and Crimean Tatar, Danish and Portuguese (with 1 each);
Grand final points for the winning entry have a range of 18 all the way up to Portugal’s barnstorming 758 in 2017, but the huge disparity is in part explained by changing point-awarding scores over the years;
To conduct a more meaningful analysis, we could look at the years 1975 to 2015, the longest period of consistent methodology, for which the range is 123 (Norway in 1985) to 387 (also Norway, this time in 2009)
On that last point, with more than three times the score of the lowest-scoring winner in this 40-year period, there’s an argument that Alexander Rybak’s Fairytale (also sung in English) is the ‘best’ song in Eurovision history, although, despite moderate chart success for his song, Rybak is hardly a household name beyond Norway’s shores; and
The winning song with the shortest title is Netta’s Toy, Israel’s entry for 2018, and the longest title is Poupée de cire, poupée de son, sung by Luxembourg’s France Gall in 1965, who demonstrated that French is sometimes better than English, which would have rendered the song Wax doll, sound doll.

The United Kingdom’s sense that it has done progressively worse in recent years (last year’s second place aside) is exaggerated, given the increasing number of participating countries. The two graphics below show:

A random portion of the ‘by year’ model, in this case for the period 1964 to 1968, which shows how easy it is to trace the progress – or lineage – of the UK’s performance; and
A graph derived from this model, which shows the UK’s absolute position alongside a line indicating the percentage of countries that finished above it. While the trend has been downwards, the orange line has been more constant because the number of countries competing has gone up.

Lineage on the UK’s position from 1964 to 1968 inclusive, extracted from a wider model

The UK’s position (blue) and percentage of countries that finished above it (orange) vs year from 1957 to 2022 (with the years it didn’t compete removed)

But take a look at these models yourself – and if you happen to be a journalist writing about Eurovision, give us a shout at hello.marketing@solidatus.com and we can walk you through our work.

Lineage, metadata and the world of data compliance

Now, if you’re a data professional with little interest in Eurovision, we’re grateful, frankly, that you’ve stuck with us. Maybe we’ve converted you along the way.

But let’s bring this back to the real world, or at least your world.

The beauty of a visualization tool like Solidatus is that there are virtually no limits to the applications its graph technology can be put to, all of it exploiting and promoting active metadata.

We have, though, found that Solidatus particularly lends itself to these solutions: governance and regulatory compliance; data risk and controls; data sharing; business integration; and environmental, social and governance (ESG).

We’re going to end with a quick review of governance and regulatory compliance.

Using Solidatus, you create living blueprints that map how your data flows – a.k.a. lineage – as it moves through your systems – both now and at other points in time. You can connect your data to the processes that create it, to the policies that guide it, and to the obligations that regulate it. With this framework in place, you can maintain transparency across your business, meet ever-evolving regulatory requirements, and accelerate change programs.

That’s the boilerplate. But what does it mean in practice?

Well, let’s finish with a few excerpts from our recently published case study, Solidatus models HSBC’s global lending book (PDF), this use case – alongside many others, including business integrations, and data risks and controls – being a key component of their several objectives.

In under six months, a team of two was able to document and model the global bank’s entire credit and lending book, demonstrating traceability from source to consumption. They now have a highly scalable and automated solution that is being applied to several applications from ESG to liquidity calculations and other regulatory uses.

Do read the case study (PDF) to see how they reduced a project’s cost from $5,000,000 to under $500,000, a saving of more than 90%.

And don’t let inefficient data management practices be your Waterloo.

Latest news

Visualize Snowflake Horizon and enhance its impact

Data lineage: The cornerstone of modern data governance

Explore the various aspects of data lineage, and understand its crucial role as a strategic ally for your organization.

Achieving Basel III compliance: A 3-Step action plan

Basel III is changing – are you prepared? In 3 easy steps, Solidatus enables banks to meet Basel III simply, and efficiently.

Solidatus Joins Forces with Artefact to Expand Consulting Partner Ecosystem

Solidatus partners with Artefact to enhance the value of data management practices and provide data-driven solutions to customers. 

HSBC & Solidatus Win at Banking Tech Awards 2023 from FinTech Futures

We're excited to announce a significant win at the Banking Tech Awards 2023 from FinTech Futures for Best Use of Tech in Business Lending

Solidatus joins the AWS Marketplace to bring connected governance to AWS customers

13 Apr 23

Lineage is reality: plug the gaps, plan to succeed

Visualize Snowflake Horizon and enhance its impact

Data lineage: The cornerstone of modern data governance

Solidatus Joins Forces with Artefact to Expand Consulting Partner Ecosystem

By Philip Miller, Co-Founder and Chief Innovation Officer

In the first part of this series on data lineage, we talked about how lineage is a tool for context and how if we use it first, we get better outcomes. In this second instalment, let’s take this a bit further and say that without lineage, governance is a false economy.

I consider myself a bit of an all-around geek, not just of data but of technology in general and of ideas. There is power in ideas, which is why one of my favourite films is Inception.

You’ll see how this relates to lineage and your work in a moment. Until then, please bear with me and enjoy the ride.

Back to Inception

For the uninitiated (how?), Inception’s premise is that there can be a tiny moment of inspiration that can affect everything. Another is one is The Terminator (and Terminator 2 – that is as far as I am willing to go!), where the protection of one single life echoes into the future.

These two films have much in common, but they clearly demonstrate the value of understanding reality.

To an extent, both deal with time and the proposition of ‘what if’, raising questions such as…

Inception: what if I question and change someone’s reality?

The Terminator: what if I could kill Hitler before he committed his crimes?

Christopher Nolan, director of the first, spent almost ten years polishing off a grand idea and created a clear closed loop in which he could simply explain the whole premise.

His original diagram is shown below (there is a cleaned-up version at the end):

For those unfamiliar with the film, there are many levels of simulated realities in which the hero characters have to unlock the secrets to a character’s deepest desires. There is a messiness as one of the characters (Cobb) hasn’t disclosed some of his own secrets, leaving the reality at risk.

In The Terminator a ‘mad’ computer decides to wipe out his only nemesis by killing his mother before he was born. The computer goes into this with partial information and in the end ensures that his enemy is actually born – whoops. From there, the Terminator film series goes on to prove what happens when you don’t plan properly. As you can see below, there is a real confusion about reality, and anyone trying to pick up the pieces and add in more lore is left with less and less sense.

Back to lineage

Let’s bring this all back to the point in hand – lineage.

Lineage is an expression of reality, it is a record of what is actually happening. If you take time to build up context as you are going along, you find yourself in a good place where you can show what’s actually happening around you.

The Law of Unintended Consequences comes into play here – The Terminator’s mission is to travel back in time and kill Sarah Connor, the mother of John Connor, who is destined to lead humanity in a future war against machines. By doing this, he hopes to change the course of history and prevent his own destruction at John’s hands. The Terminator didn’t have enough information and brought about its own demise. It was missing reference data (context) – the identity of his foe’s father!

Lineage is important to the film Inception because it helps to explain the origins of the characters and their relationships. It also provides a sense of continuity, which is essential when trying to piece together the complex plot of the film. Ultimately, Cobb didn’t disclose all his relationships to his teammates and created a situation where he created risk to his mission.

Data lineage applications

Now let’s apply these lessons.

There are relationships everywhere; they can be simple, they can be complex, but they all have context. When you spend the time to build up these connections, you find that you can describe what’s going on better and better. The less in the way of holes there are in your lineage view, the more chance you have to manipulate the system.

For example, the machines missed the fact that a child has a father as well as a mother and never sought to complete that information (which we know was freely available from the later films). We know that no one really questioned why Cobb didn’t dream more than superficially; those that did, didn’t grasp the implications properly.

We should always be looking for gaps in our reality, noting them and investigating them. To do this we need a mechanism that exposes the gaps clearly, visually, transparently. If you don’t know, you are missing an important piece of information and will end up making bad decisions. Sometimes these bad decisions will compound over time – building up until they cause big problems, often catastrophic and wide-reaching in nature. Acting on incomplete intelligence is sometimes needed, but it’ll more likely than not to lead to a paradox of some sort.

The world is literally overflowing with examples of not taking lineage seriously. It shows up as unmanaged risks, cyber defence failures, over budget/unsuccessful transformation projects, corporate failures.

Ultimately, who pays? In the end it’s penny-wise, pound foolish to ignore this. It’s always the innocent that pay – with their cash, their jobs, their prospects. Does that sound dramatic? Well, in 26 years of IT projects, I don’t think there’s one that wouldn’t have been more efficient, more sustainable, more pleasant without lineage baked in.

The reality of lineage is that when you do it right, you’re planning to succeed.

If Cobb were working in data governance, I think he’d say: “Solidatus specializes in a very specific type of governance. Lineage governance!”

If Kyle Reese were in the same field, he might comment: “The lineage is not set. There is no governance but what we make for ourselves.”

…………………………………………………….

Below: simplified Inception diagram

Latest news

Visualize Snowflake Horizon and enhance its impact

Data lineage: The cornerstone of modern data governance

Explore the various aspects of data lineage, and understand its crucial role as a strategic ally for your organization.

Achieving Basel III compliance: A 3-Step action plan

Basel III is changing – are you prepared? In 3 easy steps, Solidatus enables banks to meet Basel III simply, and efficiently.

Solidatus Joins Forces with Artefact to Expand Consulting Partner Ecosystem

Solidatus partners with Artefact to enhance the value of data management practices and provide data-driven solutions to customers. 

HSBC & Solidatus Win at Banking Tech Awards 2023 from FinTech Futures

We're excited to announce a significant win at the Banking Tech Awards 2023 from FinTech Futures for Best Use of Tech in Business Lending

Solidatus joins the AWS Marketplace to bring connected governance to AWS customers

20 Mar 23

Solidatus and BigID announce partnership to extend data governance

Visualize Snowflake Horizon and enhance its impact

Solidatus Joins Forces with Artefact to Expand Consulting Partner Ecosystem

HSBC & Solidatus Win at Banking Tech Awards 2023 from FinTech Futures

Exciting partnership will satisfy increasing user demand for a unified solution to expose, link and visualize organizational metadata that enables actionable insights from quality, privacy and security data

London and Houston, 20^th March. Solidatus, the leading data management solution that empowers organizations to connect, visualize and govern their data relationships, and BigID, the leading platform for data security, privacy, compliance and governance, have today announced a ground-breaking partnership.

This ambitious move comes in response to the companies’ joint customers, who have requested a combined best-in-breed solution to provide enhanced metadata visibility. This solution will provide users with an active governance blueprint, delivering dynamic and real-time visualization of the flow of data across their enterprise. Organizations can proactively assess dependencies, quickly identify risks and immediately action the trusted insights gained from this new partnership.

The BigID Solidatus integration (explorer) provides active metadata lineage, which is augmented and enriched by BigID’s data quality, security and rich metadata processing. Key benefits include: accelerating tool adoption; assurance and compliance across your data estate; and providing a single simple pane-of-glass view of the complex, interconnected organizational metadata increasing automation for instantaneous impact assessment and action.

BigID CDO, Peggy Tsai said: “We can’t wait for joint customers to be able to visualize the metadata we host, including the augmentation done on metadata, and link it to other sources of metadata.”

Solidatus CEO and Co-Founder, Philip Dutton said: “I’m thrilled that BigID will bring additional metadata to enrich Solidatus clients’ enterprise data blueprints. Solidatus’ users will be able to discover new data assets, classifications, profiling, quality, and risk metadata to automatically action insights gained from the deeper understanding of their data ecosystems. In short, the sum of Solidatus and BigID is greater than the parts alone.”

This new partnership follows hot on the heels of last week’s announcement that Solidatus and Corlytics have joined forces to automate the regulatory data supply chain end-to-end, from regulatory obligation to reporting compliance.

Read about Solidatus on the BigID Marketplace.

– Ends –

For more information please contact:

Solidatus press office

solidatus@babelpr.com

BigID press office

Comms@BigID.com

About Solidatus

Solidatus is an innovative data management solution that empowers organizations to connect and visualize their data relationships, simplifying how they identify, access, and understand them. With a sustainable data foundation in place, data-rich enterprises can meet regulatory requirements, drive digital transformation, capture business insights, and make better, less risky and more informed data-driven decisions. We provide solutions to several key areas of endeavor, including: governance and regulatory compliance; data risk and controls; business integration; environment, social, governance (ESG); and data sharing. Our clients and investors include top-tier global financial services brands such as Citi and HSBC, healthcare, and retail organizations as well as government institutions.

www.solidatus.com

About BigID

BigID enables organizations to know their enterprise data and take action for data-centric security, privacy, compliance and governance. Customers deploy BigID to proactively discover, manage, protect, and get more value from their regulated, sensitive, and personal data across their data landscape. BigID has been recognized for its data intelligence innovation as a 2019 World Economic Forum Technology Pioneer, named to the 2021 Forbes Cloud 100, the 2021 Inc 5000 as the #19th fastest growing company and #1 in Security, the 2021 and 2022 Deloitte 500, and an RSA Innovation Sandbox winner.

https://bigid.com

Latest news

Visualize Snowflake Horizon and enhance its impact

Data lineage: The cornerstone of modern data governance

Explore the various aspects of data lineage, and understand its crucial role as a strategic ally for your organization.

Achieving Basel III compliance: A 3-Step action plan

Basel III is changing – are you prepared? In 3 easy steps, Solidatus enables banks to meet Basel III simply, and efficiently.

Solidatus Joins Forces with Artefact to Expand Consulting Partner Ecosystem

Solidatus partners with Artefact to enhance the value of data management practices and provide data-driven solutions to customers. 

HSBC & Solidatus Win at Banking Tech Awards 2023 from FinTech Futures

We're excited to announce a significant win at the Banking Tech Awards 2023 from FinTech Futures for Best Use of Tech in Business Lending

Solidatus joins the AWS Marketplace to bring connected governance to AWS customers

15 Mar 23

Combining software to ensure your regulatory compliance

Visualize Snowflake Horizon and enhance its impact

Achieving Basel III compliance: A 3-Step action plan

Solidatus Joins Forces with Artefact to Expand Consulting Partner Ecosystem

The regulatory landscape for financial firms is complex and subject to increasingly faster rates of change. This can be seen in the graphic below, which shows the ESG disclosure landscape for banks and capital markets in Europe – and this is just part of the regulatory burdens falling on firms.

In addition to meeting each reporting requirement in these regulations, companies also need to demonstrate that they:

Understand the requirements of the regulations;
Understand and have control over their regulatory submissions;
Use reliable and comprehensive data; and
Report consistently and reliably across their business and reporting submissions.

Source AFME, ‘ESG Disclosure Landscape for Banks and Capital Markets in Europe’, Page 11 (PDF)

In problem-solving, it’s said that two heads are better than one, and the same is often true of the technology that simplifies our lives. We at Solidatus are big believers in the value that partnerships bring to our shared clients. And so it is with keeping track of regulatory changes, how they affect your data and systems, and what decisions and actions you need to take in relation to them.

In that vein, today Solidatus announced a partnership with Corlytics to provide customers with a unique and essential tool that allows firms to:

Understand the changes in regulations in detail, quickly and easily; and
Demonstrate they are in full compliance with their detailed requirements.

Corlytics is the world’s leading provider of regulatory risk intelligence to enable organizations to take a data-driven approach to regulatory resource allocation. As part of this, Corlytics provides a regulation and law library that stores regulatory content as a fully digital set of obligations in a single location. Corlytics also ensures these digitized regulations are up to date.

Combining the Corlytics digitized regulations service with the visualization and lineage capability of Solidatus gives clients the unique ability to have a visual, easy-to-use view of regulations that’s focused and shared across the organization.

What does this mean for clients?

Clients can use the combined power of Corlytics and Solidatus to:

Identify the impact of regulatory change; and
Demonstrate full compliance with regulations – ‘front-to-back compliance’.

In conclusion to this short blog post, we’ll take a look at what improvements in these two areas mean for practitioners engaged in being compliant and demonstrating this compliance.

Regulatory impact assessment

Using this web-based visualization, the drill-down and workflow capabilities of Solidatus mean firms can review up-to-date regulations and regulation changes, and:

Share the latest versions and understanding across the firm based on the business area and need;
Focus in on changes in the regulatory texts to highlight impacts; and
Reduce the dependency on key regulatory compliance experts.

Sharing specific views of regulatory changes will also help front-office staff understand the impact of changes and facilitate front-office business to take advantage of business opportunities that arrive ahead of competitors.

Front-to-back compliance

Combining the added value of Corlytics content with lineage models across the IT infrastructure of the firm gives firms the unique ability to map their complete compliance with regulations by:

Mapping regulatory texts from Corlytics;
Aligning them to the lineage across all bank systems in scope; and
Linking them to their detailed submissions.

In addition, firms can use this understanding to start to standardize and reuse compliance assets to improve the consistency and efficiency of their reporting across regulations and regulators.

These two use cases highlight the unique value of combining the digitized context from Corlytics with the visualization and lineage within Solidatus, and we look forward to developing further valued services with Corlytics going forward.

Latest news

Visualize Snowflake Horizon and enhance its impact

Data lineage: The cornerstone of modern data governance

Explore the various aspects of data lineage, and understand its crucial role as a strategic ally for your organization.

Achieving Basel III compliance: A 3-Step action plan

Basel III is changing – are you prepared? In 3 easy steps, Solidatus enables banks to meet Basel III simply, and efficiently.

Solidatus Joins Forces with Artefact to Expand Consulting Partner Ecosystem

Solidatus partners with Artefact to enhance the value of data management practices and provide data-driven solutions to customers. 

HSBC & Solidatus Win at Banking Tech Awards 2023 from FinTech Futures

We're excited to announce a significant win at the Banking Tech Awards 2023 from FinTech Futures for Best Use of Tech in Business Lending

Solidatus joins the AWS Marketplace to bring connected governance to AWS customers

01 Dec 22

Solidatus recognized in 2022 Gartner® Market Guide

Visualize Snowflake Horizon and enhance its impact

Solidatus Joins Forces with Artefact to Expand Consulting Partner Ecosystem

HSBC & Solidatus Win at Banking Tech Awards 2023 from FinTech Futures

Last month, Gartner® published its Market Guide for Active Metadata Management*.

We were delighted – but not surprised – to see that Solidatus was named a Representative Vendor in this Market Guide report as to us, active metadata is at the heart of everything we do.

But what is active metadata?

Gartner opinion

In the report, Gartner describes active metadata management as “a set of capabilities across multiple data management markets, led primarily by recent advancements in how metadata can be used for continuous analysis. Data and analytics leaders must consider the market evolution as transformational in all data-enabling technologies”.

In our opinion, this is a great overview, and we’d recommend you read the full report. Highlights include:

A strategic assumption that “[t]hrough 2024, organizations that adopt aggressive metadata analysis across their complete data management environment will decrease time to delivery of new data assets to users by as much as 70%”;
A market direction, which states that, “[o]verall, the metadata management software market grew at 21.6%, reaching $1.54 billion in U.S. dollars. This is one of the highest growing markets within data management software overall, following the DBMS market growth of 22%, although from a much smaller revenue base”; and
A market analysis that states that “[c]ollaborative utilization will require new ways to capture and visualize metadata (driven by data preparation for analytics). Included is the capability of rating, ranking, tagging of data and ability to communicate within the metadata solutions”.

But we think active metadata means slightly different things to different vendors.

In this short blog post, a prelude to a series of more detailed blog posts on this increasingly important subject, we summarize what active metadata means to Solidatus and its growing body of users.

The DNA of Solidatus

It took others to identify and name active metadata. But – as with DNA itself, which obviously existed before Watson and Crick discovered and named it in the 1950s – active metadata is, and has always been, in our DNA.

It’s what we do and it’s what underpins our technology, through whichever use case lens you view our data lineage solution.

It starts with metadata itself, which we’d define as a special kind of data that describes business processes, people, data and technology, and the relations between them, bringing context and clarity to the decisions that link them. Traditional examples include data catalog and business glossary.

This brings us to active metadata. We believe our definition resonates with Gartner’s: the way we see it, active metadata is the facility to reason about, visualize dynamically and gain continuous insight from information about data, data systems, business entities and business concepts, the relations between them, and stored knowledge about them.

How we make metadata active

So, what makes active metadata active and why is it so different from what went on before? At Solidatus, we’d answer these questions with four points:

Active metadata includes logical reasoning;
Active metadata offers a very dynamic form of visualization;
The information in active metadata is not just about the entities themselves, but about the connections between them; and
Active metadata should include stored knowledge. This is subtly different from other metadata, because it sits at a higher level, and offers more general, or more universal, information, such as business definitions.

The consequence of all of these is continuous insight. It’s more dynamic, it’s more complete, it’s based on context as well as content, and it respects standards.

It’s a whole different ballgame.

We’ll expand on these in future blog posts, but anyone familiar with Solidatus will immediately appreciate how we sit right in the centre of this space.

The wider context

We’ll finish by contextualizing active metadata, at least as we see it, in terms of what’s delivered, the attributes of an active metadata solution, and – crucially – the main areas for which it can be used.

What’s delivered
An active metadata solution:

Is embedded within an organization’s data and business practices;
Presents a continuous, coordinated, enterprise-wide capability; and
Provides monitoring, insight, alerts, recommendations and design.

Solution attributes

An active metadata analytics workflow:

Is integrated, managed and collaborative; and
Orchestrates inter-platform metadata assets and cross-platform data asset management.

What it can be used for

Active metadata assets are used to create insight solutions which, among other things, enhance:

Data integration;
Resource management;
Data quality management;
Data governance;
Corporate governance;
Regulatory control;
Risk management;
Digital transformation; and
ESG.

Above all, the benefits of good metadata capabilities boil down to: making business information complete, coherent, informed and logical; delivering faster, richer and deeper insight; keeping everything up to date; and making your processes reliable and responsive.

Watch this space for our detailed follow-up blog posts and, as ever, we encourage readers to request a demo of Solidatus.

To read more on this subject, see What is active metadata? and Mining value from active metadata.

*Gartner, “Market Guide for Active Metadata Management”, November 14, 2022.

GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved. Gartner does not endorse any vendor, product or service depicted in its research publications and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s Research & Advisory organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.

Quick Answer: What Is Active Metadata?

In the latest Gartner® report, find out what active metadata is, how to use it, and how to get started.

Latest news

Visualize Snowflake Horizon and enhance its impact

Data lineage: The cornerstone of modern data governance

Explore the various aspects of data lineage, and understand its crucial role as a strategic ally for your organization.

Achieving Basel III compliance: A 3-Step action plan

Basel III is changing – are you prepared? In 3 easy steps, Solidatus enables banks to meet Basel III simply, and efficiently.

Solidatus Joins Forces with Artefact to Expand Consulting Partner Ecosystem

Solidatus partners with Artefact to enhance the value of data management practices and provide data-driven solutions to customers. 

HSBC & Solidatus Win at Banking Tech Awards 2023 from FinTech Futures

We're excited to announce a significant win at the Banking Tech Awards 2023 from FinTech Futures for Best Use of Tech in Business Lending

Solidatus joins the AWS Marketplace to bring connected governance to AWS customers

16 Nov 22

Solidatus joins Snowflake’s partner program

Visualize Snowflake Horizon and enhance its impact

Solidatus Joins Forces with Artefact to Expand Consulting Partner Ecosystem

HSBC & Solidatus Win at Banking Tech Awards 2023 from FinTech Futures

Snowflake’s native governance capabilities gives you the power to know, control and unlock your data. They created the Data Governance Accelerated Program for partners who can integrate and enhance these capabilities, and the team at Solidatus are proud to partner with them to add new levels of context and visualization to Snowflake, enhancing its governance capabilities, giving you more control and a deeper understanding of how data is used across your organization, and where policies need to be applied. With this knowledge, you can better address regulatory requirements, drive digital transformation, capture business insights, and make better, less risky and more informed data-driven decisions.

Key benefits of the Solidatus approach

As a Snowflake user, you have access to a suite of data governance controls like access policies, data masking, object tagging and coarse grain lineage. Solidatus’ data lineage enriches these capabilities by allowing you to create living blueprints that map how your data flows as it moves through your systems – both now and at other points in time. When metadata is ingested from Snowflake into Solidatus, we integrate any security policies, data masking, and tagging information as an extra dimension to your data flows. Through understanding how and where these controls are being applied, you establish a trusted foundation for policy enforcement, and radically speed up cloud transformation projects by cutting discovery time, streamlining implementation, and mitigating future risk and costs.

To summarize, Snowflake with Solidatus will:

Cut costs by reducing the time spent managing and implementing resources.
Speed up your migration to Snowflake.
Provide needed visibility to reduce operational risk and improve operating efficiency.
Boost governance and regulatory compliance.
Establish controlled planning and implementation of change.
Monitor data sharing for compliance.
Visualize and contextualize data policies, regulations and other data assets present in Snowflake.

Play Video

Latest news

Visualize Snowflake Horizon and enhance its impact

Data lineage: The cornerstone of modern data governance

Explore the various aspects of data lineage, and understand its crucial role as a strategic ally for your organization.

Achieving Basel III compliance: A 3-Step action plan

Basel III is changing – are you prepared? In 3 easy steps, Solidatus enables banks to meet Basel III simply, and efficiently.

Solidatus Joins Forces with Artefact to Expand Consulting Partner Ecosystem

Solidatus partners with Artefact to enhance the value of data management practices and provide data-driven solutions to customers. 

HSBC & Solidatus Win at Banking Tech Awards 2023 from FinTech Futures

We're excited to announce a significant win at the Banking Tech Awards 2023 from FinTech Futures for Best Use of Tech in Business Lending

Solidatus joins the AWS Marketplace to bring connected governance to AWS customers

06 Oct 22

Shifting from defense to offense

Solidatus named in 2024 FT 1000 ranking of Europe’s fastest-growing companies

Visualize Snowflake Horizon and enhance its impact

Data lineage: The cornerstone of modern data governance

Although our software was born in London, we now call many of the world’s great cities home. Earlier this year, we set up shop in Houston, Texas to meet the demand for next generation data management solutions throughout North America. Since then, we’ve attended a myriad of events across the continent and have welcomed many new colleagues that share our passion for reimagining data management.

Last week, three of those colleagues – Solidatus’ Global Strategic Account Director, Mary Anne Bullock; Advisory Solutions Engineer, Kevin Shannon; and our Director of Business Development, Glenn Aluce – joined our CEO, Philip Dutton and our Director of Analyst and Industry Relations, Ashlee Dutton at A-Team’s data management summit in New York City.

We listened and learnt from those who attended, absorbing their insights, and suggested solutions to their complex problems. Ashlee Dutton commented on her experience at the summit: “There is a change happening across many industries. When it comes to an organization’s data strategy, they’re now shifting from defense to offense. Data is more than just another thing to report on – it’s influencing how the C-suite are steering the direction of their businesses.”

The theme of the event was centered on leveraging data to drive business and compliance insight – topics that Solidatus regularly speaks to. Addressing a packed venue, Philip Dutton joined the Chief Data Officer at Bank of New York Mellon (BNY Mellon), Eric Hirschhorn, on stage to deliver a keynote about how our data blueprints are being used for knowledge, context and better decision making. BNY Mellon is the world’s largest custodian bank, managing $43 trillion of client assets. Hirschhorn played a pivotal role in the bank acquiring the Solidatus solution to transform their data management program.

To summarize Philip’s and Eric’s key points:

Change is inevitable, preparation is paramount.

“The data world has changed so dramatically, so why hasn’t our approach or tooling? This is why we developed Solidatus – to deliver a solution that provides unique capabilities within data management to enable a fundamental shift in thinking and approach. We’re enabling organizations to create living and dynamic blueprints of their data, allowing the business to simplify the complexity and give context allowing quicker and better data driven decisions,” said Philip.

Everything is connected. Data doesn’t exist in a vacuum.

At Solidatus, we know that everything is connected, and that there is a global demand for active metadata: the continual visualization of systems, infrastructure, processes and critical data elements. Data interacts and impacts an organization’s entire ecosystem, so it must be modeled as such. Data’s WHY is more important than the WHAT or the HOW. No matter how far along you are on your data journey, you need to know where you are to understand where you want to get to. Solidatus allows you to see the data held across your enterprise architecture. Philip added: “Let’s not continue to boil the ocean, let’s start with the kettle.”

Building a strong data foundation unlocks value across your business.

Eric talked about the immeasurable value that will result from implementing Solidatus. With Solidatus, he and his colleagues can understand the flow of their data for regulatory filings; ensure their risk picture is correct; trace the journey of their market data; and comply with jurisdictional concerns around where data is stored. Having a unified view of their data landscape is vital for building new products and services, and to ensure their data is timely, complete, accurate and consistent. These are just a few of the ways in which Solidatus can be used. Departments across BNY Mellon are planning on using our software for multiple different use cases. The goal is to move from a siloed to an enterprise data management governed view.

Soon after Philip’s and Eric’s keynote, Solidatus took the stage again. Mary Anne joined leading data practitioners and innovators on a panel titled, “Shifting to agile data management practices to ensure enterprize-wide data access, analytics and visualization.”

Mary Anne made several key observations:

The main benefits of data democratization are timely and equitable access to data.

When users across your business have access to data that is held in context, numerous other benefits span from it. These include:

– Accelerating transformation programs and product innovation.

– Delivering rapid insights with confidence and trust.

– Reducing expenses associated with manual efforts and subject matter expertise.

– Building a sustainable knowledge base – once connections are built, they are retained throughout the life of the project.

Traditional data management development approaches fail because of rigidity.

An agile approach to data management stops organizations churning through multiple system implementations which lead to uncoordinated reporting and analytics because of tight coupling with the systems that are being deployed. Most existing data functions are governance- driven, which has put a burden on data producers to maintain various catalogs and data asset inventories, along with various sets of complex business rules.

To create value from data you need an active data culture.

To get businesses to think of data as an asset, any policies, procedures, and governance structures must be supported by a culture that encourages open communication and a common understanding between those that prepare and process data sets, and those who consume that data for analysis, insights and reporting. Adopting a commercial mindset is fundamental to setting up successful data functions.

We believe that Solidatus holds the keys to a new frontier of data management, so we were delighted to be recognized as such by our peers by being awarded A-Team’s Best Data Discovery and Catalog Solution at the summit.

Get in touch to find out how we can help you usher in a new era of data decision-making

Find out how you can create a living blueprint that can provide you with the knowledge and context for better-quality decision-making, robust impact analysis and reduced program time, risks and costs.

Latest news

Visualize Snowflake Horizon and enhance its impact

Data lineage: The cornerstone of modern data governance

Explore the various aspects of data lineage, and understand its crucial role as a strategic ally for your organization.

Achieving Basel III compliance: A 3-Step action plan

Basel III is changing – are you prepared? In 3 easy steps, Solidatus enables banks to meet Basel III simply, and efficiently.

Solidatus Joins Forces with Artefact to Expand Consulting Partner Ecosystem

Solidatus partners with Artefact to enhance the value of data management practices and provide data-driven solutions to customers. 

HSBC & Solidatus Win at Banking Tech Awards 2023 from FinTech Futures

We're excited to announce a significant win at the Banking Tech Awards 2023 from FinTech Futures for Best Use of Tech in Business Lending

Solidatus joins the AWS Marketplace to bring connected governance to AWS customers

27 Jul 22

Building a centralized data governance solution

Visualize Snowflake Horizon and enhance its impact

Data lineage: The cornerstone of modern data governance

Solidatus Joins Forces with Artefact to Expand Consulting Partner Ecosystem

Good data governance includes a set of robust processes, policies and standards that ensure the efficient and effective use of information. It stipulates who is responsible for data, how it can be used and how its quality is assured. Part of NOW: Pensions’ newly defined Data Governance Program included creating a new business data model to improve and organize their data structure. Once built, the model was then enriched through defining a set of data quality rules that prevented the uptake of bad data, as well as measuring the quality of the data that was already held by NOW: Pensions.

The team at NOW: Pensions identified three main challenges:

The ‘as-is’ data wasn’t clear.

A business data model was needed to guarantee that the right data was available to users and that there was one source of truth.
Data quality rules were required to measure current data quality and to stop bad data re-permeating systems.

Key to addressing all these challenges was achieving an understanding of where data originated and what data flowed to where.

How NOW: Pensions used Solidatus to address their key challenges

Data Distress: Is the Data Office on the Brink of Breakdown?

Mapping data lineage

NOW: Pensions used our powerful data lineage functionality to map their ‘as-is’ data lineage from its point of origin right through to the data warehouse. This gave them an accurate snapshot of how their databases were structured and provided cataloguing of the technical metadata for every column within each individual table, such as data type and field length. Individual source system models could then be joined to show the transitions from the source systems through the staging databases to their data warehouse.

Creating a Business Glossary

NOW: Pensions created a business glossary to provide a common language for all the terminology within each source system. Solidatus Reference Models allow subject-matter experts (SMEs) across the organization to record the terms their area uses; these can then be shared, centrally collated, and disseminated across the business to ensure consistency of understanding and eliminate confusion.

Building a Business Data Model

Once a new business data model was created within Solidatus, the source systems could then be mapped to entities within the lineage models so that the data architects and developers were confident with what each of the column headers referred to. Business rules could then be defined to ensure that all the data fields in the new model were correct even though the data came from a variety of systems.

NOW: Pensions is using Solidatus as their central data governance tool. This has resulted in:

A single workflow for managing collaboration among users and facilitating the effective development of Models.
‘As-is’ models providing a clear view of where the data entered their systems, which was invaluable in identifying where to execute Data Quality Rules.
A complete view of how their entire data landscape is structured.
A common language for their terminology.
Full transparency across their data landscape.
A Business Data Model built with the confidence that only correct and timely data can be viewed with the proper business rules in place to consolidate various sources.

As part of NOW: Pensions’ ongoing Data Governance Program, the company will soon be mapping access rights to each data view and using it as a centralized access control log.

“When we started our data governance project, we did not fully understand how our data was produced, transformed and, ultimately, used. We hadn’t mapped our data’s lineage or populated a data dictionary. Now, all that has changed. With Solidatus, our data estate is now mapped, modelled and catalogued. In a single view, I can show the business where their data resides, how it flows through systems and applications, what data quality rules apply and what data is subject to GDPR. Solidatus is central to the way that we govern data.”

Phil Yeoman, Group CDO, Cardano and NOW: Pensions

Get in touch to find out how we can help you deliver on your governance program.

Latest news

Visualize Snowflake Horizon and enhance its impact

Data lineage: The cornerstone of modern data governance

Explore the various aspects of data lineage, and understand its crucial role as a strategic ally for your organization.

Achieving Basel III compliance: A 3-Step action plan

Basel III is changing – are you prepared? In 3 easy steps, Solidatus enables banks to meet Basel III simply, and efficiently.

Solidatus Joins Forces with Artefact to Expand Consulting Partner Ecosystem

Solidatus partners with Artefact to enhance the value of data management practices and provide data-driven solutions to customers. 

HSBC & Solidatus Win at Banking Tech Awards 2023 from FinTech Futures

We're excited to announce a significant win at the Banking Tech Awards 2023 from FinTech Futures for Best Use of Tech in Business Lending

Solidatus joins the AWS Marketplace to bring connected governance to AWS customers

12 Jul 22

Discover infinite possibilities in your data solar system

Visualize Snowflake Horizon and enhance its impact

Data lineage: The cornerstone of modern data governance

Solidatus Joins Forces with Artefact to Expand Consulting Partner Ecosystem

Last week, NASA blew the world away with the first images from its James Webb Space Telescope. Promising to far exceed the wonder and quality of its predecessor, Hubble, the JWST has delivered a ground-breaking new view of our universe. These images have told the story of “the hidden universe through every phase of cosmic history” in a way that has never been seen by the naked eye of man.

A “singular and historic moment”, the JWST has and will continue to explore the depths of our solar system, as well as what lies beyond it and “the most distant observable galaxies in the early universe.”
Scientists believe that the data derived from this will answer questions mankind has asked for millennia, as well as questions we do not yet know that need to be asked. It will help us better understand neighbouring planets, the universe and our own place within it. The possibilities this opens up are, for us, utterly endless. The voyage of discovery that lies ahead will undoubtedly be as immense as space itself, and the value it will bring us as humans – for ourselves and our planet – will change the face of our lives and this world for generations to come.

Parallel data universes

And this got us thinking something a bit crazy – how different is the data of our universe, planets, exoplanets, galaxies and solar systems from the data of a business? Okay, we know this is a bit ambitious, but take a step back for a moment. Forgetting sheer size and scientific importance, there are many parallels to be drawn. The data that builds the fabric of an organisation is vast, rich and seemingly endless. The different facets and ecosystems of that data can be seemingly unfathomable to many trying to grapple with its complexities. Is understanding what exists beyond us in the deep, infinite spans of space really all that different to understanding one’s own data?

Businesses have a mammoth task ahead of them when it comes to data management. At its most complex, data can be messy, confusing and even misleading. But at its best, data can be a powerful force driving organisations forward – not only through compliance, but also innovation. Getting to this point in a data journey is no easy feat, though. It requires data quality, sound data governance and efficient data lineage to fully understand what data there is, how it is moving through an organisation, who it is available to and how it is reported on.

Achieving this requires more than just good will and a few strategically drawn up Excel spreadsheets. For this, you need the James Webb Space Telescope of data management. You need a tool that is going to not just show you your data in clear, HD view, but also show your metadata, how it is all moving, who is responsible and, most importantly, the value it can deliver to your business. Something that will reveal the data and findings you were looking for in a matter of minutes, rather than days, weeks or months.

Professor Gillian Wright, the British researcher who is co-principal investigator on one of Webb’s four infrared instruments said, “Whenever you look at the sky in a new way, you see things that you didn’t expect.”

Discovery is knowledge, and knowledge is power

When looking at your data through the lens of a Solidatus model, you can see it in a new way; in a way you might not have expected. It will answer questions you have been asking for years. It will give you solutions for the problems you knew you had, and those you didn’t. It will answer questions you hadn’t even thought to ask yet. It will take you on a journey through a solar system of data that orbits your organisation, uncovering hidden gems that can change the way you do business. It will tell your story, from inception to current day, showing you the way forward through innovation and creativity.

As the Director of UK Astronomy Technology Centre told BBC News about the James Webb Space Telescope, “…that’s telling you that the discoveries are just sitting out there waiting to be made.”

There are infinite discoveries to be made in an organisation’s data – whether you know it or not. There are new lessons to learn, new ways of working to uncover and boundless benefits not only for your own business, but for your clients too.

Knowing your organisation’s history and story through its data can change the way it operates forever. And there is no better way of knowing this than through Solidatus.

Discover your data solar system and the possibilities that lie beyond with Solidatus. Request a free demo today.

Latest news

Visualize Snowflake Horizon and enhance its impact

Data lineage: The cornerstone of modern data governance

Explore the various aspects of data lineage, and understand its crucial role as a strategic ally for your organization.

Achieving Basel III compliance: A 3-Step action plan

Basel III is changing – are you prepared? In 3 easy steps, Solidatus enables banks to meet Basel III simply, and efficiently.

Solidatus Joins Forces with Artefact to Expand Consulting Partner Ecosystem

Solidatus partners with Artefact to enhance the value of data management practices and provide data-driven solutions to customers. 

HSBC & Solidatus Win at Banking Tech Awards 2023 from FinTech Futures

We're excited to announce a significant win at the Banking Tech Awards 2023 from FinTech Futures for Best Use of Tech in Business Lending