Home » Cloud

Solidatus joins the AWS Marketplace to bring connected governance to AWS customers

amazon web services solidatus partnership scaled

Solidatus joins the AWS Marketplace to bring connected governance to AWS customers

Unlocking the true power of AWS with the world’s leading data lineage solution 

Solidatus, the leading enterprise data lineage and metadata management provider, today announced its availability in the Amazon Web Services (AWS) Marketplace, allowing AWS customers to seamlessly discover, purchase, and deploy Solidatus. With integrations to AWS services like S3, Redshift, and Glue, plus connectivity to other AWS data sources, Solidatus delivers an integrated data landscape view across AWS and non-AWS environments. This unified approach allows for effective governance and risk management by bridging cloud, on-premises, and hybrid systems on one pane of glass. 

Simon Bustamante-Dick, Partner Analytics Leader, EMEA AWS, said: “AWS Marketplace transforms how enterprises worldwide find, subscribe to, deploy, and govern third-party software, data, containers, machine learning models, and professional services. It’s also become the most strategic channel for ISVs, data providers, and resellers to acquire new customers, migrate existing customers to the cloud, and grow revenue. I’m excited to welcome Solidatus to the AWS Marketplace. Their solution lets customers create integrated data maps spanning AWS and non-AWS environments, and this consolidated visibility allows organizations to simplify governance across AWS cloud, multicloud, hybrid, and on premises systems. Solidatus fills a key need for our customers seeking end-to-end data insight.”

With an extensive list of trusted partnerships and out-of-the-box integrations with leading data management vendors like Collibra, Snowflake, Alteryx, Google BigQuery, BigID, Denodo, Data.World, Informatica, Oracle, Microsoft, and IBM Redhat, Solidatus gives AWS customers confidence in leveraging their data from end to end.

Philip Dutton, Solidatus CEO, added: “Through AWS Marketplace, it’s now simpler than ever for customers to leverage Solidatus to visualize their data journeys and unlock deeper insights across their business. Our technology’s flexibility is a key asset, and we look forward to supporting AWS users with diverse data intelligence needs, from governance to analytics.”  

Key benefits of Solidatus for AWS users:

It doesn’t matter how data enters your business, how it’s transported, or where it’s stored, Solidatus consolidates your enterprise metadata into a single, manageable view. This gives users the power to create dynamic blueprints that visualize real-time data flow, providing an operational control center for comprehensive data management and analysis at scale.   

  • Automate end-to-end data lineage across AWS services  
  • Connect data sources for integrated analytics and improved decisions 
  • Map data to critical business processes to streamline operations 
  • Streamline data governance workflows  
  • Identify security gaps for enhanced data protection
  • Rapidly embed risk controls into data and processes  
  • Identify data quality issues affecting reporting 

View us on the Marketplace

For more information contact hello@solidatus.com

For AWS customers contact aws-marketplace@solidatus.com 

photo 1505664194779 8beaceb93744

Solidatus Co-CEO Philip Dutton recently caught up with Baz Khuti, President US Modak, about the difference between data and metadata, data lineage, and its role in the cloud space.

First up in the discussion was the somewhat grey area of the difference between data and metadata. A fan of analogies, Philip drew the perfect comparison:

“The Library of congress has 164 million items. How do you find the information you’re looking for? You do that by going to the index which points you to the right area. That is the metadata – describing where it is. And what it is. It gives us more classification and categorisation to help us determine if this the information we are looking for. Metadata gives the ability to find and understand the data you are looking for. If we move from a library to the internet, we can see that in this instance the data is the internet. We have trillions of webpages. How do you find what you are looking for? You go to a search engine which has indexed the metadata. That really is the crux of what metadata is – it’s the description of the data we are looking for.”

Philip and Baz went on to chat about important key points regarding lineage and cloud space:

  • The role data lineage plays in metadata – it is all about connectivity. Lineage is the hyperlink between documents, the connection between data sources
  • The opportunities and challenges of moving to the cloud – it generates infinite possibilities, but it’s important to not replicate mistakes from the past
  • Cloud gives us greenfield opportunities to build better by default and by design – but we must first understand our data before it is moved

Watch their interview for CDO Magazine on-demand now to learn more and look out for the next segment on regulations and the responsibilities organizations bear when it comes to compliance.

cloud blog image

Effective data governance has become a vital requirement for a successful enterprise in an increasingly data-driven world.

But, as well as the traditional challenges of maintaining high-quality, fit-for-purpose data while complying with ever-growing legislative and regulatory requirements the data world has added the new imperative: gaining the competitive advantages made possible by moving some or all of your data to the cloud.

With such changes to the data environment, it is immediately apparent that traditional, manual, spreadsheet-driven data governance processes aren’t up to the task. How can those who wish to modernize their data governance processes in line with their progress to the cloud achieve this?

CDMC review

An initiative of the EDMC, the CDMC (Cloud Data Management Capabilities) framework is a thorough review of best practices for managing data in the cloud.

It comprises six major categories:

  • governance and accountability
  • cataloguing
  • accessibility
  • protection
  • data lifecycle
  • data and technical architecture.

Through these, it lays out how the implementation and maintenance of cloud storage may best be achieved.

A holistic approach to the many and varied challenges posed by cloud adoption has not been attempted before. This is one of the most significant and comprehensive analyses of cloud data management ever compiled.

The ‘skeleton’ of the CDMC framework was laid out initially as a series of six Components, containing 14 major Capabilities, further divided into 37 Sub-Capabilities, to which, as a practitioner in cloud and data management, data architecture, lineage and catalog, I contributed. These formed the basic ‘building blocks’ assigned to relevant subject-matter experts on the various subgroups that took ownership of them. In addition, specialist working groups were set up to deal with areas of particular importance, such as data lineage, of which I was also a participant.

Once a draft of a topic (either Upper Matter, Capability or Sub-Capability) was developed, members of the working group would be informed. They could read, review, and comment on it in the EDMC Dropbox before a full Review Committee review took place in one of the scheduled sessions. 

Transformation with the cloud

It brought home to me how rapidly and radically good data management has been transformed by the arrival of cloud technology and how the consequences and ramifications of its adoption have remained nebulous until now. For example, with the exception of global multinational companies, data sovereignty and cross-border data movement were not concerns that typically would have bothered an on-premises installation.

Once a company decides to put its data into the cloud, though, questions on where it may end up being held become pertinent, and you must establish, understand and observe jurisdictional requirements. In turn, this is just one example of how vital a comprehensive data catalog is to successful cloud adoption. Without one, the metadata essential to complying with the myriad regulations and data privacy obligations to which holders of data are subjected cannot be stored and used to avoid regulatory infractions.

Understanding the data

Over and over again in these review sessions, practitioners stressed the absolute importance of understanding the data that you are putting into the cloud, at the time it is put into the cloud, and that without catalogs containing metadata to describe those data assets fully, controlling and managing that data becomes impossible. 

Multi-cloud and hybrid-cloud environments also present additional challenges for keeping data lineage accurately recorded and up to date. Unsurprisingly, automation is a major theme throughout. For example, “The data lineage in cloud environments must be captured automatically, and changes to lineage must be tracked and managed. Visualization and reporting of lineage must be implemented to meet the needs of both business and technical users.”

Five objectives for data lineage are included:

  • Implement automated functionality that identifies processes that move data.
  • Record data lineage metadata for data movement processes that are discovered automatically.
  • Ensure lineage auto-discovery identifies processes that move data across jurisdictions, availability zones and physical boundaries.
  • Ensure lineage auto-discovery is enabled in hybrid and multiple cloud environments, and identifies data movement between those environments.
  • Define and implement processes for the review of auto-discovered lineage information.

Data governance platforms

The best contemporary data management solutions, including those that help with data governance, are designed to provide exactly these capabilities, in radical contrast to cumbersome, manual, text-based methods.

For organizations using DCAM to assess and monitor their data governance maturity, you can use Solidatus to map to CDMC, empowering a holistic data governance approach across both on-premises and cloud data on an enterprise-wide basis; one which can be automatically maintained, giving management confidence that its data governance solutions are optimised for the future, not stuck in the past.

A note on Solidatus for EDMC members

Solidatus is a powerful tool for efficient data management, visualization and discovery. As the selected EDM Council knowledge modeling solution, EDM Council members are given read-only access to the modeled frameworks and how they interact with each other as well as map on to further privacy and financial regulations.

Using Solidatus, you create living blueprints that map how your data flows, and how it’s affected, as it moves around your systems, both now and at other points in time.

By revealing hidden opportunities and threats, and showing you the impact of change, a Solidatus data lineage blueprint delivers clarity and an enriched understanding of your ecosystem, so you can optimize your infrastructure, operate more efficiently, and minimize risk.

If you are not an EDM Council member but would like to view the models, please email us at hello@solidatus.com.

Cloud computing binary HIGH RES scaled

Cloud migration has become one of the key issues large businesses are facing, and the global pandemic has only accelerated this. We recently hosted a panel discussion ‘Cloud Transformation – The Greenfield Opportunity, but where do you start?’ to dive deeper into this key focus area.

We handpicked an esteemed panel with years of experience and knowledge to share. The panellists included: David Knott, Digital Transformation Officer Google, Tom Jenkins, Head of Data and Analytics Risk at HSBC and Colin Gibson, EDM Council CDMC Product Owner and our own CDO, Lorraine Waters.

The panel covered these four important topics:

  • Strategies – are most businesses adopting a Cloud-First approach?
  • Challenges – regulatory, cataloging/inventorying, lift & shift vs. Greenfield?
  • Enablers – what are the Cloud providers doing to ease the journey to Cloud?
  • Controls – overview of CDMC recommendations

One of the key challenges around the cloud is cataloging your data. You must catalog everything. If you are not sure what you are capturing, how do you know if you have put something important onto the cloud that you shouldn’t have? Similarly, you must classify everything to know precisely where it sits in your data system. When everything is cataloged and classified, you start to apply the controls and use the new tools and approaches that the cloud enables. It’s getting that foundation in place upfront for cataloging and classification that allows you to make the most of the cloud. To make sure you keep up good cataloging and classification, you must also implement good data management practices. Ensuring you have data ownership, authorised sources, and a privacy framework sets you up in the right direction. It’s crucial to highlight that cloud won’t magically fix these problems if you don’t have these already on-prem, then the cloud won’t be any easier.

The fundamental strategy which brings the most value out of the cloud is to think of the cloud as a stand-alone platform. Many people start from the position that it’s an excellent solution to fit a specific niche problem, and it might be problems with cost or risk or with agility. But increasingly, we’re seeing customers think of the cloud as a platform where you can do a broad category of things, for example, analytics and insights or AI engineering. I think we see that platform strategy become increasingly vital. Companies who adopt a strategy where they view the cloud as a stand-alone platform have started to experience success because it allows them to focus the business outcomes rather than the problems the company faces.

One of the primary stumbling blocks cloud providers are working on is making the journey to the cloud more accessible. To solve this, providers keep adding more products to fill gaps in capabilities coverage. With all these new products and capabilities, does it make it easier for customers or more confusing? It’s like getting a bicycle in a cardboard box with loads of parts in it; how helpful for you is that? Increasingly providers need to package those products together into more end-to-end platforms. The idea of those is to help you perform end-to-end data management capabilities and end-to-end AI engineering and management capabilities.

We want to finish with a quote from David Knott, Digital Transformation Officer Google, who joined us on the panel discussion, “we can’t do everything ourselves, which is why we have companies such as yourselves and others to give us a rounded portfolio of things.” This collaborative approach whereby large cloud providers partner with smaller, more agile and modern tech solutions to provide an end-to-end solution is quickly being adopted. This approach not only gives greater flexibility to customers but allows for a much faster and more robust and automated solution.

Watch the whole panel from the EDM Council DataVision EMEA 20201 virtual event: Cloud Transformation – The Greenfield Opportunity, but where do you start?  on-demand now:

Play Video
Sand white HIGH RES scaled

From 29 – 30 September, A-Team Insight brought together the US data management community to explore the latest challenges, opportunities and data innovations facing sell side and buy side financial institutions.

On day two of the event, Solidatus Co-CEO, Philip Dutton, joined data experts Dennis Slaterry from EDMworks, Andrew Foster from Deutsche Bank, Olga Maydanchik from Voya Financial and Harpreet Singh from Luxoft on a panel titled: “The power of data lineage to deliver compliance and business insight”. A wide range of key challenges were covered from managing ever increasing data volumes to the mainstream adoption of cloud.

The panel started with each member stating their definitions of data lineage, with Philip adding an important point: the power of lineage transcends technical flows.

As a panel, we’re trying to educate the community that lineage enables a lot more than just the technical pipes where the data flows – it brings in huge amounts of impact analysis and transformational change to an organisation. There is an additional layer of value which can be utilised when we start to think about lineage in a different way than just the purely technical sense.

Play Video

The term automation means many things to many people – Philip believed that its definitions in the market are misguided.

Unfortunately, automation takes time, it takes understanding, and it takes a lot of effort. It really comes down to the ‘why’ – what’s the value that we’re generating? The state of automation that most people are looking to achieve is if a system changes, tell me about it. The problem in global investment banks and complex organisations is that if you’re hearing about a change after it’s happened in production, you’ve got a bigger problem in the organisation. We need to move away from this reactive approach to a more proactive lineage approach – we should be thinking about planning and execution, with the automation as the validation that what we’ve done is what we plan to do.

The creation and retention of data has increased exponentially in recent years, with a growing remote workforce catalysing the production and addition of more cloud applications to companies’ data landscapes. Philip made the point that the journey to the cloud is like any other transformational change companies enact.

You need to know where you’re starting from and where you want to end up – an organisation looks at their current state on-premise and plans to move these elements into the cloud, and typically, people don’t pick up their whole organisation and start moving all the data into the cloud at once; it’s this journey that takes time and part of that is designing a cloud migration strategy to ensure you don’t breach regulations like cross-border data sharing, or that you sufficiently structure the migration so you’re not moving huge volumes of data up to the cloud and then back down into on-prem, generating huge costs. Lineage allows us to simulate what that future state of an organisation looks like while remaining in the safety of a confined space.”

Before the session ended, the panel discussed the relevance on AI and Machine learning within metadata management – Philip observed that much of what the vendors are pushing in the market is overhyped mainly because we work in metadata, and not data. The record sets for metadata management are so shallow that you can never train an AI to really deal with it. When you’re working with the actual underlying data and trying to infer lineage based on that data, there’s a much better case for using an AI or an ML – it’s important to understand the distinction between the two.

Play Video

Solidatus for the cloud – Migrate, Optimise and Transform

Businesses are turning to the cloud for reliability, scalability, flexibility and consistent processing – but, with great transformation comes complexity and risk. With Solidatus, prepare your migration strategy by identifying mission–critical data and eliminating redundant information – enrich this data with current regulations, service agreements and business intelligence allowing for in-depth impact analysis before each asset is moved into the cloud.

ESG green net low RES

Solidatus has been chosen by the UK Department for International Trade (DIT) as one of 11 UK-headquartered FinTech companies to join a virtual trade delegation to New York Climate Week from Monday, 20 September to Friday, 1 October 2021.

The constant rise of ESG has created a mandate for the FinTech and RegTech companies of the world – as organisations race to become compliant against a backdrop of vague standards and complex data, the need for a clear, concise and innovative solution has become paramount.

With one in $3 invested in ESG and sustainability across the United States, it’s easy to see why the need has become so great. Alongside other pioneering UK firms specialising in ESG, we will be showcasing our award-winning solution to further accelerate the financial industry’s ESG and sustainability goals – from achieving net-zero, to enabling climate action – all through the use of new and emerging technologies.

Speaking about this thought-provoking mission, Kunal Khatri, Her Majesty’s Acting Trade Commissioner for the US comments, “The UK has been at the heart of global financial services innovation for decades, and as we lead up to the COP26 Summit in Glasgow, UK companies are poised to play an even greater part in driving ESG and sustainability across the financial industry. Fintech in particular will have a crucial role to play in achieving net-zero by spurring green innovation and expertise into sustainable investments, by providing companies with better tools to measure and mitigate their climate impact, and by giving consumers the information they need to make more informed decisions. The UK has world leading talent, technology and businesses with an enormous amount to offer to the US market we are excited to bring this delegation of cutting-edge technology firms to New York Climate Week”.

We look forward to working with truly trailblazing tech companies from across the UK to not only enable the financial services industry to better understand their data, and build and develop successful ESG and sustainability strategies, but to be a part of a mission designed to protect our environment and bring about positive change for the future.

Anti money laundering HIGH RES scaled

“I see my team as both my colleagues within Solidatus, but also our customers and the community we work with … I love to see how of our customers are innovating using our capabilities to deliver some amazing outcomes and solutions.”

In the second of four segments with CDO Mag, Solidatus Chief Data Officer Lorraine Waters continues her conversation with Ben DeBow, CEO & Founder of Fortified Data. Together, they discussed the following key points: 

  • How data lineage is enabling cloud services, with Solidatus’ cataloguing capabilities allowing organisations to safely move data to the cloud
  • Solidatus’ strong collaborative capabilities enables businesses to federate the work that needs to be done as they develop their model and blue print
  • Solidatus makes it simple to give and manage access to crowdsource content
  • Our solution doesn’t need to be managed by a central single CDO team exclusively – colleagues and teams from across a business can be engaged to validate the data in a particular system, for a particular policy or regulation
Play Video