Data, Cloud, and Analytics Outlook 2021: Hedging the cloud and looking for Explainable AI

It’s safe to say that 2020 is a year that we would probably all want to regret. It was a year where survival through adaptation became the rule. At the outset of 2020, we forecasted that generational change in back office systems and growing demand for taking advantage of AI services would drive the next wave of cloud adoption. Looking back, countless Zoom meetings later, the pandemic accelerated enterprise adoption of cloud services as reflected in the very healthy double digit growth rates of each of the major clouds. Hold that thought.

So, what’s ahead for data, cloud, and analytics? It’s hard to review any of those domains without the 16-ton elephant sitting in the room: AI. Just as enterprises followed the consumer tech world in embracing mobile devices and apps, history is repeating itself with AI. The appetite for AI services is catalyzing movement to the cloud because machine learning and deep learning are each ravenous for data, much of the data already resides in the cloud, and the cloud is the best place for harnessing the scale of compute to train and run models.

And with greater use of AI comes the inevitable scrutiny. It’s no secret that the sociopolitical climate on both sides of the Atlantic is demanding greater accountability on the part of tech companies, and accountability for how enterprises use technology – read AI – to make decisions. So, while we expect to see more calls for “Responsible AI” over the next year, progress will be limited until we can make AI more explainable.

The appetite for enterprises to harness AI has not been lost on cloud data warehousing services; we’re finding that in-database machine learning will become a checkbox feature in the coming year. Harnessing all that data requires access to lots of data, and so what is the best way to get at it? Until recently, conventional wisdom pointed to the data lake, which typically resides in cloud object storage. As the data warehousing household names have opened cloud-native services, the are taking advantage of the scale of the cloud to promote the relational data lake – the data lake that is accessible to SQL database programmers. As to the Java, Python, and Scala-skilled data scientists who have taken programmatic approaches to building analytics, and increasingly, notebooks to develop machine learning  models, a number of providers are embracing an emerging concept originated at Databricks for the Data Lakehouse – a hybrid of data warehouse and data lake that is managed and performs almost like a data warehouse. We’re likely to hear a lot more of this relational data lake vs. data lake vs. data lakehouse in the coming year – how are the sides going to divvy up?

We mentioned up top about the growing embrace of the cloud. Well, as enterprises find cloud bills consuming more of the IT budget, and more to the point, as they look to onramp more business-critical services, the question about cloud vendor lock-in becomes more than academic. Our take is that the issue will get a lot less clear cut next year as cloud providers start talking multi-cloud and third parties that are supposed to be neutral increasingly find themselves in strategic partnerships. What does this mean for the enterprise?

We air these trends out in our latest annual year-head outlooks that are up on ZDnet. We have so much to say that we’ve split them into two posts.

For our take on the outlook for data warehousing, analytics and AI, click here.

For our take on what’s going to happen with multi-cloud, click here.