The Rubber-Band Effect: How Organizations Are Catching Up To Themselves
In 2020, in response to the pandemic, we saw an urgent shift to SaaS and various emerging technologies. It was covered at length in “Introducing Trends 2021 – 'The Great Digital Switch'.” Largely driven by necessity, organizations needed to make drastic moves “to keep the lights on” and cater to operations in a more virtual and remote style. This big leap forward drastically changed the IT landscape and infrastructure in a lot of organizations.
Overall, this a positive, as it’s driven transformations at organizations to become more modern and data-driven, while overcoming inertia, red tape and regulations. But, paradoxically, now those are the exact reasons organizations are stopping. Sometimes, after taking two steps forward, you need to take one back, or at least stand still for a while. Like a stretched-out rubber-band, the compression from the elastic force is inevitable. As we are gradually returning to a post-pandemic world, we’re seeing some of those effects. See below, where I’ve outlined three areas.
Security: In 2021, security and compliance teams are playing catch-up to the rapid “digital switch” that happened in 2020, because not doing so can carry consequences. What happens if your business stops functioning tomorrow? In Sweden, a major grocery chain had to close 500 stores for several days. Its system couldn’t function after a ransomware attack on their cash-handling software supplier in the United States. What happens if you violate data protection laws? Amazon knows, because it just received an $887M fine for having done exactly that. Regulations are now conflating data management, data privacy, data security, as well as identity and access management. And, the more you embed analytics, trigger actions, and share APIs and data, the more you need to protect against failures. It is, therefore, no coincidence that security just moved to a top priority for CIOs in Gartner’s annual survey1.
Cloud Repatriation and Hybrid: We’re in the middle of “the great migration” of data. This was massively accelerated in 2020, and it generally goes in the direction of public cloud – but not always. Data management and governance initiatives, especially across geographies, are facing complications from a diversity of data sources spread across on-premises, edge, private and public clouds. Specialized workloads exist for a reason. Processing is sometimes faster at the edge. And compliance is critical. China’s new data privacy law will become one of the strictest in the world. In Europe, the massive GAIA-X project is developing the foundations for a federated, open-data infrastructure, aiming to connect centralized and decentralized infrastructures. According to a recent report by 451 Research2, most organizations no longer look for a single all-encompassing solution to their IT needs, but rather an IT estate that accommodates the cost, performance and governance requirements of different workloads. A survey in the same report, with 600 datacenter/colocation respondents, indicated that 48% had transitioned a workload or application away from hyperscale public cloud providers in the past 12 months.
Run-Away Cloud Compute Costs: As cloud data warehouses and lakes have been modernized and broadly adopted in 2020, it’s opened new opportunities to live-query huge amounts of data directly, adding another powerful tool for discovery. But, when you use this technique, you can end up with run-away query and compute costs. And performance can be a concern, too. Rather than using live-query exclusively, organizations are increasingly looking for hybrid data management and analytics approaches that’s based on frequency and latency requirements. A “heat map” of typical queries could show that the majority of questions are exploratory; without the need for real-time updates, they can run in-memory. On the other hand, your more coordinated queries may need to hit compute at the data-source level. For organizations that have made the leap to become more data-driven, insight velocity and cost per insight has increased. This may now require some data hygiene around running the right queries in the right place.
So, after a wild 2020, organizations are increasingly catching up to themselves. The realization that things aren’t binary is coming back. We’re now maturing into in a new situation and becoming more measured. This has prompted more focus on security, where the data sits, as well as cost-per-insight. In 2020, we had to make a fast switch to “keep the lights on.” Now, we have to make sure the wiring to support it is stable, and that the bulb is of a good enough quality, so the light can stay on.
References
1. Gartner, 2021 CIO Agenda: ‘Seize This Opportunity for Digital Business Acceleration,” Published 5 March 2021, ID G00734305
2. 451 Research, Cloud repatriation: What It Is, What It Isn't, and Why It's Not Going Away,” Published September 3, 2021
