The Power of DataOps

Businesses today are under siege, with cutthroat competition and disruption on all sides, cyber warfare and regulatory pressure around every corner. Success will not come to those who hunker down, clinging to their vanishing budget and burdensome processes of yesteryear. It’s a do-or-die world in the digital economy, every day. Businesses must continually re-invent themselves by adapting to disruptive changes like cloud, mobile, machine learning, and IoT. Half of today’s business leaders are unsure what their industry will look like in just three years, and no one wants to be tomorrow’s Blockbuster.

To move fast, companies are scrambling to unleash innovation within their workforce. They’ve made great strides in their Agile transformation. They’ve begun their cloud journey. They’ve implemented new DevOps practices. They’ve hired the best and brightest people to harness the latest and greatest technology and processes. Yet they continue to be disappointed by the results. In fact, 84% of companies still fail at digital transformation, in part because they fail to address one of the most critical aspects of the new digital economy: data.

The digital economy has created an unquenchable thirst for data across all aspects of business.

Every company is now a software company; and the market shift from complex, feature-heavy products for the masses to highly personalized, on-demand experiences, mandates a sophisticated data strategy. Data, and access for those that need it, is a competitive advantage. Those that can leverage data to drive innovation will win; those that can’t, will lose.

The core problem is not data itself, but the data friction caused when constraints on data prevent people from meeting the ever-growing demands of the business. On one hand is an explosion of data needs, users, and environments that mandates data be everywhere it needs to be, for anyone that needs it, in whatever form is most suitable for the task at hand. On the other hand, data has grown exponentially in size, complexity, and cost with escalating security and privacy concerns, pushing to limit data access and availability.

As a result, companies are locked in a Sisyphean battle with data friction across people, process, and technology. Companies made headway with Agile, cloud, and DevOps — for a while. Predictable delivery of compute environments has gone from weeks to minutes, with automated, elastic, and on-demand infrastructure. But data is unlike compute. It’s expensive to maintain, full of sensitive information, difficult to copy, hard to track over time, and slow to deliver to the teams that need it.

So while cloud and DevOps have helped, they are ultimately insufficient. In fact, getting to the cloud and eliminating infrastructure as a constraint actually exacerbates the problem of data friction. As DevOps and cloud tore down the barriers between people and infrastructure, more environments, more automation, and more speed meant increased demand for data, faster. IT is still struggling to manage, secure, and deliver the data environments demanded of them. And users are still struggling to access, manipulate, and share the data they need.

When data friction becomes the blocker to innovation, customers leave, competitors win, and businesses spend more time reacting instead of leading.

But it doesn’t need to be this way — companies can win against data friction. IT can overcome cost, complexity, and risk to become an enabler for the business. Users can get the data they need to unleash their capacity for innovation. And everyone can work as one team to drive massive outcomes for the business. But today’s methodologies are ill-equipped for the challenge. We need a new approach; one that does for data what DevOps did for infrastructure. We need DataOps, a new means to connect people to data, empowering them to overcome data friction and achieve the velocity of innovation demanded by the digital economy.

DataOps is the alignment of people, process, and technology to enable the rapid, automated, and secure management of data. Its goal is to improve outcomes by bringing together those that need data with those that provide it, eliminating friction throughout the data lifecycle.

DataOps has previously been used to describe analytics systems. But this nascent movement must evolve to be broad enough to cover the complete spectrum of data challenges while being specific enough to guide decisions around processes and technology.

Mastering DataOps requires overcoming the organizational and cultural barriers that separate people from data. It starts with bringing two key audiences together as one team:

• Data Operators: Responsible for infrastructure, security, and maintenance. Includes DBAs, security and compliance, system administrators, and more.

• Data Consumers: Responsible for using data to drive new projects and innovation. Includes developers, testers, data scientists, analysts, and more.

But all the cultural transformation in the world won’t help if your infrastructure can’t support the new demands placed upon it. DataOps also mandates a comprehensive technology approach that eliminates key points of friction across:

• Governance — Security, quality, and integrity of data, including auditing and access controls.

• Operation — Scalability, availability, monitoring, recovery, and reliability of data systems.

• Delivery — Distribution and provisioning of data environments.

• Transformation — Modification of data, including masking and platform migration.

• Versioning — Capture of data as it changes over time, with ability to access, publish, and share state across users and environments.

This is a complex technology landscape, ranging from legacy backup tools to new cloud offerings. The key to DataOps is leveraging the right solutions for the right tasks, and making sure they are powerful and easy to use for both data operators and consumers.

Velocity of innovation makes and breaks winners in the digital economy, and data friction is the next constraint. DataOps has the power to bring people and technology together to eliminate data friction as a barrier to innovation so that companies can thrive in the digital economy.

It’s time to tear down the barriers between people and data. It’s time to unleash innovation held hostage by data friction. It’s time to begin the DataOps journey.