3 Best Practices for Test Data Management in Cloud-Based App Dev

Cloud programmability provides DevOps teams with unparalleled flexibility in application development. When infrastructure is code and CI/CD pipelines are automated, enterprise development teams can establish repeatable software delivery processes to optimize workflows and improve productivity.

Yet test data management remains stuck in a bygone waterfall era, hindering release velocity. App dev in the cloud adds unique hurdles that further burdens legacy test data practices, including:

• Diverse data stores. A cloud-based microservices architecture replaces the traditional monolithic data store with multiple fit-for-purpose databases. Every microservice has its own release train, infrastructure and storage are customized, and access is exclusive to each service.

• Distributed data locations. Production data isn’t necessarily in the same location where test environments are configured. An enterprise application stack will have data sources on-prem and in multiple clouds, with both IaaS and PaaS deployments.

• Demand for test environments. When high-velocity DevOps teams work in smaller increments in an automated CI/CD toolchain, the need for test environments grows exponentially. Application teams struggle to create environments at the rate required.

• Noncompliant test environments. A new Pulse survey shows 81% of enterprise IT executives find it difficult to maintain data security and compliance in cloud-based test environments. This isn’t surprising given continually expanding privacy regulations and manual processes for anonymizing sensitive data.

Updating test data management processes ensures data delivery and security will meet the needs of modern application development in the cloud. Here are essential best practices.

Best Practice #1: Automate Data Delivery

Keep pace with CI/CD workflows: Waiting even a few minutes for test data to be delivered breaks microservices efficiency. Automating test data delivery improves timelines for the entire software lifecycle, reducing wait states from weeks, or even months, to just minutes.

Contain costs with ephemeral test environments: Adding a single API call for data provisioning when standing up ephemeral test environments both increases productivity and helps control cloud costs. Seventy-five percent of IT executives say one of the biggest advantages of application development in the cloud is cost savings because test environments can be torn down when not in use.

Improve software quality with production-quality test data: Automating delivery provides a continuous flow of production-quality data into test environments. Software quality is impacted when teams use stale data or resort to subsets just to keep the release train moving. IDC research on Delphix customers showed that shifting left with production-quality data decreased errors leaking into production by a whopping 70%.

Best Practice #2: Virtualize Test Data

Establish data portability within and across clouds: Abstracting data from the underlying storage infrastructure provides a highly agile test data solution. Lightweight virtual clones can be readily transported when production and non-production environments are not co-located. Plus, virtual databases can share underlying data blocks to contain cloud data sprawl and reduce data footprints by 10x.

Provide independent control of virtual databases: Developers have independent, read/write datasets that they can bookmark, rewind, and version control data like code. Associating the state of the database with code changes also helps when triaging production issues, facilitating root cause analysis for faster MTTR.

Sync heterogeneous databases for integration testing: Testing transactions that traverse an enterprise application can be challenging, and thousands of microservices across clouds only increases integration complexity. Wherever the data is located, virtualized heterogeneous data sources are easily synced to the same point in time to increase testing accuracy and improve software quality for the loosely coupled architecture..

Best Practice #3: Eliminate Compliance Risk in Test Environments

Automatically identify and anonymize sensitive data: Profiling and anonymizing sensitive data for use in non-production environments is a key component of a holistic data governance program. API-driven masking leverages regulation or application-specific algorithms to accurately de-identify data while maintaining referential integrity for comprehensive testing.

Conclusion

Application development in the cloud brings speed and agility to the software delivery lifecycle, but a cloud architecture also presents new and evolving challenges for efficiently delivering data into test environments. Modernizing test data management and establishing best practices ensures continuous access to secure, production-quality data that keeps pace with an accelerated DevOps cadence.

Watch this webinar on How to Automate Data for Hybrid Cloud App Dev to learn more.