Mizuho Securities leverages DevOps test data management to transform application development while enabling compliance

Background

Responding to Demands of Securities Industry by Improving Development Speeds
Mizuho Securities provides a wide range of services, such as investment banking and trading. The company’s annual retail and corporate introductory assets exceeded $7 billion for seven years, and they totaled $325 billion at the end of March 2022. Mizuho Securities’ wholesale sector has also established a large presence by acquiring a top market share in offering domestic bonds.

In the securities industry, application development commonly accelerates when market conditions are favorable. But demand for faster releases presents continuous challenges that limit speed, such as compliance with regulations.

“In response to this situation, we created a task force several years ago to automate IT platforms and have been progressively automating operations and management, such as server IP address assignments. One of our strengths is our proactive approach to providing quicker development environments,” said Hirokazu Kobayashi, Manager of Mizuho Securities' IT Infrastructure Management Department.

Objectives

Achieving Compliance with Industry and Privacy Regulations
One of the major challenges that Mizuho Securities' IT Infrastructure Management Department faces is the separation of production and development environments in accordance with Japan’s Financial Instruments and Exchange Act. Mizuho Securities’ databases contain many instances of sensitive data — such as customer names, contact information, and account numbers — that developers manually mask while testing the data. Separation of production and development environments requires the department with authority over the production environment to handle this manual masking, which increases load on the database administrators.

These issues led Mizuho Securities to focus on creating masking automation using the Delphix DevOps Data Platform. “We had five database administrators looking over 800 systems, so we expected that continuing manual masking would create a massive bottleneck for our offered development environment. In order to avoid this, we looked to Delphix. At the time, there was no solution with a track record in Japan, but with our corporate culture, we were able to introduce a good solution quickly,” said Mizuho Securities IT Infrastructure Management Department Chief, Satoshi Ando.

Delphix was introduced at the end of 2016 and became fully operational in 2018. Presently, the virtual test data that is copied daily or weekly can be used for development immediately after replication.

“We used to copy the data for a week after the request and then it took a day or two to mask it, so I think it’s a huge effect that we can provide a development environment as quickly as possible. Our development team has also been highly praised,” said Ando.

Using the DevOps Data Platform for CI/CD Test Automation
The next step after using Delphix was adopting CI/CD for the purpose of automating application testing. Mizuho Securities’ IT Infrastructure Management Department prepared test environments based on scenarios, and the department made pass/fail judgments by comparing the results with expected values. But conducting environment preparation and the results analysis manually required too many hours, which lengthened the lead time for releases and contributed to compromises within the actual testing.

“We thought this could be connected to improvements on Delphix’s testing data self service function challenges, which is the reason we worked on constructing a CI/CD system based on Delphix’s virtual data copies,” Ando said.

The development team that participated in the evaluation test needed data from outside of the database that they were managing for the test. This task produced three major challenges for the team. First, aligning multiple database sections was technically difficult. Second, multiple database managers creating requests and the resulting lead time took too long. Lastly, the team needed time to check differences between the profiles of the databases and to manually fix an issue preventing the profiles from being aligned.

Overcoming those issues accurately measured the effectiveness of Delphix as it functioned within Mizuho Securities’ solution as a whole. The complete picture of Mizuho Securities’ testing automation solution includes a basic system construction of GitHub and GitLab as the source repository, Jenkins as the testing pipeline, Python for data processing and research, and Delphix as the production and testing data manager. The test consisted of Delphix using data sections from multiple databases including reference dates and times with Jenkins kicking the batch and the data linkage via APIs from Delphix.

“There were two big results from the Delphix automation. First was that the testing environment was prepared without the database manager having to do anything. The second was by using the Delphix bookmarking function, it was possible to automate matching of the databases before and after testing in tests that involve schema changes,” Ando said.

Additionally, a data section would traditionally require a week of preparation time to retest— but Delphix’s bookmarking function enabled multiple tests to be performed in one day. “There are many demands for wanting to reuse repeated data sections in development testing, but usually, you would need the database manager to restore the section each time. Due to the bookmarking function, there’s no need to ask the database manager to restore anything, and it was great that the virtual database could be restored instantly,” Ando added.

Business Impact

The implementation of Delphix also produced marked results in cost. “Presently, Mizuho Securities is automating masking mainly for systems that have large needs in the development department, because if it were to be done manually, it would require over 10 dedicated workers. This can be said to contribute to a labor cost reduction of about $700,000,” said Kobayashi.

Delphix shows great promise in surmounting the IT Infrastructure Management Department’s testing environments and speedy delivery challenges. “Accelerating the delivery of developmental environments is considered a much more important issue now. We want to make full use of these functions and take CI/CD to the next level,” said Kobayashi.