Testing infrastructure is like a butcher's knife, WOT Ru Bingsheng tells you how to avoid "falling into the trap"

【51CTO.com original article】Seven years of hard work, a gorgeous transformation. Since 2012, 15 summits have been held for 6 consecutive years, bringing together a large number of technical experts, learning from a wide range of knowledge and making a breakthrough. The 2018 WOT Global Software and Operation Technology Summit has set sail, focusing on 12 core hotspots, bringing together 60 first-line experts from home and abroad to create a high-end technology feast!

At the "DevOps Transformation Road" forum, Ru Bingsheng, technical director of eBay China R&D Center, will give the audience a keynote speech titled "The Evolution of eBay's Test Infrastructure". Before the meeting, 51CTO reporters interviewed him and asked him to "spoiler" in advance. Let's "sneak peek" at the wonderful speech content!

Business is the driving force behind the evolution of high-performance testing infrastructure

Ru Bingsheng, Technical Director of eBay China R&D Center

Ru Bingsheng believes that "knowing what it is and why it is" is the core of learning and deeply understanding the essence of technology. Faced with the complexity of the test infrastructure and test framework of large websites, as a senior test development engineer, it is necessary to fundamentally understand the nature of the problem and the original intention of the design. He told reporters that he would abandon the traditional way of "talking about technology in terms of technology" and instead use business-driven as the entry point, with actual engineering problems as the main line, and the main trunk from raising problems to solving problems, to explain how the high-performance test infrastructure of large e-commerce websites evolves and develops under the drive of business.

The reporter learned that the main content of Ru Bingsheng's speech can be summarized into the following five parts:

The past and present of GUI Automation Test Framework

2. Origin and Development of Test Data Platform

3. The Evolution of API Automation Test Framework

4. Evolution of Test Execution Environment

5. Evolution of Test Report Platform

Through the sorting and explanation of these five parts, the audience can be said to have "benefited a lot". In Ru Bingsheng's opinion, the audience can learn from the speech how to improve the input-output ratio (ROI) of GUI automated testing from the test framework level, how to design and plan the granularity of GUI test scripts to reduce the maintenance workload of scripts, how to design a GUI Flow model based on business processes based on BDD, how to deal with the test coverage problem of multiple browsers, and how to solve a series of test data problems, such as testing dirty data, stress test data preparation, test data diversity, efficient test data preparation, etc.

After the introduction, the reporters who listened to it were a little yearning. Interested netizens should quickly sign up for the conference!

"Speed ​​is everything" is the basic principle

As we all know, eBay is a large e-commerce website. So what are the special features of the test infrastructure of this type of enterprise compared with the test infrastructure of other industries?

Ru Bingsheng summed it up in one sentence: The biggest difference lies in "speed".

Generally, traditional IT companies release products in months, so in this context, the time for test execution, the maintenance workload of test cases, and the stability requirements of test cases will not become key issues.

However, for e-commerce websites, especially large e-commerce websites with global business, the product launch cycle is measured in days or even hours, which places extremely high requirements on the execution time of full regression testing, the maintenance workload of test cases, and the stability of test cases. For example, the time for full regression testing cannot exceed 100 minutes, the stability of API testing must be 99%, the stability of GUI testing must be 95%, the time cycle for tracking and repairing failed cases must be less than 2 hours, etc. The realization of these requirements is directly related to the design of the test infrastructure.

In other words, without the support of an efficient testing infrastructure and CI/CD system, the realization of these goals will be impossible. Therefore, the audience will find that the design principles of the entire testing infrastructure will revolve around the basic principle of "speed is everything".

Have you ever fallen into these test traps?

Ru Bingsheng told reporters that all students who do testing, whether it is automated testing, performance stress testing, or manual testing, will encounter a large number of test data problems, which are mainly reflected in the following points:

Challenge 1: Test data preparation spans across different business departments, and testers lack the ability to build diverse test data;

Challenge 2: Although out-of-box test data improves test execution efficiency, there has been a lack of systematic solutions to the dirty data problem.

Challenge 3: Although on-the-fly test data provides the possibility for test data diversity, the data generation efficiency of the test case runtime will seriously slow down the test execution efficiency;

Challenge 4: Efficiency has always been a bottleneck in preparing test data for full-link stress testing and capacity planning.

Challenge 5: The solution based on Test Data Utility seems to be the best, but the amount of code in the actual use process and the difficulty of preparing related data have always been criticized;

Challenge 6: There is a lack of open source mechanism within the organization for test data preparation for new features, and it is difficult for developers to contribute to the Test Data Utility.

In response to the above practical problems, Ru Bingsheng and his team proposed a one-stop solution called Test Data as a Service, which was fully promoted and used within the company, and has won unanimous praise from various R&D centers around the world. (Good things should be shared by everyone. 51CTO reporter secretly tells you that Ru Bingsheng will share his design philosophy and corresponding implementation technology for this system at the conference. I hope everyone will benefit from it.)

What does a good testing infrastructure look like?

Since he has been focusing on the field of test infrastructure for so long, Ru Bingsheng also has his own ideas about what characteristics an excellent test infrastructure should have. He believes that there are four characteristics that should be possessed:

The first is specificity: an excellent testing infrastructure should allow test developers to only focus on their own test logic without having to pay attention to non-business testing related content such as where the test is executed and how the test data is generated.

The second is uniformity: an excellent test infrastructure should be able to be easily integrated with various CI/CD systems. The ideal situation is Unified Test Execution as a Service. Just a simple RESTful call can initiate a test request. The background test framework, test execution environment, test report, etc. are all transparent to the CI/CD system.

The next is scalability: when a new testing framework (such as Puppeteer, NightWatch, etc.) needs to be integrated, the changes to the infrastructure should be as small as possible, and even seamless integration can be achieved.

The best is flexibility: each module of the test infrastructure can be loaded and used on demand. For example, in our system, the AI-based Defect classification module and the multilingual global site test comparison report module are all loaded by project or phase.

In the interview, Ru Bingsheng revealed that the R&D teams of multinational Internet companies including Google and eBay are actually undergoing an organizational structure transformation of "removing full-time testers". For this reason, Google has also suspended the 2017 Google Test Automation Conference and is seeking to transform to engineering efficiency. Correspondingly, the QE team is also gradually transforming to an engineering efficiency team, and the original testing work, whether manual testing or automated testing, will be completed by the DEV itself. He believes that such a transformation will be highly dependent on the maturity of the test infrastructure. "In the future, the test infrastructure system will move towards the development path of internal productization within the company. The demand for products will come from DEV, DEVOPS and CI/CD, and will be deeply integrated with PaaS and cloud platforms."

On May 18-19, the world's most noteworthy IT technology feast will be held at Beijing JW Marriott Hotel. The 2018 WOT Global Software and Operation Technology Summit will definitely be an important platform for you to discover new ideas, explore the latest ideas, and expand your network.

At present, all our tickets are on sale. Please note that the earlier you buy your tickets, the bigger the discount! Communicate with KOLs up close and present a different "hero feast"!

Click on the official website for details: wot..com

20% off pre-sale, grab your tickets as soon as possible.

[51CTO original article, please indicate the original author and source as 51CTO.com when reprinting on partner sites]