This article discusses the decision Compuware to work together developers and testers in the localization performance problems, which can dramatically reduce the average time of elimination of defects for this type of testing.
Guarantee of performance and scalability of complex distributed applications running on Microsoft platforms or Java, – not an easy task facing the participants in the process of development. The complexity of this problem is compounded by the fact that the TE, who are performance problems, can rarely understand what they are due, and conversely, those who are able to explore such issues, it is rarely by those who are looking for.
In other words, developers are better investigate and find causes of problems in performance and scalability, because they wrote the code and understand its structure and logic. However, testing the code only at the level of individual functions, they rarely check the performance at the level of the entire application. Moreover, their tests are usually not reproduce the actual behavior of users and does not contain representative data. Thus, developers are often unable to test the application so as to realistically evaluate the performance of applications, including under different loads.
Testers, on the contrary, it is better able to estimate application performance, that is, checking that the application meets the specified performance requirements and sustains the load in the form of a certain number of concurrent users.
Consequently, the problem is that testers could provide information not only about the fact that the problem of performance or scalability exists, but also about where to look for its cause. Developers rarely examine the application in conditions close to real, so in that sense they are dependent on the testers. In turn, testers rarely provide developers with all the necessary and sufficient information about the problem.
Compuware offers such an opportunity in the form of solutions Automated Software Quality, or ASQ (Automated Software Quality). ASQ consists of two parts – the first part provides automatic generation of virtual users to simulate a real load, and the second – provides the opportunity to analyze the performance of individual operations on the code level.
Together, the two halves of the ASQ allow developers and testers work in tandem to ensure that the application meets the requirements of performance and scalability. If in the process of testing the performance and scalability problems found, the developers have all the information needed to quickly find the causes of these problems and return the application in testing or operation with minimal time on the correction.
Compuware ASQ solution implemented for the two most common distributed platforms – Miscrosoft. Net and Java. ASQ can be used for specific applications, as well as for distributed applications – like a thick client and Web applications.
IT organizations can find their own unique way of using the ASQ solutions for research problems in performance and scalability. Nevertheless, if we speak in the context of the joint work of developers and testers, then there is the classic approach, which should start with exercise testing on beta, using the tools of Compuware QACenter Performance Edition.
QACenter Performance Edition enables testers to create virtual users and configure the load so that it was as realistic as possible. For example, you can configure so that users begin to perform the transaction at different times, so that saved a pause between the actions of users, etc.
In addition, QACenter Performance Edition collects and analyzes a wide range of data on these users, applications and operating system. For example, queried, and demonstrates the use of memory and CPU time, response time for each user and the overall response time, load change over time – as you add and remove users.
This type of testing and output information is a valuable resource for the testers, because their task – to evaluate the performance, response times and scalability of applications. An application was tested for performance under different load levels. The diagram shows that while the load was less than 100 virtual users, response time was acceptable and was kept at approximately the same level, but with the addition of new users to the response time has been increasing, as well as increased and server load.
Compuware QACenter Performance Edition generates virtual users. Simultaneously collected data server, providing a visualization of the relationship between increasing load, response time and the use of server resources.
With this information we can conclude that for this test configuration, there is a limit to the number of users that it can withstand. CPU utilization on the test server increases as you add new users. When the number of virtual users reaches 200, the CPU usage is almost constantly at 100%. At this point, the response time starts to rapidly increase. Obviously, the test server no longer cope with the load, which generates QACenter Performance Edition. Thus, we found a bottleneck in the performance of our test environment in the form of processor capacity server.
This information can help testers to make a prediction as to whether the level is a real nazgruzki, where application response time will be unacceptable to users. Further, this prediction can be used already in the planning and management capacity and level of service quality (SLA).
So, QA-Team, successfully coping with their role as finders of defects, is powerless in assisting developers to study the causes of problems found and remove them. Testers, of course, can claim that the defects – is entirely guilt and responsibility of the developers, but to accelerate application development, which is a vital requirement of reality in our days, testers need to be proactive and do everything possible to ensure that defects are corrected quickly.
And here come to the aid of the second part of the solution Compuware ASQ. Using the performance analysis Compuware DevPartner, testers can perform sophisticated analysis of performance directly during exercise testing and see what parts of the code are slow. With the help of another function DevPartner – analysis of memory – they can easily detect the problem of memory allocation or diversion for Java and. Net technologies.
DevPartner performance analysis feature allows you to see the distribution of transaction times for each function, and even the lines of code, and allocate time of the function and execution time of all calls to other functions within it. As a result, you may want to consider an application’s performance from all sides and quickly detect slow-running sections of code.
Despite the fact that DevPartner is primarily intended for developers, not testers, however, using it together during the exercise testing, testers can do more than just predostavyat reports of unsatisfactory performance of the application. They can provide developers the information needed to analyze, diagnose and repair problems, reducing the average time ustraeneniya defects.
Moving through the list of methods to easily identify what fukntsii or functions carried out slowly with increasing load. In this case, the analysis should focus on function, accesses the database, since this function takes longer than most other operations. A more detailed analysis of the code showed that the database connection that is opened for reading, not zakryvaetya after reading. This lack of close operation causes the connection to leak: the object is no longer used, but there is a link to him so he can not be removed by the garbage collector. If you look at the column Count we can see that this operation was carried out several million times, resulting in the amount of memory capacity has grown significantly, and productivity, respectively, decreased.
Chart memory DevPartner confirms the presence of a memory leak caused by a lack of explicit close connections to the database. On it shows that the amount of memory is steadily increasing. Explicitly close the connection eliminates leakage and increases performance and reliability of the application.
DevPartner performance analysis itself takes quite a few resources. Tests show that the data collection process requires 2.5 to 3 percent of the CPU. If, however, this level of additional overhead costs can distort the results of exercise testing, we recommend that you perform stress testing alone to detect whether there are problems of performance or scalability, and then do it already, along with performance analysis to isolate these problems.
To use performance analysis DevPartner need to install any DevPartner Studio (Microsoft), or DevPartner Java Edition c DevPartner Studio Server license or DevPartner Java Server.
Providing good performance and scalability of distributed applications need not be complicated and expensive. Combining Compuware solutions for load testing and performance analysis, testers are now able to provide developers with all the necessary information to analyze, diagnose and correct performance problems without special technical training or knowledge of the source code. If during exercise testing reveals a problem, they can connect to the configuration of Compuware DevPartner and gather information that allows application developers to quickly and easily find and correct defects. This approach can be used to Miscrosoft-, Java-applications, or combinations thereof, for applications with a thick client, Web-based applications or services.
The described approach significantly reduces the average time to fix defects (Mean Time To Repair). Applying it, we can achieve shorter development cycles, or refine your critical applications, thereby reducing operational costs and providing competitive advantages. The application of the approach provides greater application reliability in service, reducing costs and poddezhrku losses during downtime.