Problems of testing in general and stress testing in particular is well known among software vendors (software). It is known that, in contrast to the functional and regression testing, which focuses on testing the completeness of coverage of all branches of the algorithm of the subsystems (applications) and the backward compatibility of the functional with older versions, a "headache" departments exercise testing is precisely the perspective of the approximate simulation to the reality of the load relatively few resources (both hardware and human), as well as finding bottlenecks in the functioning of the whole system.
All compounded by the fact that the process of issuing new versions or patches of errors in relation to billing systems, looks a little different than any other software. Here, the traditional process of "Staging – Development – Testing – Implementation – Support comes in a very large number of relatively small components (subsystems). Since the billing system – a product of a living, constantly changing, the number of subsystems produced per day can be measured in tens.
Attempt to "integrate with" business-process software release identified a number of conflicting objectives, where it is necessary to seek a compromise.
First, each scenario is the use of billing separately (in the image and likeness, as is done at the customers) for stress testing is impossible to recreate (the number of clients is steadily growing volumes of data to be processed grows geometrically). On the other hand, make a universal stand on which to try to fully test the load all the functionality of billing and not simply because there is mutual exclusion as the algorithms within the same subsystem and the mutual exclusion of subsystems as a whole.
Second, as practice shows, the complexity of the preparations for the load testing, in contrast to the functional (regression), estimated totally different scale. By the degree of complexity of preparation for exercise testing, the software can be divided as follows:
Third, a separate task is profiling loads. It is no secret that every application can work well individually. However, in view of "interprocess" correlation, the results of groupware applications, can be very unpredictable. Hence – the more accurate the profiling done loads for each of the applications, the better will stress test, and as a consequence, the conclusions drawn. There is also an application can be divided into "light" and "heavy." By the light, in terms of profiling, include those whose input flow measure (for example, the number of processed records for some period), or, if it comes to client workstations, a set of operations leading to the appearance of a certain amount of information per unit time . By "heavy" in terms of profiling, are those applications whose work poses a serious burden and, thus, leaves no "footprints" of their "life" (for example, the work «Call Center» – a group of users, giving information on telephone and, accordingly, only viewing information). Another example is a batch job (or set), process the data generated by other processes. Here, for example, there may be an avalanche effect, where due to imbalances in any of the grounds, the subsystem does not behave like in a real operation.
Currently, the group exercise testing "Peter-Service", based on the developed test methods of billing system «PETER-SERVICE BIS», was seeking answers to a rather narrow range of issues. Among the tasks we have set, such as the answer to the question of raising / lowering the productivity of key business flows billing with the upgrade version of ORACLE with the existing (9 Release 2) to the new (10 Release 2) or a comparison of hardware from different vendors at the same load scenarios. And to answer such questions today is quite simple. However, fully integrate into the business process of software production is not yet possible because of the complexity of solving the problems voiced above. Nevertheless, we are looking forward with optimism.