A similar view is shared by the leaders of software companies, seeking to reduce the implicit costs of production. Although many of them are not noticeable at first glance, they are quite noticeable effect on the financial result. Among the implicit costs include higher costs due to inefficient testing of developed programs.
Any more or less significant upgrade of software code requires cross-checking its operation. Used for this test are often typical and can be repeated many times in successive cycles modernization program. This is true for desktop systems, and for "heavy" information systems – from Web portals, which play a representative role, to the financial accounting systems. Apparently, so more and more managers realize the importance of automated testing, which will be discussed in this article.
The main purpose of the test automation – reducing the cost of testing the program after its modernization. Periodically repeated the same type testing are time-consuming in the development cycle. Automation reduces the testing phase and releases the main resource of the company – the working time of specialists. Other, no less obvious benefit of such testing – improving the quality of testing that ensures product reliability. After losses from defects, identified only at the stage of commercial operation, can be very high. A dissatisfied customer of them are hard incalculable.
Until now, most companies still practice tests without the use of automation. Sometimes it is formed by a team of specialists. But more often involved in testing by the developers. They’re trying to do all sorts of users and check whether the program achieved the expected result. Testing is conducted mainly on the technical side for "works – does not work, regardless of the expectations of the business. This practice, which we call manual testing, does not allow the complex to check the full-featured system for the designated project period, which leads to various negative consequences.
Here is an example of a situation with negative consequences. The Bank offers its customers use "electronic purse". If, due to software defects to the client shows an incorrect balance of the account. As a result, the client is unlikely to again take advantage of this service. Another example. The mobile operator uses an automated information system that records the clock received from subscribers across Russia. Latent defects of this system are fraught with very large financial losses.
If a similar situation, you know, it’s time to think about test automation. First of all, automation can improve the reliability of software and reduce the risk of defects at the stage of commercial operation. Improves the accuracy of testing and the possibility of finding more defects in the early stages. It becomes possible to identify and eliminate performance bottlenecks throughout the system development life cycle. With automation you can see an accurate picture of system performance at all levels, including entry into commercial operation.
Technologically, automation allows you to use the same tests repeatedly. Tests run faster, thus reducing costs (released time, which used to take to manual testing). Repeatedly increases the test coverage. It is possible to carry out inspections of new or modified applications for compliance with the requirements of functionality and performance.
Automation solves the problem of preparation of the initial test data representing the documents, incoming messages, etc. For a comprehensive testing is necessary to create a significant number of these sets of data. How come in manual testing? Of course, restrict the initial data, based on certain assumptions. If automated testing is not necessary to restrict the sets of input data that can cover a lot more product features. You can conduct testing at the fullest set. IV compared with manual testing, automated tests can be performed much faster.
Important form of testing – load testing. Carry it by hand is very difficult. While all known cases of manual load testing, when a team of 10 people at one time tried to load the system, but it’s more from the anecdotal situations. At the same time there is a tool for testing software systems under the same loads both during operation and under peak loads. This tool allows you to simultaneously emulate the actions of any number of users in the system. System running 24×7, serving millions of customers, must be extremely reliable. Tools for automated testing can detect the limit of reliability and stability.
Starting a project to automate testing, first of all, you must select the tools for automated testing and train personnel. When selecting the tools we recommend guided by such factors as:
We also strongly recommend to pay attention to unified management environment testing. The advantages of using them is the ability of different types of testing and monitoring results in a single shell. Using a unified environment allows the monitoring of test requirements and defects in real time, which allows you to make a coverage map and a picture of risk.
Unified test management enables a holistic control of the production process and evaluate the effectiveness of the quality system. Within a single environment increases the efficiency of interaction within the global system between the various departments of the company throughout the entire process of quality control.
For example, one of the media, test management is a product of Quality Center of Hewlett-Packard.
Personnel to perform automated testing makes sense to cook from among its core professionals involved in manual testing. The reason for this – very low number of professionals in the labor market and lack of serious training programs. On the other hand, modern computer-aided testing are fairly simple to master. Use them may even novice testers after baseline examination. Fastest teachable experts owning programming languages.
The group of automated testing is very important to include people with different skills and initial training: some of them are engaged in a test analysis and test design, while others – directly coding, and others – running and monitoring the results. It is important that the group had an understanding of the system components, how they interact, an understanding of the transport layer.
The level of training and qualifications of members of the group may seriously vary. In this regard, it is recommended to use a common methodology, in which the work will be carried out.
The basic idea is to develop a model based on the allocated project roles. During its development it is important to adhere to the following recommendations:
You can select multiple levels of the model.
Test analysis. At this level usually goes the transfer of knowledge between business users and systems analysts. The result of transmission – high-level test cases. An example of such a high-level test-case: log-in, doing a search postings, report generation, the output from the system.
At this level, an analysis of the object test, the definition of system states, transitions between these states and conditions of transitions. Identification of leverage, signs of a system’s behavior, the definition of subordinate systems and facilities. Defined processes to be automated. Identifies the general nature and regularity.
Test design. At this level is designing automated test scripts, define the basic functions of the library. Also produced a strategy of naming tests.
Development tests. At this level coding test their debugging.
Running tests. At this level is run and control run tests. Tests are grouped according to the criteria defined testing strategies.
When writing automated tests recommend that you follow the recommendations regarding the naming of the structure and parameterization tests.
Naming Test. Each automated test must have the title to the information message "Test X exited with an error" significantly narrowed the list of damaging the functionality behind the error.
The structure of tests. The highest level of automatic test that directly determines the test script must be linear and contain no more than 6-7 steps. Every line of code to improve the visual coverage shall not exceed the width of the page of text, horizontal scrolling is not valid. The test suite for execution should be able to be represented linearly. Action, independent from the others and performed more than twice, shall be made in separate functions.
Parameterization. It is recommended that the concept of the default settings for high-level steps of the script. This is somewhat less efficient understanding of the test at the initial stage (not yet studied and mastered the context of use of those or other methods), but upon the expiry of exploration techniques will increase the information content of each test and its visibility (why man ever to see the options that 80% of cases remain unchanged). For the values of some parameters, whose number is small, enter mnemokody. For example, for a list of currencies, etc.
Part of the above recommendations is due to psychological factors that arise when the group testers. In particular, the linear representation of the test and limit the number of steps to bring you to their writing, people with different skills. Novice testers tend to think linearly, so this idea to test them the most natural, that can quickly be drawn into the work. They can write scripts based on the linear makrokonstruktsy. The implementation of such makrokonstruktsy – a task more qualified testers. In addition, the linear representation increases the readability of tests and their heritability.
The second point – performance tests on a single screen without scrolling. This is important as the information supplied on a single screen, placed in the context of thinking. Scrolling violates the context of thinking, as a result of disrupted holistic view of what makes this test.
When the automated testing tool is selected and the group formed, it makes sense to think about plans for automated testing.
Among the stages of the plan test automation classically distinguished analysis, planning, design, development, debugging and running.
According to the results of the tests is important to consider the following factors:
Application of the above recommendations and methods allow companies to successfully implement a process for automated testing for a functional complex intensive products. The replacement of manual regression testing of automated greatly increases the efficiency and speed of this type of testing, which in turn leads to a decrease in labor costs and, as a consequence – to reduce costs.