Unlike in the V Model and the waterfall model where the testing time is dedicated 2-3 months for a typical was 50-100 kloc projects, in Agile Iterative life cycle this is split into each of the iterations. Each Iteration is tested by the test professionals and is certified about the quality of the release. This work is done in collaboration with the development team. During the iterations the test team participates actively in the brainstorming and provides active contribution towards the various customer scenarios from the test perspective. I have seen that in most of the organization the test staff is relatively less staffed. The typical ratio of Tester to developer is around 1:3 or 1:4 or may be less. The tester tests at each of the user story level. Sometimes the pace at which the developer completes the story is much faster than the tester can complete the story testing since the number of tester is less and also due to poor infrastructure. The developer testing is mostly at unit testing level. For the functional testing mostly the team relies on the testers. Some of the practices of automation like have a scalable automation framework, good test suite organization, test naming convention; test logging etc. is slightly weak from the test perspective. The Iteration level automation is weak and sometimes not complete. Since the Iterative releases are being given at every 2-3 weeks once and the testing also happens parallel. Due to lack of good automation, I have seen test sometimes is not able to cope up with the pace of the releases. Due to this the quality of the software sometimes suffers.
- Test Automation Framework Scalability- Refer to my earlier post “Top Reasons why Continuous Integration practice fail or result not seen “where I have given some aspects that should be considered for the test automation framework. It is observed that the Test framework either focuses on either the tool part or the scripting part. The Test Frameworks does not ensure that the SUT system architecture is taken into consideration while designing the overall automation framework. This creates issues, where the frameworks are not scalable and there are issues related to overall test execution later. Test Teams must ensure that the Automation framework considers all the aspects of testing like the system under test architecture, test inputs, test outputs, SUT precondition program state, SUT post condition program state, environmental inputs and outputs, component dependency, APIs , capturing the results, analysis of the results etc, the various monitoring tools. In the following sample system the automation framework should be divided into each of the respective areas like GUI, APIs.
- Test Suite Organization, Naming Convention - It is seen that the test suites are not organized well based on the features and the functionality. The unit testing, functional suites, performance, reliability, volume testing suites should be rightly organized at each of the feature level. Sometimes the test cases are run as monolithic systems where all the test cases are run at one go with no organizations. There is lack of appropriate naming convention due to which it is not clear which functionality is failing.
Some of these issues is due to lack of enough competency & awareness and others due to readiness for the iterative testing. It is important as agile teams to plug these gaps.
References
1. Agile Testing- A Practical Guide for Testers and Agile Teams- Lisa Crispin, Janet Gregory
No comments:
Post a Comment