Saturday, July 16, 2011

Continuous Integration Build Duration or Time

The important point while running the CI build is to provide feedback as soon as possible. A good rule of thumb is to keep the integration build no more than 10 minutes. The next question would be - what all test cases and inspections tools should we run in the integration build. Well, the 10 minute build need not run all type of test and inspection tools. The key aspect is whatever test cases that could be run as soon as possible and that gives maximum confidence about the system can be run. For Example the Overall Build be separated into stages to get faster feedback. In the Stage 1 only the Unit Test Cases and the Smoke test cases can be run , In the Stage 2 the Functional test cases can be run and in the Stage 3 the non functional test cases can be run. The build is considered PASS from end to end perspective only when Stage 1, Stage 2 and Stage 3 is Pass.Some Rule of Thumb for reference:

Local Build / Private Build < 3 minutes
Stage 1 (Integration / Primary build )< 10 minutes
Stage 2 (Functional / Secondary Buid) <2-3 hours
Stage 3 (Non Functional Build ) - Depending about the type of test cases.
Inspection Build (Can be run in any stages depending on the time taken for each tool)

Typically the Stage 3 build are long running test cases for reliability, performance which has many combinations and it can also take 1 day - weeks depending on the type of test cases. For daily operations for development purpose, the Build till Stage 2 can be considered.

No comments: