Saturday, June 25, 2011

Continuous Integration - What is Staged Build / Staging of Build

The main purpose of Staging of the Build is to get faster feedback from the Continuous Integration System. A programmer cannot wait for more than 2 hours to get feedback about the check-in. So the Build is distributed in various stages to ensure faster and step by step feedback. This is applicable for typical teams maintaining (or going to maintain) larger code and test code base.  Following are the various Stages.

  • Once the programmer checks-in the code, the Inspection and the Commit Build can be parallely triggered. In the Inspection Build, the tools related to Code Complexity of Code and Test Code, Static Tools and other Inspection Tools like Design Validation , Architecture Validation etc. can be run. Commit Build should give feedback in less than 10 minutes. If teams can get feedback in less than 5 minutes then it is great result. The important point is- teams should try all possible means to get faster feedback from the commit build.

  • If the Commit Build is Pass, then the Stage 2- Secondary Build is triggered. In this Build , the Longer running Integration, Component and System Level Test Cases are run. Typically the thumb rule is to get the result from the secondary build in less than 2-3 hours. To get this result we can run the Build in a Cluster Setup as shown in the Figure above.The inspection build may take longer time based on  the different tools being run. Teams can decide to fail the build based on analysis of the report.s For example if the Static Code errors are introduced in the new build then immediately the secondary build can be stopped.

  • Once the Secondary Build is Pass, the Non Functional Test Build can be triggered. This comprises of Reliability, Availability, Volume, Stress, Performance Related Test Cases. These test cases takes much longer time and it depends upon the kind of System under Test and the related Test Cases that are being executed.

  • Further Stages can be added depending upon the need and also based on the kind of system.


Saturday, June 18, 2011

References for Continuous Integration Tools

In this post, I have tried to bring the various Continuous Integration tools which are available in the industry. Also, below is the link in which all the detailed features of continuous integration tool has been compared. This is a good reference to teams which are planning to choose a CI tool for the development purpose.

An important point while selecting the CI tool is to check if the CI tool supports all the tools related to Continuous Inspection (Static Code related tools, Code Coverage, Design Validation , Architecture validation etc), Continuous Testing ( Functional, Memory leak , volume, stress related test tools), support for the various plugins. This is very important as team can really focus on the core development work rather than focusing on integration of tools to CI which may not be a priority work at the time of core development. With the right tool selection, teams can really improve the productivity.

List of Tools which are available














Friday, June 17, 2011

Agile Story Card Writing - Some Facts

"How to write User Stories in Agile Projects"- I am very sure that this question would have a very different answer from different business analyst, product owners, technical community. Story Card should should follow the INVEST characterstics [ I- Independent, N- Negotiable, V- Valuable, E- Estimatable, S- Small, T- Testable]
You may refer to the following article by Bill Wake where he describes about the details of good Story

In this post I would give a different perspective of the issues teams faces while writing user stories. Writing User Stories from the Customer's perspective and the usage by Developers is relatively easy in Application Development based projects. But in case of embedded , operating system domain teams face issues of vertically slicing the story card and usage by the development and test team.
In my opinion there are following kind of issues. For each kind of issues, I have tried to provide some solutions:

Issue 1- Lack of enough competency where the Business Analyst can really slice the stories effectively cutting across the multi layer (presentation layer, logic layer, persistance layer etc). In some teams the technical members play the role of Business Analyst. And there is no training given to these members. Sometimes it is default assumed that all the members who are technically very good can write good user stories. There is less authentic references available in industry about writing user stories in each domain. User story writing is specific to the domain. It is important that good training be provided to the members involved in writing user story.

Issue 2- There may be some stories that need some background preparation as Spike Stories so that the solution is ready. These strories can also be said as Infrastructure readiness stories. Sometimes these stories are not ready and team members get the stories which are vertically sliced across the layers. In this case the team faces issues related to the technical solution. The solution to such kind of issue is that anytime before the actual development starts, the Business Analyst or the Technical member need to check if the Spike Stories need to be played before the actual story execution. Accordingly, the actions should be taken.

Issue 3- Team members are very junior or dont have enough expertise. In such case stories may need to contain more details and in some cases, solution artefact. There may be also cases where at implementation level, the key algorithm / solution for the similiar implementation can be generalized. Such cases the Designer and the Technical Person need to provide some technical solution so that the implementation is effectively done. This issue may not be there in teams where there are senior members working or where the team size is small.

The important point is that there is no one stop solution for writing story cards. The requirement documentation process needs to be customized based on the situation and domain. If teams face issues with vertical story slicing then team can taken minimum marketable feature based approach and implement the customer requirements. What best fits to the context need to be taken. Story Card has many advantages: if the stories are vertically sliced, it can be developed faster, it gives an end to end value to the customer, customer can give feedback easily, it also helps in other agile practices like continous integration where, faster feedback can be given. However if teams have there own constraints like complexity of the domain (due to which slicing is very difficult), lack of enough competency etc., team can use Minimum Marketable Feature (MMF) approach. The team that use the feature based approach is also as much as Agile as teams using the story card based approach.


Friday, June 3, 2011

How to Plan in Agile

Planning is an essential activity for anything and when it comes to project execution, a good plan gives right direction for the execution of the work. Some questions that comes to me when teams follow agile approach:
" How much planning to do", "Should we do estimation in agile", "Should we assign work to the team member level", "Who all should get involved in the planning" etc.
In my opinion the focus in Agile team should be more towards value driven approach than a plan oriented approach. The focus and direction should be towards iterative and incremental development.

Following are some suggestions for doing overall planning in agile team. In this context I am considering a slightly bigger project (3000 person month). For smaller projects the context and the related activity can be decided accordingly. These need to be considered as only suggestions and not a prescription. During my Consultancy to various teams I have come across such similar questions, so this post. In this I would be only covering the steps and key activities.

Step 1- Project Charter and Vision : The overall vision and objective of the project is decided. Some of the activities which are conducted as part of this are as follows:
• The key features required for the customer is decided. Here the features may be at a very high level. It may not be at an implementation level.
• Some overall level estimation is done. Here the question comes that which estimation techniques can be used. The best is to use Team Velocity based approach. But I have seen that some team faces issues in deciding the team velocity due to reasons like either no historical information available, different team members, not able to identify the ideal story etc. It is even okay to estimate based on effort, lines of code, function point etc. The important point here is to be pragmatic and using a collaborative approach for estimation.
• Based on the work estimation the release dates are decided. In a typical project there can be more than 1 release. The Release Milestones are decided as part of this step.

Step 2- Release Plan : The next step is to have the Release plan in place. A Release comprises of more than 1 Iteration. To come up with the Release Plan following are some of the important activities done:
• Release planning meeting is done where all the team members attend. Before the meeting the Business Analyst / Requirement Analyst provides overview of the User Stories that are going to be taken up. The attempt should be to ensure that all the members are at the same page for doing the feature estimation.
• The overall estimation is being done keeping following key activities in mind : requirement analysis, architecture design, design analysis (if any), coding, test coding, infrastructure preparation, automation infrastructure readiness, training , testing, customer level test etc.
• The Project manager or Scrum master should give enough importance for team training, readiness of the automation, continuous integration, and competency of team. Without good preparation in this direction it may be very difficult during actual project execution.
• Sometimes the project team needs to give commitment to the client about the release dates after the release planning meeting. There may be cases where some of the user stories are not clear. How to give commitment is a challenge which most of the teams faces. While there is no right answer to this, some suggestions - we can plan for some buffer based on our experience and past history. We can make the client intimated about buffer and we can deliver the project early if we are able to do so. Please note there is a difference between Buffer and Padding. Padding (giving some additional effort for each user story level without any basis and not making it known) is wrong and it should not be done. It is like Gold Plating. Buffer is planned to ensure that any risks can be handled due to wrong customer priority, not clear story / technical readiness etc and it is intimated to the customer in advance.
• Based on the amount of work planned, the dates of Iterations are planned. The thumb rule is to keep the iteration fixed for 2 weeks and put the prioritized user stories in each of the iterations.
• The prioritization of the stories are done based on the following (in the order of priority)- customer priority, technical readiness, internal project dynamics and others.
• SCRUM is a very good management framework that can be used for the team structure and following scrum and scrum of scrum teams. This gives very good way of the project management and governance.
• Release Retrospective shall be conducted at the end of each release.

Step 3- Iteration Plan and tracking : The next step is to have the Iteration plan in place. Iteration comprises of stories that are executed and each story has a list of key tasks that are performed to realize the user story. Following are some key aspects:
• Iteration Planning Meeting is conducted where there is an overview given by the business analyst about the user stories being taken up.
• Planning Poker Method can be used for estimation. Refer to book by Agile Estimating and Planning by Mike Cohn.
• If team is not comfortable with velocity based approach then either effort or LOC based approach can be used. It is always good to have a parallel usage of Velocity for the team transitioning into agile way of estimating and over a period we can make our estimation better by using only Velocity based approach.
• User Story level estimation is being done keeping following aspects in mind- story analysis, story design, coding, test coding, TDD approach, local build , continuous integration, testing, release etc.
• Ensuring the “Done “ criteria at each story level is very important. Teams can decide the common process or criteria how a story can be considered as “Done”
• Daily SCRUM meetings should be conducted for ensuring that all team members are in sync with each other’s work and there is a collective ownership.
• At the end of each Iteration Review / Sprint Review should be conducted and the feedback should be included to make the product better. It is always better that the customer or the customer representative is available during the review.
• Sprint retrospective is conducted at the end of each iteration.

Agile Project Management and the planning should be value based and the focus should be to remove the non valued added activities continuously and focus on the value that can be delivered to the customer ,always.


Architecting for Continuous Delivery

This short article will provide details about the various architecture specific requirements for good implementation of continuous delivery...