Agile Process Performance Model supporting CMMI- Product Quality Assessment

Image Courtesy- http://www.freedigitalphotos.net

Continuous Integration System can be used for effectively assessing the product quality in a very systematic manner. Each product team can select some of the key product quality parameters that are decisive about the quality. In this post we would propose a simple way of assessment which can be built up further by teams to come up with a kind of performance model based on the data that is captured.

Aspect	Details	Measurement/ Formula	Expected Value	Actual Value
Static Code Errors and Warnings	Comprises of selected set of rules OR All rules	Total no. of errors and warnings open	0	<< From CI System>>
Function Coverage	Can be measured for set of Functions OR for all Functions	No of Functions Covered	> 90%	<< From CI System>>
Feature Stability Measure	List of important features OR All features. Feature Stability	Average of the Daily Feature pass rate	100%	<< From CI System>>

Above aspects are just suggestions. There can be other parameters like Code Complexity, Maintainability Index, Line Coverage, Branch Coverage and others. Each team can customize the product quality parameters and measure it accordingly. Each build can be monitored on a daily basis for the key product quality parameters and the historical data can be stored. Using statistical or analytical techniques process performance model can be built based on the dependent, independent and controlled variables among these parameters. Based on the process performance model the product quality can be predicted / assessed at a sprint or release level.

Reference:
http://ppqc.blogspot.in/2008/08/help-understanding-process-performance.html

Agile Process Performance Model supporting CMMI- Feature Stability Prediction

Image Courtesy- http://www.freedigitalphotos.net

The intention of a process performance model is to identify a quantitative understanding of the performance of  the sub-processes which in turn  helps in providing process performance data, baselines and models to quantitatively manage the organization's projects. Process Performance Models are used to represent past and current process performance and to predict future results of the process. At the first glance this looks like very data intensive and very difficult for the agile projects to maintain. In this post we would give some insight into how continuous integration system can support in building feature stability prediction model.


Feature Stability Assessment is one method / technique based on which the feature stability can be assessed and there can be some prediction done for the quality of the product.

When the test cases are being designed, the following naming convention can be used:
<<FeatureName>>_<<UT_TestSuiteName>>
Each Test Suite can be further mapped to the test cases level.
In each build the suite and the test cases which are failing can be monitored closely. Also there can be a trend which can be brought out about the features which are defective in last few builds.
Example-

Feature Stability Assessment can be done at Build Level and at the release level to check the quality of the features being built over a period. This can help in better prediction of the quality of the features and decisions can be taken to focus on specific improvements at feature level. Based on these factors regression models can be built to predict the quality of a product effectively.

CMMI and Agile- Need a balanced approach for organizational success

In our association with various teams of different sizes and different domains, we have come across many questions related to adoption of CMMI® and Agile. Questions like, "We are CMMI® Level 3 Organization, how can we be agile", " Our company is doing Agile, but our customers are looking for CMMI® Level 5, how do we adopt both" etc. In this post and the subsequent posts in this category, we would provide our perspective about CMMI and Agile and how both the good practices can be merged to bring organizational success.

Image Courtesy- http://www.freedigitalphotos.net

Any model or methodology is a set of good practices that have been proven and can be adopted for better results in similar kind of work. All companies wants to deliver the products and services better, faster and cheaper. Nearly all organizations have found that the products and services have varied levels of complexity. It is unusual today for a single organization to develop all the components that compose a  complex product or service. More commonly, some components are built in-house and some are acquired. The integration of these components needs better planning and strong engineering practices.Organization must be able to manage and deliver such complex development and maintenance.

CMMI®-Dev comprises of 22 process areas. These process areas covers aspects related to Engineering, Planning, Support and Organizational processes.  Agile is a way of thinking about software development. The canonical description of this way of thinking is the Agile Manifesto, a collection of 4 Values and 12 principles. (http://www.thinkinginagile.com/2011/03/agile-manifesto.html). Agile Methods are available in different flavors like SCRUM, XP, Lean Software Development (Kanban), DSDM, Crystal and others. Most of the teams adopt SCRUM and XP together. SCRUM comprises of the management practices and XP consists of the engineering practices.

One important fundamental which needs to be noted is : A method or process is a way of working. Whenever we do something, we are following a process. Some processes are written, as-when we are assembling a machine but others are adhoc and informal, as-when sometimes we clean our house. Most of members we have worked have said that CMMI is about excessive documentation. In fact this is not correct about CMMI. As per Appraisal Requirements for CMMI®Version 1.3 (ARC, V1.3) Section 4.3.4," the finding can be derived from objective evidence seen or heard during data collection sessions." This means that for the rating purpose when the data is being collected Objective evidence is seen or heard or observed during the appraisal time. While following the Agile practices there are many instances where the documentation is less. Some examples are given below :

• Daily Standup Meeting is done where the actions are being tracked on a daily basis unlike Weekly Meeting.
• Simple Design is achieved through XP practices like Test Driven Development, Refactoring unlike detailed low level design documents.
• Continuous Integration is practiced instead of dedicated Integration testing phase.

Above list may go on and on. In fact these practices are less documentation intensive but it can ensure that the management and engineering level practices are followed well. It is really important to understand that CMMI looks for the intent of the process area rather than documentation. So CMMI and Agile compliment to each other. The basic issue is about the right understanding about both the set of good practices.

CMMI® provides set of process areas that covers all the activities of software development. Agile methods like SCRUM and XP provides simplified solutions for management and engineering practices. Collection of agile best practices through these methods makes the delivery process fast and robust. It is important to adopt both CMMI and Agile practices together rather than bringing conflicts with each other. All it requires is to balance and customize the process based on the business and the team working for the development. In the next posts we would provide insight about how the good practices of  agile and the CMMI® process areas can be integrated effectively for project and organization success.

Build Optimization Tip 1 - Release Preparation using zipfileset

Continuing with the focus on the importance of Build Scripts in Continuous Integration System, (Refer to previous post : http://www.thinkinginagile.com/2011/05/build-optimization-for-effective.html) we would be covering some of the Tips of Build Optimization in small snippets in Build Optimization Tips Series:

Image Courtesy of www.freedigitalphotos.net

Release Preparation

In any build scripts one of the important activities is the release/artifact preparation. It may be a simple jar/war/ear or a huge zip file which contains different set of jars/wars, configuration files, etc.

What most team do?

a) Create a release directory.

b) Create all necessary folders as per the release structure.

c) Copy all the jars, files etc to respective folders.

d) Prepare the release zip file.

e) Copy the release zip file to the actual release directory.

 

What can be done?

There is a <zipfileset> nested element for jar/zip/war etc. We can use this type to pack the required files or jars from different location.

Sample Code

<zip destfile="${dist}/manual.zip">
    <zipfileset dir="htdocs/manual" prefix="docs/user-guide"/>
    <zipfileset dir="." includes="ChangeLog27.txt" fullpath="docs/ChangeLog.txt"/>
    <zipfileset src="examples.zip" includes="**/*.html" prefix="docs/examples"/>
  </zip>

By this way we can reduce the time to copy different files to a common location.

Refer: http://ant.apache.org/manual/Types/zipfileset.html

Does Process QA role exist in Agile

The above question is a very controvorsial one. In this context QA means Quality Assurance Professional. In some companies QA and QC terms are being used interchangeably. QA professional are eyes and ears of the management. They check whether the defined processes are being followed. If not, provides visibility about the key issues to the management so that timely actions can be taken. In SCRUM, XP or other Agile methodologies there is no clear role like QA, defined.It is left upto the companies to define the roles and responsibilities. However CMMI has a defined Process Area PPQA (Process and Product Quality Assurance) where there is specific mention about some of the roles of QA.
Any organization or project, irrespective of the methodology or life cycle or standards it follows, any role exists due to the value add it brings to the organization or the project. If there is less or no value add some day the role may phase off. This simple rule is applicable for Process QA role too. Following are some of the operational differences of a Process QA role in Agile as compared to Waterfall approach:

• QA should use a judgemental approach towards auditing where there are less records. For example sometimes there may not be record of minutes of meeting or review records as these things happens almost everyday. In Agile there are alternative practices like Daily Standup, Pair Programming for meeting effectiveness and effective reviews. QA should be in a position and should be competent to judge whether the alternative practices are working well.
• Should be able to propose the the life cyle of the project and the required metrics based on the agile process defined.
• Should have good competency to assess the Continuous Integration Situation and Build Management of the team and provide suggestions for improvement.
• QA should be able to understand the domain and the technical details of the project. This sometimes is not a mandatory skills. But if QA has this skill then it makes them more powerful and able to take decision in a more effective manner.
• Should be able to drive key meetings like Retrospection, Kaizen.
• QA should be Agile by himself or herself and should keep upto the pace of the team's progress and ensure that the processes are monitored closely.

The above list may go on and on. We don't wish this post to be a real comprehensive one. The important point is that QA role is very important. Any organization, big or small, require governance mechanism and Process QA professional can really help the organizations to perform better with effective governance system.