A Process Based Unification of Process-Oriented Software Quality Approaches

2009 Fourth IEEE International Conference on Global Software Engineering A Process Based Unification of Process-Oriented Software Quality Approaches Zádor Dániel Kelemen1,2 Dissertation advisors: Jos Trienekens3, Rob Kusters3, Katalin Balla1,2 1 Budapest University of Technology and Economics, 2 SQI – Hungarian Software Quality Consulting Institute Ltd. 3 Technical University of Eindhoven [email protected], [email protected], [email protected], [email protected] 1. Research area and sub-area 4. Background Multi-model software process improvement. This chapter gives a brief summary of the solutions arisen in the last decade for the multi-model problem, namely: mappings, combined models, combined appraisals, standardization, formal solutions, a multimodel framework and a multi-model process improvement research. One possible solution for multi-model process improvement is the mapping of approaches. Mappings of different quality approaches are developed to help companies in the synergic use of the mapped/compared approaches. A wide literature is available on the comparison of CMMI to other approaches like ISO 9001, TSP, SPICE, Agile/Lean Development, Six Sigma, PMBOK and IEEE software engineering standards [18]. One of the most widely known mappings of CMMI is developed by Mutafeilja et al [16]. They described a detailed mapping of CMMI-v1.1 and ISO 9001:2000 requirements, comparing each ISO9001:2000 requirement to CMMI specific and generic practices. This source is very well applicable at companies which already have an ISO 9001 quality management system and try to move towards the CMMI process improvement. Combined models are also available. One such combined model is CITIL, which a combination of CMMI and ITIL. It “supports the improvement of both the development and the operation aspects of IT products and services” [21]. Software products are also available which are based on combination of models such as the Appraisal Assistant, developed at Griffith University. The software facilitates the combined appraisal of two similar process improvement models: SPICE and 2. Brief proposal description of the research This research defines a method for process based unification of multiple process-oriented software quality approaches. 3. Description of the research problem and its importance in the field Several quality methods, standards and models are available to guide software developing organizations in defining and institutionalizing their processes. These approaches are essential in improving the company’s quality system, but each of them uses an own view on quality. However, several primarily global software companies use more quality approaches simultaneously. They often struggle with interpreting them, due to their different structure, terminology, level of detail and point of view on quality. Tailoring multiple quality approaches to the company’s needs is a time-consuming process and needs special expertise. It could be especially difficult in global environment. Using a method and/or guidelines can facilitate it, having a positive impact on the work of software engineering process group and on the whole organization, helping this way the company to focus on its business goals instead of wasting time and resource on harmonization of regulations, models and standards. 978-0-7695-3710-8/09 $25.00 © 2009 IEEE DOI 10.1109/ICGSE.2009.39 285 Authorized licensed use limited to: Eindhoven University of Technology. Downloaded on September 21, 2009 at 05:36 from IEEE Xplore. Restrictions apply. The PrIME (Process Improvement in Multimodel Environments) project launched in 2008 by SEI focuses also on multi-model process improvement, having amongst its goals the maximization of the ROI of process improvement in multi-model environment [19, 20]. This is a three-year long project, still in the early stage. It is visible, that a significant effort has been spent on multi-model problem on the part of different researchers, organizations and practical process engineers with considerable results. Mappings are useful when a company tries to use two different models or standards; combined appraisal models reduce the cost of the appraisals; more formal methods could serve good solutions in the future; process modeling facilitate the better understanding and maintenance of processes; QMIM framework helps companies to start their process improvement. Nevertheless these solutions have significant importance, it is also noticeable that the multi-model process improvement is still an issue for companies when they want to use more, or still not mapped approaches. CMMI [3]. By making use of this software, separate appraisals can be conducted as well. Another solution for the multi-model problem could be a well-planned and maintained standardization. Following this was of thinking, the International Organization for Standardization (ISO) tries to harmonize its most widely used standards. Eg. standards 9001, ISO 90003 and ISO 14001 has been developed with the same structure (chapters) and terminology in order to help companies in implementing an ISO 9001 quality management system extended with ISO 90003 and/or ISO 14001. Moreover, if we look for the reason of releasing the newest version of ISO 9001, released in 2008, we can find that a main goal of this release was to enhance the compatibility with ISO 140001:2004. No new requirements were added and all the modifications are focused on concretization and refining expressions in requirements. Similar reasons are defined to release the next version of CMMI, v1.3 [19]. A further example of standard harmonization is the IEEE 1028:2008 and IEEE 12207:2008 synergy. The first one has been developed to facilitate the achievement of different outcomes of the latter one (namely the chapters 7.2.6 – Software Review Process and 7.2.7 – Software Audit Process) [4]. These examples show that standardization organizations also concerned in the problem of multimodel improvement. The problem is practical and ISO and IEEE make steps for facilitating the companies’ situation in this environment. Other researchers create more formal solutions, trying to use the natural language processing know-how for processing standards and models or building ontologies. Eg. an ontology based solution for multimodel process improvement is presented in [15]. Formal methods can be a good solution for enhancing the understanding and improving software processes. However, most of these methods are at the beginning of their development, still not widely accessible and applicable by quality/process engineers in real environments. Another approach for combining different quality approaches is the QMIM (- Quality through Managed Improvement and Measurement) framework, developed by Katalin Balla [1, 2]. This framework shows how the quality standards and models are connected to quality objects (process, product and resource) and quality specifications (definition, quality attribute and metric) predefined in the framework. Such a solution could be very useful when a company tries to understand in which way needs to move and what are the possible models and standards that can help in moving towards that direction. 5. Research focus, questions and objectives In the background section a brief review of current solutions for multi-model process improvement was shown. It was also discovered that there is still no solution for software companies which want to tailor more than two quality approaches to their needs or want to use still not mapped quality approaches. In this research we are focusing on the harmonization of process-oriented (software) quality approaches, excluding in this way several other approaches in which the main focus is not on the process but on other quality objects, such as the product or resource. We would like to give an alternative for using multiple process-oriented software quality approaches by defining a method taking into account the main current issues in the multi-model process improvement. Table 1. shows the research questions and connected research objectives. Research questions 1. Is a process model based method usable to support the use of multiple processoriented software quality approaches? Research objectives 1. To develop a process based method for supporting the use of multiple process-oriented software quality approaches. a. Does the outcome of the method provide sufficient information a. To determine the completeness (sufficiency) of the material developed using the method. (eg. 286 Authorized licensed use limited to: Eindhoven University of Technology. Downloaded on September 21, 2009 at 05:36 from IEEE Xplore. Restrictions apply. Research questions to unify selected parts of different processoriented software quality approaches? b. Is the outcome of the method traceable? c. Is the outcome of the method adaptable and expandable? d. Is the outcome of the method usable in practice? oriented quality approaches which can be used to build organizational processes. We will present elements which can be directly converted to process elements, as well as elements which contain information which cannot be converted directly. This kind of quality approach elements need to be processes “manually” before a unification. The method will help companies to make the most important steps for using quality approaches (step 3). At step 4 a peer review material will be developed which will include a unified peer review process, templates and guidelines and training material for the process. Important characteristics of the outcomes such as traceability, adoptability, expandability and completeness will be on the focus thorough the steps 2-4. Research objectives to verify that a process can be built on the peer review material developed using the method) b. To sustain the traceability of outcomes. 1. To identify the main element types (element – a part of a quality approach, which can be mapped to a process element, for more details see [13]) to be extracted from quality approaches. 2. To determine the level of traceability of derived elements back to the selected quality approaches. c. To identify possible reactions to the changes in and number of quality approaches. 1. To identify the adaptability of the developed material to the new versions of quality approaches; to develop a guideline for managing such changes. 2. To identify the expandability of a developed unified material with further quality approaches; to develop a guideline for managing the expansion d. To use the method and its outcomes in practice. 1. To draw conclusion about the practical usability/ applicability of the material developed based on the theoretical method. 2. To tailor the peer review process to a concrete organizational needs from multiple process-oriented quality approaches using the method. Table 1. Research questions and objectives 6. Research methodology The research can be divided to a theoretical and an empirical part. Figure 1. shows the outline of the main research steps. Steps 1, 2, 3 and 4 are more theoretical and are intended to answer questions 1, 1.a, 1.b and 1.c. Steps 5-10 are connected to the question 1.d and are more empirical in their nature. Step 11 represents the final evaluation and generalization of the research. Theoretical part At the beginning of the theoretical research a critical review on the available literature will be conducted (step 1). The second emphasis of the theoretical part will be on the discovery of the structure and the main elements of process-oriented quality approaches (step 2). The methodology will show elements of process- Figure 1. Research steps Empirical part The second, more practical part of the research will be the concretization of the method. This part will validate the theoretical part and answers the question 1d. The empirical part will be divided into three main steps such as: • Conducting expert consultation (5 and 10); • Redesigning the peer review material and the method (6 and 8 if it is needed). • Conducting a peer review case study at Hungarian and/or Dutch company (step 7 and 9 if it is needed); 287 Authorized licensed use limited to: Eindhoven University of Technology. Downloaded on September 21, 2009 at 05:36 from IEEE Xplore. Restrictions apply. Steps 8 and 9 are optionally performed based on the results of the first case study (step 7). As a conclusion of the research the possibilities of evaluation and generalization will be analyzed using the results of both the theoretical and empirical parts (step 11). Measurement”. In Proceedings of European Software Control and Metrics Conference, London , pp. 315-326 [3] Griffith University: Appraisal Assistant and its connected publications, 2009 https://www.sqi.gu.edu.au/AppraisalAssistant/about.html [4] IEEE 1028-2008 IEEE Standard for Software Reviews and Audits [5] ISO/IEC 12207-95 – “Information technology - Software life cycle process” [6] ISO 9001:2000: Quality Management Systems. Requirements. [7] ISO 9004:2000 Quality management systems – Guidelines for performance improvements [8] ISO/IEC 90003:2000 Software Engineering – Guidelines for the application of ISO9001:2000 to computer software [9] ISO/IEC 15939-2002 – “Information technology Software measurement process” [10] ISO/IEC/IEEE 12207:2008 Systems and software engineering — Software life cycle processes [11] Kelemen Z. D., Balla K., Bóka G.: “Quality Organizer: a support tool in using multiple quality approaches”, in Conference Proceedings of ICCC 2007, pp. 280-285 [12] Kelemen Z. D., Balla K. „Comparing CMMI-DEV v1.2 and MSZ EN ISO 9001:2000”, article in Hungarian Quality 2008 February, pp. 6-12 (in Hungarian) [13] Kelemen Z. D., Balla K., Trienekens J., Kusters R. „Towards Supporting Simultaneous Use of Process Based Quality Approaches”, in Conference Proceedings of ICCC 2008, paper id: 69 [14] Kelemen Z. D. “Structure of Process-Based Quality Approaches: Elements of a Meta-model for Process-based Quality Approaches and Methods”, Proceedings of the EuroSPI 2008 Doctoral Symposium. Dublin, Ireland, 2008. [15] Malzahn D.: Ontology based multi-model process improvement for business organizations, SEGP Europe 2008 [16] Mutafelija, B., Stromberg, H.: Systematic Process Improvement Using ISO 9001:2000 and CMMI 2003, ISBN 1-58053-487-2 [17] Software Engineering Institute (2006), – “CMMI for Development, Version 1.2” [18] SEI publications on CMMI mappings and comparisons https://www.sei.cmu.edu/cmmi/adoption/comparisons.html [19] SEI PrIME project, 2008 https://www.sei.cmu.edu/prime/primedesc.html [20] J. Siviy, P. Kirwan, J. Morley, L. Marino: Maximizing your Process Improvement ROI through Harmonization, 2008 https://www.sei.cmu.edu/prime/documents/multimodelExecut ive_wp_harmonizationROI_032008_v1.pdf [21] CITIL Webpage https://www.wibas.com/training/citil__cmmi__itil/citil_citil__ cmmiitil_compact/index_en.html 7. Results achieved so far A comparison of MSZ EN ISO 9001:2000 and CMMIDEV v.12 was presented at Hungarian Quality Week, in Hungarian [11]. A web-based software has been developed to support the use of multiple quality approaches a with the QMIM framework [10]. These two experiences serve a good basis for research step 1. The structure of the following quality approaches has been analyzed (and partially published in [12,13]): CMMI-DEV v1.2, ISO 9001:2000, ISO 9004:2000, ISO/IEC 90003:2000, ISO/IEC 15939-2002, ISO/IEC 12207- (-95 and -2008) and IEEE 1028:2008. Findings of this analysis will be more elaborated and extended to be used at the research step 2. A unified peer review training material was developed using main steps of the proposed method, taking into account the following process-oriented software quality approaches: CMMI, ISO 12207:2008 and IEEE 1028:2008. The training material was used in practice four times in 2008 and 2009: at IRR Software Testing Workshop in Budapest 3-4 hours presentation and practice, at a Hungarian mid-size, IBM-partner company, providing a 5-hours-long training, and also at BUTE Software testing laboratory in two combined theoretical and practical lectures. 8. Expected contributions of the research The developed method will fill an important gap in the field of multi-model software process improvement. Companies using more than one or not mapped process-based software quality approaches can follow the steps defined in the guidelines and tailor their own software quality system from different parts of quality approaches. As an additional product of this research, a unified peer review process and training material will be provided. 9. References [1] K. BALLA 2001, The complex quality world. Developing quality management systems for software companies. Ph.D. thesis. Beta Books, Technische Universiteit Eindhoven, 2001. ISBN: 90-386-1003-3 [2] Balla K., Bemelmans T., Kusters R., Trienekens J. J. M . (2001) “QMIM - Quality through Managed Improvement and 288 View publicationlicensed stats Authorized use limited to: Eindhoven University of Technology. Downloaded on September 21, 2009 at 05:36 from IEEE Xplore. Restrictions apply.