Agenda Day 1

8.45 Registration

9.15  Opening
C. Ferrarotti  Conference Manager
R.Meli  Conference Chiarperson
T.Dekkers Program Committee Chairperson

9.30 Risk measurement & perception : a critical issue
Roberto Meli (DPO, Italia)
Risk Management is a rapidly growing knowledge and process area within the discipline of Project Management. Risk Management starts from an intuitive perception of the factors that can represent obstacles to the success of a project, goes through an increasingly objective evaluation of the same factors and then proceeds to identify a plan of adequate responses. The measurement (estimate) of risks is fundamentally based on the measurement (estimate) of the probability of occurrence of an event and the measurement (estimate) of the damage associated to it. In addition, it is necessary to find a way of "summing up" numbers which do not own the additive property.
The new area of Cognitive Science is revealing new paths of knowledge related to the so-called Cognitive Traps or Tunnels that, such as optical illusions, can lead to wrong decisions on the basis of strong intuitive biased believes. Cognitive illusions may lead to identification of false positives or false negatives, i.e. situations where we take care of a factor it would be better to neglect or, on the contrary, where we neglect to consider a factor that would have been better put into analysis. In both cases we are experiencing a waste of resources. Knowing the cognitive traps can avoid dangerous falls and can allow adequate adjustments to the Risk Assessment phase.

10.15 No Metrics, No Risk Control
Celestina Bianco (NTE SA, Spain)

Analyse principles and steps of Risk Management
Analysis of data to weigh risks in common projects
Correlate process indicators with project risks

Risk Analysis and Control, Risk Driven, ALARP (AsLowAsReasonablePossible). all these are known principles and basic in Critical Software.
Next version of ISO 9000 will recognize the importance of Risk Management in any company, and will include the process among those needed in the management of the organization.
Risk Management is a stepwise and cyclic activity: analysis of potential risks, of the root causes, weighing the consequences, design, implement and verify protections. Then control the efficacy of the protection and update the list of risks during the life cycle.
There is a risk in risk management: try to do it without objective evidences. Weighing the risks before and after mitigations is the key to an effective risk control.
ISO 9000:2000 has introduced the concept of indicators, mandatory for the evaluation of the company's objectives, of process performance, of quality of product and services. Those measures will be the weights and checks in the Risk Management of next version.
In an engineering company, when producing critical apparatus and software, the Project Plan includes a Risk Management Plan. A simple mapping can be made from typical risks of a software program, in terms of time and quality, for example:
"Requirements not properly specified",
"High change request rate",
"Use of non-innovative platforms",
"Newcomer project leaders",
"Newly entered business",
"Limited resources, time, budget", 
"Use of III Party Software",
"Difficult to access to customer acceptance criteria"
An analysis and control of risks through process indicators, gathered during the program or during previous ones, such as:
"Number of Requirements"
"Percentage of changes"
"Testability issues"
"Efforts distribution for activities"
"Schedule deviation"
"Test Coverage"
"Defect distributions (for originators, phase, regression, ..)"
This presentation illustrates and analyses examples of risks, mitigation and effective mapping to indicators for control of efficacy and examples of failure of risk control due to failure of use of objective indicators.

10.45 Evaluating project risks through utilization of software process measurement
Jari Soini (Tampere University of Technology, Finlandia)

* The study that we have executed provides empirical information related to software measurement. The results show where the measurement is focused in practice and which are the less measured sectors, and thus the potential risk areas in which a process could fail during software development work. This paper aims to enhance the awareness of the relationship between software processes and process measurement as well as recognizing the sectors that receive less or no measurement at all in practice and therefore as a consequence, are potential risk factors in software projects..

Nowadays software projects are carried out mainly by means of processes. The basic idea behind using processes is to optimize operations and also stabilize the quality of the software products. The quality of the software product is just one of the key aspects when evaluating the final result of the software project. On the other hand, it is well known that the quality of the product is highly dependent on the quality of the process used during its creation. Moreover), from the process point of view, measurement is one of the key instruments for controlling the process and its quality. In practice, the purpose of software process measurement is to control and monitor the state of the process as well as the product quality and also to provide for predictability. It can be stated that measurement offers an instrument, which enables software development work to be more visible. Measurement can also be used to help perceive the potential risks well before they can evolve into a real problem. Furthermore, experience has shown that the utilization of process measurement to prevent problems from occurring, as well as recognizing the possible risks related to software development work is a far from easy task. In the software industry in general, the nature of the software products being developed is unique and this aspect causes certain challenges when performing and and focusing on measurement during the implementation of the software development process. However, with the help of process measurement, both product quality and  process performance can be monitored and also affected positively. There are a lot of measurement targets on offer, but in the real world, the volume of measurement is effectively limited by time and money constraints. This paper presents, how measurement of the software process and product quality is implemented in practice, based on insights gained from experience in the IT industry. Firstly, the results of an empirical research project illustrate, where the measurement is usually focused. Secondly, we evaluate the possible risk factors for software project success in relation to the process sectors, which are less covered, or totally without measurement. Finally, we discuss the probable reasons for observations made i.e. that process measurement research and practice lag behind the use of other measurements and its possible correlation with the difficulties related to measuring software processes.

11.15 Coffee break 

11.45 Risk Framework for Outsourced Strategic IT System Development from the Client PerspectiveLili
Marziana Abdullah  (University of New South Wales, National ICT Australia NICTA Australia)

•Offers insight into strategic IT systems and strategic IT systems development outsourcing;
•Provides an understanding of existing research on risks related to the area and  strategic IT system development project failures;
•Highlights to organizations considering and embarking such outsourcing, the risks that are most likely to cause failure so that appropriate measures can be taken; and
•Provides a framework of risks for outsourced strategic projects and validates this framework on a case study.

BACKGROUND:  The academic literature has identified a series of risks connected with IT development and IT outsourcing.  Much of the research has been centered on the vendor side, identifying risks that can threaten internal software development.  Some of the research, however, does not take a particular perspective on outsourced risk, i.e. does not specify if the client or vendor is the owner of the risk. However, research on risks in the context of strategic IT system development outsourcing is almost non-existent.  Failure to understand and manage risks can result in serious losses and outright failure of the outsourced strategic IT system development projects.  This will subsequently seriously affect the achievement of client's strategic business objectives. 
AIM:  This study investigates from the client perspective, risks that are most likely to cause failure in a strategic IT system development outsourcing project.  Discussion on the nature of a strategic project is presented and definitions of what a strategic IT project actually is, are explored.  This study examines strategic IT systems from a business strategy perspective and presents the characteristics of a strategic IT system by synthesizing material from the management literature as well as the IT literature.  Such definitions are currently lacking. The research then develops a framework that categorizes strategic IT development risk.
METHOD:  An initial review of the literature covers the opposing aspects, as risks from three different streams of literature, namely IT development, strategic IT and IT outsourcing, and reported examples of strategic IT system development outsourcing project failure in the literature are examined.  Examples of strategic IT systems are determined based on the proposed characteristics of strategic IT systems.  Risks are then evaluated against the project failures and the failure factors existed in the literature to determine which risks that client perceives to be most likely to contribute to outsourced project failure.  Using the variety of risks in the literature and the different IT outsourcing stages as a basis, a conceptual risk framework is developed and tested in a preliminary case study which investigates the development of a strategic system developed concurrently in Australia and Malaysia.
RESULTS:  The conceptual framework proposed identifies and differentiates between risks that are important to client organization and most likely to cause failure, and those that are not important to the client in an outsourced strategic IT system development. The framework is tested with an outsourced project and feedback from the case study has enabled us to modify our framework.
CONCLUSIONS: We developed a comprehensive definition of a strategic IT system, and then identified risks that are of importance to client organizations that develop such systems.  We defined and tested a conceptual risk framework that will assist clients to focus their attention on potential problems that can lead to failure of the outsourced strategic IT system development.  While this study presents many findings based on a review of the literature it does test the framework developed though a preliminary case study.  Additional case studies are required to further validate both the risks and the framework.

12.15 Scope Management - A case that sailed the seas of change
Frank Vogelezang (Sogeti Nederland B.V. Olanda)

* Learn about the northernSCOPE approach for Scope Management
* See a practical case where northernSCOPE was applied and see how this approach has helped the project manager to manage changes to a project

We all hear stories about failed IT projects and how hard it is to manage those projects properly. But we hardly hear why those projects failed. Many IT projects fail because they had to live up to unrealistic expectations. A lot of research has been done about the dynamics of software projects and a great deal about the dynamic behaviour of software projects is already known. All too often we do not use this knowledge and all too often another software projects goes wrong.
With the northernSCOPE approach we are able to manage the scope of a software project by making sensible use of experience from the past and what research has taught us about the dynamic behaviour of software projects.
In this paper we describe a real-life project. All the way from idea to implementation. We demonstrate that northernSCOPE is not a complicated academic endeavour, but a straightforward, down-to-earth approach that helps the project manager manage the scope of a software project. For each of the twelve steps of the northernSCOPE approach the practical steps that have been taken to navigate the project out of possible danger zones is described. For each step the underlying theory is translated to practical use to the project manager and his customer. Each time the project ran into new requests that were out of scope or added new requirements or constraints to the project, the northernSCOPE approach was used to demonstrate the effect of giving in to these new requirements and of not giving in. It is not a story of a great success with a textbook project, but that of a project that could prove all along the way to deliver value for money for the customer.

12.45 Outsourcing Testing Activities - How to prove cost reductions?
Harold van Heeringen (Sogeti Nederland B.V. Olanda)

Delegates will learn about the difficulties in measurement of cost savings related to outsourcing testing activities. There are a number of variables that heavily influence the outcomes of test projects and most of these influences are coming from outside of the test project. The model proposed will be helpful in order to be able to assess cost efficiencies properly when dealing with the outsourcing of testing activities.

Outsourcing (parts) of the ICT organization is these days for many organizations a hot item. In some cases the entire ICT department is outsourced to a domestic or to a foreign ICT supplier, but most organizations outsource only part of their ICT operations. In many cases this part is the development part or the enhancement part, but nowadays also a trend can be watched with regard to the outsourcing of all system testing activities. One of the major drivers in outsourcing is always the possibility to realize cost reductions. When outsourcing development activities, these cost reductions are quite easily measurable. It is possible to calculate the costs per function point for the development activities in-house and also the price per function point of the outsourcing partner(s) and the difference indicates the amount of cost reductions that can be realized.

When outsourcing test activities, this model will not suffice. Although it is possible to measure the price per function point for test projects, there is one more important variable. This is the test performance delivered: how many defects have been found compared to the defects that should have been found. While the price per function point for test projects may be lower when outsourcing test activities, it still may be possible that there will be no cost efficiencies. This is due to a larger number of defects than necessary residing in the software and the work that is needed to correct these defects as they occur in acceptance test and in production phase. The number of defects that are expected to be found in the testing phase is highly dependent on the way that the development of the project has been carried out. If this was a project with a relatively short duration and with a relatively large team size, the number of defects expected is relatively high. It is therefore not possible to evaluate the testing performance objectively without taking into account the way the project has been developed.

In this paper a model will be presented that will help organizations to assess the real cost efficiencies that they can gain when outsourcing testing activities. The model benchmarks the real project data against the expected project data based on the results from one of the most widely used project estimation tools: QSM SLIM Estimate. Both the amount of hours spended while testing and the number of defects found will be taken into account.

13.15 Networking Luncheon 
 
14.30 Evaluating Legacy Assets in the Context of Migration to SOA
Vinay Kumar Reddy Alpana Dubey, Sala Lakshmanan, Srihari Sukumaran and Rajendra Sisodia (Philips Research Asia -Bangalore)

Gives insights about the metrics and guidelines (to be used for evaluating legacy software assets) for ensuring proper migration to SOA.

The key thing in the introduction of service oriented architecture (SOA) for an organization is to evaluate the suitability of existing assets for service orientation. We identify the core principles of SOA, viz., cohesion, reusability, discoverability, loose coupling, abstraction, formal contract, composability and statelessness as the guiding principles in evaluating the suitability of existing assets for service orientation. In this paper, we survey the existing metrics and guidelines that could be helpful in evaluating these principles. We expect this survey to help organizations moving to SOA in two ways. Firstly, to understand the effort needed for the migration of existing assets to SOA and secondly to assist for the proper selection of services to be developed from existing assets.

15.00 Practical Guidelines for Introducing Software Cockpits in Industry
Marcus Ciolkowski, Jens Heidrich, Jürgen Münch, (Fraunhofer Institute for Experimental Software Engineering IESE, Germania)

The readers will learn about challenges in the area of setting up and introducing software cockpits in industry and will be supported with practical guidelines on how to overcome those problems. The guidelines are based on experience that was systematically gathered in several industrial case studies and research projects. The guidelines address, for instance, the following issues:
• Selecting the right key performance indicators and determining context-specific thresholds;
• Integrating software cockpits into the organizational processes;
• Analyzing data for project control in the context of project goals and higher-level business goals;
• Social and organizational issues related to the deployment of software cockpits (such as overcoming acceptance problems);
• Specific issues that need to be considered in controlling global projects.
In addition, the article will provide a research roadmap, including a list of the most important research questions to be tackled from a practitioner's viewpoint.

Industrial case studies illustrate that benefits from following the guidelines include: being able to identify and reduce risks related to introducing software cockpits, being more efficient in setting up and adapting project controlling mechanisms, allowing for more transparent decision making regarding project control, being able to achieve higher acceptance of the underlying measurement program within a company, reducing the overhead of data collection, increasing data quality (especially for manually collected data), and, finally, achieving projects that are easier to plan and control.

In order to successfully conduct development projects, one crucial success factor is the existence of well-specified and coordinated development processes. Therefore, it is necessary to have efficient management and controlling mechanisms in place. Many companies are currently establishing so-called software cockpits for systematically supporting quality assurance and management. A software cockpit is comparable to an aircraft cockpit, which centrally integrates all relevant information for monitoring, controlling, and management purposes. In practice, a variety of simple dashboard approaches for project control exists. More comprehensive approaches that support advanced management techniques and allow for organization-wide data collection, customized interpretation, and visualization are rarely implemented.
Despite significant progress in recent years, setting up and establishing software cockpits such that organizations have sustainable benefits is still a challenging task: First, a measurement culture has to be established within an organization and has to become part of daily life in order to assure high quality and up-to-date data for project control. This requires goal-oriented derivation of key performance indicators, integration of data collection procedures and control mechanisms into the development processes, and analysis and interpretation of results in the context of project goals and organizational goals. In addition, strategies are needed that define how to effectively introduce controlling mechanisms (e.g., through appropriate training of all team members) and gain acceptance by team members. Second, successful deployment of software cockpits needs to take many factors into account, some of which are changing frequently. These factors include the number of organizations and people involved, the number of distributed development locations, the heterogeneity of development platforms and techniques, the complexity of functional and non-functional requirements, as well the experience of the people responsible for controlling and steering a project. Especially in the area of global development projects (across different organizations within different countries), heterogeneous processes resulting in incompatible workflows and data make it even harder to control and steer the project. Third, to create a sustainable measurement program, gaining management support for software cockpits is crucial. This requires empirical evidence on the benefits, risks, and costs of software control mechanisms compared to projects without quantitative project control. However, there are currently only very few empirical studies that address these issues.
Empirically validated guidelines can support successful deployment of software cockpits by addressing the challenges mentioned above. This article presents guidelines derived from experiences with setting up and establishing software cockpits in industry. They cover issues related to defining appropriate measurement systems, implementing and tailoring software cockpits, deploying software cockpits, controlling distributed and global projects, and analyzing success factors and benefits of software cockpits. The guidelines were initially derived from industrial case studies (several implementations of software cockpits in industrial practice, including software development for logistics systems, information systems, and web applications) and from two large research projects (the special research project SFB 501 and the applied research project Soft-Pit). We enriched the guidelines by illustrating typical problems and solutions encountered during our industrial case studies. In addition, we used expert opinions collected during two workshops on software cockpits to interpret and enrich findings from the case studies. The article concludes with a research roadmap for controlling software and system development projects and an outlook on future research.

15.30 Software Quality Management- A Balanced Scorecard Approach
Ritesh Jain (Software Expert Group Limited Gran Bretagna)

Quality management in testing has played an important role in a project. It is divided into three areas of quality, which is planning, quality assurance and quality control. The quality planning elements are implemented during the test planning after a project plan has a sign-off. Quality assurance is focused on the audit activity that is performed throughout the entire testing methodology. After these two elements have been completed, a project should perform the metrics measurement in order to fulfill the quality control elements in each of the processes.
To satisfy the above three areas of quality management, we implement the Balance Scorecard approach in our test methodology. The implementation of Balance Scorecard approach for quality management in testing aims for long term success by not only focusing on customer satisfaction but also satisfy the elements of cost effectiveness, resource utilization and quality process implementation. Generally, software testing is accomplished in a project to finding defects or meeting of both contract and customer requirements. The software testing objectives should focus into different perspectives instead of containing to finding defects. By implementing the Balance Scorecard approach into the testing process, the Test Manager is able to establish testing objectives based on four different perspectives which is customer, internal business process, financial and learning and growth. This approach is developed to help the Test Manager to manage and drive the testing team to achieve not only the project objectives but also to fulfill the quality objectives in each of the testing process. Furthermore, a good quality management in testing is implemented in a project to produce high quality product by reducing business cost and improving productivity.

16.00 Coffee break
 
16.30 Evaluation of a Standard- and Metric-Based Software Quality Model - A Case Study
Rüdiger Lince (Växjö University, Svezia)

Gain insight in ongoing research in the field of software quality assessment.

The goal of our research is to replace ISO 9126 metrics manually assessing internal software qualities with metrics allowing for automatic measurement. We validate the significance of the new, automated Software Quality Model with practical experiments in selected software development projects. In the end we want to support or invalidate the hypothesis that static metric analyses allow for an assessment of (some) internal qualities of software systems.

Currently, there is no validated approach of automatically measuring software metrics allowing to draw conclusions on the quality of a system observed. Thus it is either possible to derive metric values in an automated way, but their interpretation is rather vague, or to evaluate a software quality model based on manually derived values, which is rather expensive and time consuming.
To bridge the gap between software metrics and well interpretable software quality models, we associated the software quality model defined in ISO 9126 with a number of well known metrics from literature. In detail, we evaluate if software quality can be assessed in an automated and reliable way through static analysis and software quality metrics, and to draw conclusions from this data organized in a software quality model about the quality of a software system as it can be understood by a software engineer.
With the case study outlined in the paper, we validate or invalidate a correlation between automatically assessable software metrics, from now on called model metrics, and classic, manually derived assessments of quality, called validation metrics. We define the model and validation metrics used and the measurement procedures, as well as the analysis of the correlation and our conclusion.
This pre-study is a first step toward the more general goal described above. We deliberately restrict ourselves to "Maintainability" as the only quality assessed and conduct the experiment in only one industrial project. Model metrics are assessed automatically and include about 25 metrics measured on method, class and package level. They have been selected from (object-oriented) metric suites well described in literature like, e.g., by Chidamber and Kemerer, Li and Henry. Validation metrics involve quantitative and qualitative data manually collected from bug trackers, questionnaires, expert opinion, and interviews, etc. The qualitative data is used for interpretation of the quantitative data and for triangulation between the model and validation metrics.
We consider it important to conduct this limited study prior to conducting the full scale experiment to confirm that our tools and methods work in a single project. With the experience made we can refine our tools and methods to guarantee a controlled execution of the experiment involving the projects of all our partner companies and reliable and reproducible conclusions. Although not statistically save, we even present preliminary results of the experiment.

17.00 110 Maturity Metrics
Celestina Bianco (NTE SA, Spagna)

• Learn about SPiCE and maturity assessment frameworks
• See a practical example of metric plan
• See examples of application to projects and services

SPiCE (ISO 15504) is a standard for a capability assessment framework. Standard means it has been agreed by all members of the committee and therefore comes from experience and consensus. The capability of a company to satisfy customer needs is assessed as a profile, which is given by the bi-dimensional combination of a set of processes that are relevant to the company and level of maturity of each of them. That level is assessed analyzing the compliance to attributes that grow at each level. From the "0 level" of chaos, the next step is to perform the minimum activities to assure deliverables of each product or service of the company. Next level is doing it in similar manner for different programs, next is to formalize this similar approach.
Only at 4th level does a metric plan come on board. At that stage a metric plan is applied to the entire company and to each project in a default or customized manner. This allows high synergy and reuse of all of experience and knowledge, the possibility to optimize resources and flexibility to implement changes requested by the customer in rapid, controlled and safe manner.
In the previous level a project Life Cycle has been defined and all processes proper of the project design and supporting it; indicators have been identified for each of them. The database has been populated with values gathered during experience, and are now used in a systematic way. Measures such as design and metrics review, audit performance metrics, defect distribution, efforts and results, costs, all enter in this plan. And no more than these are often requested.
The paper will analyse an example of metric plan considering the concrete process profile of a Company.
Simple metrics, when used alone, depend on personal initiative and upon events, represent an indispensable base for the project, contingency and resource control. The same metrics become maturity indicators if inserted in a plan that includes the method and strategy to systematically collect, and then analyze data and trends, and above all REACT!

17.30 Closing