Multicriterial evaluation of alternative decisions in software development process

Maria Ganzha


During design of new software social and ethical issues are too seldom examined. Complexity of procedures examining ethical issues can be pointed to as one of the main reasons of this situation. Moreover, an ethical audit of the project demands additional time and expenditure of resources. Developers and managers often claim that perfect software will be excellent from the ethical point of view. However, numerous examples of failures of software projects support the necessity of including ethical analysis into the development process from the very beginning.

There exist methods for ethical analysis of a software project. Let us recall some of them: (1) “procedural” algorithm for ethical analysis proposed by Maner; (2) method of multidimensional analysis of quality and performance during software development LIME (Lifecycle Measurement) by Buglione and Abran; (3) five-step process of ethical analysis by Rahanu et. al.; (4) Bynum and Rogerson method based on natural ability of most people to make ethical judgments and others. There exist also special programs, which support the impact analysis of the software project throughout its lifecycle (for example, the SoDIS Project Auditor). But the main objective of these methods is only to define ethical “bottlenecks” of the analyzed project. This means that the method serves to define these parts of the project or of already completed program modules, which violate ethical principles.

While the latter approach is very valuable, we have to notice, that it is during all stages of a the lifecycle of the project (LCP) that managers (and possibly developers) as well as managers representing customer(s) have to make choices between several alternatives. These consideration points to a definite need for creation of a procedure(s) for multicriterial ethical decision making. In this context, we recognize that there exists a well known AHP procedure for making multicriterial decision in business (Saaty (1990)). However, this procedure does not take into account all possible stakeholders impacted by the project. Instead, only the developer team and customers are considered.

Let us further stress that in research on evaluation of software quality it is the completed “product” that is discussed. All methods, tools and procedures for evaluation of software quality deal with already implemented software. If we accept this approach, in a way we implicitly agree with failure of completed software projects due to disregarding of stakeholders’ interests, designing program on the base of past experience (which may be wrong in the new situation), having no procedure to support decision making in crucial junctures of project development, etc. To avoid these pitfalls procedure(s) should be developed, which permit to evaluate broadly understood “costs” of every decision during project development.

In this paper a new procedure for the estimation of “goodness” of alternative paths in project development is proposed. This procedure is capable of accommodating into consideration all stakeholders of the project. Furthermore, it can be used to evaluate “costs” every time when a major decision is to be made. The main idea behind of this procedure is to combine the existing methods for making complex decisions and stakeholders ethical analysis. This new combined method

  • helps to capture subjective and objective evaluation measures,
  • takes into consideration Customers’ and Suppliers’ interests as well other possible stakeholders,
  • provides a useful mechanism for checking the consistency of the evaluation measures and alternatives suggested by the team,
  • is designed in such a way to be applied in every step of the project,
  • reduces possible bias in the decision making process.

A very important part of the proposed method is the selection of criteria for the project estimation. It has to be stressed that in the proposed method software criteria and ethical criteria are not separated. Numerous examples prove that violation of any criteria impacts failure/success of the project. As a base set of criteria used in our approach we use the well-known set of criteria proposed by McCall and his colleagues. They have introduced a categorization of factors that affect software quality by focusing on three important aspects of a software product: (1) its operational characteristics, (2) its ability to undergo change, and (3) its adaptability to new environments. However, from the perspective suggested here these factors are insufficient for complete estimation of the project. For instance, resources, financial and ethical factors are not taken under consideration and therefore the McCall approach has to be augmented by additional criteria. The process of selecting the augmented set of the criteria is also a part of the proposed method. It is very important that experts – group of people which perform analysis of a project – collaborate in the process of selecting criteria to be used in the process (as the final list may depend on the nature of the process itself as well as the existing set of stakeholders). As an example let us list a few factors that can be taken into considerations:

  • financial management – the project does not cause unnecessary expenditure;
  • privacy protection – execution of the project will not break somebody’s privacy and will not enlarge number of people, who has access to data without agreement of this data owners;
  • data protection – realization of the project will not damage database security;
  • intellectual property – realization of the project will not damage omebody intellectual property;
  • human resource management – project does not require the developers to work beyond their ability;
  • reasonableness of schedule – all work is to be done in a steady fashion, amount of work does not lower quality of developers’ (program team) life during the lifetime of the project development;
  • feasibility of objectives or goals – all formulated tasks and subtask are feasible;
  • agreeability of the schedule – the schedule of project is suitable to time allotted by the customer.

After defining the set of criteria to be used in evaluating the process we can calculate the criteria weights using the method of comparison estimates. Naturally, satisfaction of different criteria has different significance for the project. This is reflected in comparison estimation. For example, let us assume that the customer requires completing the project in a short time. Thus, the time criterion has (for the customer) value higher than values assigned to the estimates of other criteria.

The next important part of the proposed procedure is choice of stakeholders. First, let us note, that stakeholders are understood in a very broad sense. This means that not only Customer and Developer, but all humans and non-humans affected by the project are taken under consideration. Furthermore, every alternative to be considered may have its own set of stakeholders. To deal with this issue we decided to consider a united set of stakeholders. However, since not all stakeholders are equally important to the project and the project does not affect all stakeholders in the say way, stakeholders’ weights are also defined.

The last step of the procedure consists of evaluation of every alternative “cost”. The cost of alternative is calculated as the weighted sum of its estimates with respect to every criterion and to every stakeholder. This is achieved by applying linear algebraic techniques abased on the AHP procedure.

The proposed procedure is going to be illustrated by an example.