AUTHOR
David H. Gleason
ABSTRACT
This paper expands the author’s earlier work on subsumption ethics by elaborating on the axioms presented at the Tangled Web conference at Dartmouth in August of 1998, and the paper “Subsumption Ethics” published in Computers and Society, March 1999. The paper first describes the principle of subsumption ethics, then discusses its four axioms in detail. It concludes with a discussion of the application of the axioms to practical IT problems within the context of various ethical frameworks.
Subsumption ethics is the process by which decisions become incorporated into the operation of information technology (IT) systems, and subsequently forgotten. IT systems, by nature, repeat operations over and over. If those operations have unethical impacts, the system will continue to execute them anyway. Unlike a human operator, there is no point in the cycle where the machine pauses to ask, “Should I do this?” Subsumption in general is the process of building larger components from smaller ones. In this sense, a cell subsumes DNA function, American common law subsumes judicial decisions, and a hairdryer subsumes an electric motor. Subsumption in computers is different because there is so much more of it going on than in simple machines.
In computer systems, small components are developed and tested, and once they are working reliably they are subsumed into larger systems. This is the enabling technique of object oriented programming. The larger systems, in turn, are subsumed into still larger systems. Once components, subsystems and applications are operating, the subsumed process becomes invisible and unavailable to the user, what James Moor calls the “invisibility factor.”
Systems seem like they should be extremely malleable. People tend to think that changes to software should be easy because programming is just a set of instructions, and not like a building made up of hard materials. However, the principle of subsumption makes it clear that changing base components is like moving building foundations, and can require changes to entire systems. The year 2000 problem, for example, is a result of subsumed date processing. There are thousands of layers of subsumption in a typical computer system. There are four axioms of subsumption ethics:
- Information systems subsume design, policy and implementation decisions in programming code and content. Code segments and content become “subsumed objects.” While it is demonstrable that systems are built from subsumed components, it is less easy to show exactly how decisions are subsumed. This axiom posits that the decisions themselves, including many subtle factors, are incorporated into systems operation.
- Subsumed objects have determinate moral value. Anecdotally, we can see the moral value of subsumed objects. A windowing system that can only display certain colors, thereby excluding users with certain visual disabilities, has a negative moral value for those users.
- Subsumed objects have a high “invisibility factor.” Subsumed objects are invisible to most users. It is not possible, for example, to know all the calculations that mortgage eligibility software might use without seeing the source code. Such software could systematically discriminate without a user’s knowledge.
- Subsumptive complexity increases over time. As systems are developed, components become subsumed more and more deeply. For example, purchasers of automated toll booth data subsume it into much larger databases, which might incorporate home ownership, family and estimated income information. These axioms can be applied to practical problems in IT by using them in conjunction with established ethical frameworks. A series of imperatives for IT development can be constructed from a matrix that combines the axioms with a code of ethics. The paper describes this process in detail.
Since the axioms are not dependant on a specific ethical framework, they can be applied in many situations, across cultural as well as industrial boundaries. This process offers developers a powerful tool to use to enhance project management and facilitate software impact analysis.