Effects-driven IT development attempts to provide a sustained focus on the effects to be achieved by users through their adoption and use of a system.
Read the latest publication about Effects-driven IT development:
Hertzum, M., and Simonsen, J. (2011). Effects-Driven IT Development: Status 2004-2011. In M. Hertzum and C. Jørgensen (eds.), Balancing Sourcing and innovation in Information Systems Development (Chapter 8). Tapir Academic Press, Trondheim, NO.
Simply put, the overall idea is to capture the purpose of a system in terms of effects that are both measurable and meaningful to the customer, and to systematically evaluate whether these effects are attained during real use of the system. A sustained focus on effects accentuates that the functionality of a system is merely a means to an end, but it also entails that effects must not only be specified but also evaluated in the course of the development process. That is, effects-driven IT development blurs the distinction between design and organizational implementation – between design and use. This focus is summarized in our definition of effects: Effects = system quality × system adoption
Concrete examples of effects may include:
- The physician can complete the medical ward round without an escorting nurse, thereby making the clinical work more cost effective.
- A reduction in clinicians’ mental workload at the daily team conference, thereby reducing the risk of errors in their assessments of patient status.
Effects will often form a hierarchy where higher-level effects specify why effects at lower levels are desirable and lower-level effects specify how effects at higher levels can be attained. For example, national healthcare policies may state political effects, which influence individual hospitals’ choice of strategic effects, which in turn are reflected in effects directly concerning different aspects of the clinical work. While these examples relate to healthcare (the domain of our research program), the idea of effects-driven IT development is generally applicable to IT projects. The primary focus of effects-driven IT development will typically be on direct effects on the users’ work. The main reason for this is that these effects can be specified most precisely, whereas effects at political and strategic levels are more indirect and thereby subject to additional sources of ambiguity. A supplementary reason is that effects-driven IT development recognizes that system success is critically dependent on the users’ participation, support of and attitude toward the system and thereby on whether they agree with the sought-for effects and can relate them to their work.
Working systematically with effects involves two critical activities:
- Specification of desired effects: It is our contention that effects are more stable than functional requirements because effects are higher level and far fewer. If a focus on effects is to provide a framework within which different designs can be explored, it must, however, be possible to specify effects. This involves identifying, formulating, and prioritizing effects as well as devising methods for their measurement and evaluation. We suggest that this is done in collaboration with users following a Participatory Design approach such as the MUST method.
- Formative evaluation of effects: Effects-driven IT development presupposes that it is feasible to use the presence or absence of effects as an active means of managing IT projects. For this to work it must be possible to demonstrate effects within the timeframe of IT projects. This involves setting up and conducting evaluations to measure effects of system usage during real work – and this must be done while the system is being developed, not after it has been completed. We contend that this can be accomplished by, for example, configuring systems based on standardized and flexible development platforms (e.g., HL7 (www.hl7.org) and XML (www.w3.org/XML)) and using Wizard-of-Oz techniques.
Our empirical study investigates whether and how these two critical activities can be performed. At the same time, the two critical activities capture how a sustained focus on effects adds to related approaches in Participatory Design and user-centred design (UCD). Participatory Design and UCD techniques such as diagnostic maps, future workshops, mock-ups, and exploratory prototyping focus mostly on the early stages of technical implementation and do not involve evaluation of whether identified user needs are subsequently satisfied by the developed system. Usability evaluation, a widespread UCD technique, is commonly performed on set tasks and test data and with a focus on usability problems rather than usage effects. A UCD technique particularly related to effects-driven IT development is usability specifications. A usability specification gives the worst, planned, best, and present levels of user performance for a specified set of tasks. In giving values defining the different levels of performance, usability specifications specify a set of effects and provide for a process alternating between design and evaluation until the effects have been attained. For the rather narrowly scoped tasks mostly associated with usability specifications it has not been considered a problem to obtain precise performance measurements, but for usage effects that involve establishment of new organizational procedures, collaborative practices, and individual competences reliable measures are difficult to obtain.
Specification of effects has also been suggested as an analytic device; that is, without measuring whether the specified effects are actually achieved. Conversely, feedback from users based on their actual use of (parts of) a system is available in incremental development and delivery. However, incremental development and delivery does not involve specification and measurement of usage effects as a means of systematically evaluating whether a system provides desired effects. An exception is results-driven incrementalism, which has a lot in common with effects-driven IT development. Finally, our work on effects-driven IT development has been inspired by performance-based procurement and benefits management.