Measuring up to E-Government, A view from the Shelves

M.J.Phythian, Dr. N.B.Fairweather and Dr. R.G Howley


Following an initial investigating in 2007 it appeared that there had been little academic work in the area of measurement and electronic government. There further appeared to have been no analysis of whether the large sums of money spent had been truly worth it or whether the public were getting a better service. To confirm or deny this, a lengthy literature review was commenced in 2008. Much of the available material comes from a political science, ethnographic or political science perspectives, with little from a systems or computing background, particularly involving authors involved in the delivery of such systems. This is confirmed by Bloomfield & Hayes (2005, p.4) in their analysis of e-government.

In the United Kingdom, and in particular in England there was from the outset little in the way of strategy or guidance for the development of e-government. The work of delivering or monitoring change was largely done by trial and error following the initial Public Service Agreement in 2001, as local government was encouraged by financial carrots to report progress back to central government. These reports were entitled Implementing Electronic Government Statements (IEG’s) and were initially required annually and numbered one through three and from these central government developed a model and standardised upon the format of them. A further four reports were called for in subsequent years. Subject to progress being reported in these, grant was received.

The lessons learned from the early IEG’s were further developed into a new set of targets described as Priority Service Outcomes, again sometimes contradictory and inconsistent, but requiring a large investment in technology. These were produced without the establishment of baselines of existing efficiency to measure improvement against. Instead the country is presented with an array of technological solutions that promise little opportunity of being able to join up to deliver any fundamental efficiencies. Further, as Irani et al (2005, p.77) conclude, their research ‘has highlighted that many information system e-government decisions are political and that evaluation is always subjective.’

In a long-awaited government review of public service transformation, Lyons (2007, p.80 on) emphasises what he describes as ‘crowding out of local choice’ by central control, which demonstrates the dichotomy involved in service delivery when prescription from government becomes involved. Whilst Lyons (2007, p.202) is keen to see improved cross-boundary working and identifies the use of technology to do this, at the same time he highlights the tendency for re-inventing the wheel and again, that centrally-imposed restraints limit the ability to implement joined-up services (Lyons (2007) s.4.139).

In reality, the exercise of e-government should have been about improving the customer experience whilst making the most efficient use of all possible channels of communication. Local authorities and government, unlike banks, do not have the option of driving customers down the most cost effective channel for the supplier and instead have to deliver services in a manner appropriate to their local community, which will entail having a range of channels dependent upon need. This can be dependent upon the local community being urban or rural, wealthy or poor and various combinations, which is confirmed in the research of King & Cotterill (2007, p.351).

A key challenge will be to find what metrics have been used to measure the improvements and whether and how they can be adopted in the later stages of implementation. Di Maio (2007) made an excellent effort at selecting and bringing together Public-Value-of-IT frameworks from across the world but what he presents are complex tools and appear too large for local government bodies. What is needed is a framework suitable for the smaller organisation, which was confirmed by the initial investigations done by Carbo & Williams (2004, p.97), unfortunately that work was never extended due to lack of funding.

The ideal unit of measurement might be, in customer terms, satisfaction across all channels. This aligns well with Pumphrey (2006) in terms of a strategy but also with experiences in the banking sector where this was investigated by Patricio et al (2003) and Joseph et al (1999). A danger is highlighted in Delivery and Transformation Group (2006, p.15) of recreating fresh service cliques or silos in the form of the new channels, recreating the original issue and hence there is a need to equate the user view across all of the channels. If there are any parallels with retail customers, research by Tesco, (Young, 2007), has indicated that shoppers want the range of channels operated by the company to be linked together as much as possible. This is reinforced by Straub et al (2002, p.119) who argue that:

“firms that want their traditional and new network-based channels to work together require metrics for interoperability.”

It is also supported by Vesanen and Raulas (2006, p.17) who propose that:

“marketing today is about managing the elements described by the process model, and the entire process itself.”

This examination of the wider literature indicates that:

  • improvement for the citizen will primarily be delivered by organisational change with the end-to-end processes involved
  • a gap exists in the literature around metrics but there is evidence that customer satisfaction is a potential base measure for identifying movements in public value/social capital, which may help to success or otherwise with the changes
  • there is a strong historical basis behind the service structures which have been additionally confused by attempts to make them more efficient, this indicates that change should be carried out carefully

In order to fill the gap in the literature around metrics, the researchers’ instruments include a weblog as a discussion point to examine any suitable metrics that can be fairly measured across a range of service delivery channels, along with an online questionnaire. The blog has been established as: . Piloting of the instruments commenced in March 2008.

This paper will:

  • embellish upon findings from the literature and report additional avenues for investigation
  • reveal any initial feedback from the research instruments


Bloomfield, B., Hayes, N., (2005) “Modernisation and the joining-up of local government services in the UK: Boundaries, knowledge & technology.” LUMS Working Papers

Carbo, T., Williams, J.G., (2004). Models and Metrics for Evaluating Local Electronic Government Systems and Services. 4th European Conference on e-Government. Dublin, Electronic Journal of E-Government. 2.

Di Maio, A. (2007) “Worldwide Examples of Public-Value-of-IT Frameworks.” Gartner Industry Research

Irani, Z., Love, P.E.D., Elliman, T., Jones, S., Themistocleus, M., (2005). “Evaluating e-government: learning from the experiences of two UK local authorities.” Information Systems Journal 15: pp. 61 – 82.

King, S., Cotterill, S., (2007). “Transformational Government? The role of information technology in delivering citizen-centric local public services.” Local Government Studies 33(3): 333 – 354.

Lyons, M. (2007). Lyons Inquiry into Local Government, HMSO.

Patricio, L., Fisk, R.P., e Cunha, J.F., (2003). “Improving satisfaction with bank service offerings: measuring the contribution of each delivery channel.” Managing Service Quality 13(6): pp. 471 – 482.

Pumphrey, S. (2006) “Mastering Channel Strategy.”

Straub, D. W., Hoffman, D.L., Weber, B.W., Steinfield, C., (2002). “Toward New Metrics for Net-Enhanced Organiizations.” Information Systems Research 13(3): pp. 227 – 238.

Vesanen, J., Raulas, M., (2006). “Building Bridges for Personalization: A Process Model for Marketing.” Journal of Interactive Marketing 20(1): pp. 5 – 20.

Young, T. (2007). Tesco wants to link the web and high street. Computing. London. Weekly.