What is success? The challenge of evaluating impact

There is no shortage of views on the fitness for purpose of the civil service. The paradox of UK civil service reform is that it is subject to two quite contradictory narratives. 

​

The positive, sometimes evangelical, narratives come from international institutions, practitioners and some academics. The World Bank, International Monetary Fund and strategy consultancies continue to champion UK NPM reforms (Pollitt, 2013a). The academic industry that sprang up around the NPM Paradigm did much to promote the UK’s reform experiments. There is a lucrative global industry promulgating often mangled, ahistorical and acontextual interpretations of major UK reforms.

​

On the other hand, assorted select committees, self-appointed commissions and think tank reports seem united in a negative narrative which portrays a civil service that despite endless reform is either unfit for purpose or a shadow of former glories. This negativity is fuelled by those academics with strong governance and ‘Whitehall Model’ interests rooted in the classic view of bureaucracy and public service.

​

Has so little changed for the better after the frenzy of reform over the last 40 years? 

​

Efforts by researchers to focus on bundles of reforms and test whether they met their intended outcomes provide a mixed picture on the impact of reforms. 

​

So discerning the impact of managerial reforms is a tough task that faces several almost insurmountable barriers.

These barriers partly explain the lack of evaluation and the often-unsatisfactory findings of those few evaluations that are undertaken. In the face of this evaluation void the confidence of those taking a positive view of the benefits of managerial reform efforts has become almost ideological (Hood, 2009). Equally the wistful certainty of those who articulate a narrative of decline can appear as an ideological distaste for the notion of business and management in public administration (Chapman & O’Toole, 2010), (O’Toole, 2004). There is a persistent negative tone to much of the research on the development and impact of NPM which colours the story told of its impact (Funck & Karlsson, 2020).

 

None of this helps with the challenge of better understanding how and why reforms have positively changed the civil service in a way that is of practical use to those who would shape and run future reform efforts.

​

The starting point for my research is that both the public administration view and the prevailing narratives of decline are partial, often misleading and generally fail to capture the cumulative and transformational impact of 65 years of reforms. 

It is possible to believe the civil service has substantially improved its capability through decades of reforms whilst also holding the view that it is still not fit enough for today’s purpose and tomorrow’s challenges.

​

The difficulties of evaluation point to the need for a different approach to trying to understand the impact of reforms and how they improve the effectiveness of government. Consequently, I am looking elsewhere for frameworks and theories that can help me understand the impact of civil service reforms.

​

I am also exploring an alternative approach to the futility of seeking evidenced impact and causality. One such route is encapsulated by Joullie and Gould (2023) in the conclusions of their review of the limitations of management research:

 

Rather than seek causality management researchers would better aim to ‘understand and explain deliberate actions, situational choices, ambiguities and constraints’ accepting that the actions of agents are not deterministically constrained. Such understanding needs to be rooted in the context of values, opportunities, and an imperfect pool of ideas and experience that are drawn on as choices are made and paths pursued. 

​

An increasing number of researchers have sought to address the theoretical limitations of the public administration tradition by drawing on theories from other fields. They have used theories from strategy process, strategy as practice, institutional work, sense-making and dynamic capability - often in combination - to understand change and its impact in public sector institutions. 

​

My Searching for success article below outlines the most promising areas of research for illuminating the impact of reforms. A longer article exploring the potential of this approach will be published in summer 2024.