In response to idle inquiries about the subject of this book, the author learned long before its completion that almost any answer including the word ‘audit’ or ‘auditors’ –or even, sad to say, the words ‘council’ or ‘local government’ –could generally be relied on to prompt a glazed eye and rapid change of topic. So why a whole book devoted to the Audit Commission? The case for an ‘institutional biography’ of such a little-understood body is easily summarized. It has pulled off some remarkable, yet mostly unremarked, feats over the past twenty-five years. Tracing the story of the Commission offers a revealing glimpse of Whitehall and town hall at work through a period of extraordinary changes. It is also the tale of a hybrid organization, set up to combine an innovative and highly creative culture on the one hand with a by-the-book quasi-regulatory culture on the other. That was never to be the recipe for a dull bureaucracy.
The Audit Commission has been a pioneer, in ways that seem almost to have been taken for granted in Britain even while being widely acclaimed overseas. It has also found itself at the centre of an often bitter controversy over the impact of an indisputable drift towards centralized government. Many commentators have blamed this greater centralization in Britain for a corrosion both of local democracy and of the integrity of the professional classes working in the public sector. Some among them have eyed the Commission as one of the arch-culprits. The reality that emerges in this book is much more complicated. Arguably the Commission has been guilty on occasions of acting naively. But to see it simply as a vehicle of centralization would be to stand the story on its head. The Commission was driven from the outset by a fierce belief in the need to lift the public sector’s performance, not least to buttress it against the encroachments of central government. How this stance has shifted to and fro over the years, in response to increasingly directive government policies, is a key theme of the book.
The Commission was established a quarter of a century ago, during Margaret Thatcher’s first administration, thanks entirely to the drive of one of the very few effective government ministers in recent times to have enjoyed a successful career in business, Michael Heseltine. He had learned from his business life that structures and processes can be as important as strategy and policy. It gave him a natural interest in the machinery of government — which always rather bored Margaret Thatcher, as it did Tony Blair later — and he was ready to take an inventive approach. As secretary of state at the Department of the Environment, he wanted to find new ways of opening up the public sector to the disciplines of the private market. In one of several initiatives to this end, he took up an esoteric branch of officialdom, staffed by ‘district auditors’ and essentially responsible for probity in the local government of England and Wales. Then he rewrote their rule-book, opened up their franchise to competitors from the private sector (whose access to date, while established since 1972, had in practice been problematic) and created a new body to preside over them. This was the Audit Commission, comprising sixteen non-executive commissioners drawn from across the political spectrum and both sides of the public/private divide.
Heseltine endowed the Commission with the political equivalent of three magic powers. First, it could turn its hand just like a private sector consultancy to any analysis of the public sector beyond Westminster that, in its own view, might further the cause of better management. Second, it could take the traditional notion of auditing and add one novel twist scarcely applied anywhere before 1983 (and even now untested anywhere in the private sector, though it is much discussed): public bodies within its franchise would be audited by genuinely autonomous auditors — appointed and coordinated by the Commission itself, and answerable only to the public and the courts rather than to their ‘client’ bodies in the field. And third, it could operate within Whitehall while remaining entirely outside the civil service, with an independent status that even included a licence to publish unsolicited critiques of the impact ‘on economy, efficiency and effectiveness in the provision of local authority services’ of any statute, or ‘any directions or guidance given by a Minister of the Crown’.1 The consequences of Heseltine’s boldness soon outran anything anticipated by his colleagues in government or their officials. The impact on public life in Britain has been profound, and often quite surprising.
Take first its remit to roam as a consultant. Crucially, Heseltine in 1983 chose as the first executive head of the Commission (its ‘controller’) a successful alumnus of a private sector organization with a formidably disciplined approach to the analysis of managerial problems in any context — McKinsey. This was John Banham, and over the next four years he proved an inspired choice. Banham saw immediately the opportunity to shape a role for the Commission as a very special kind of management consultancy. By taking the existing field force of auditors, with all their local knowledge and networks, and harnessing it to a freshly recruited central staff, most of them with backgrounds in operational research, he built a Commission that functioned as a sort of McKinsey-plus in the public sector. And after his departure, a second ex-McKinsey man, Howard Davies, continued and refined the process for a further five years.∗
Banham and Davies built a powerful business model that owed more than a little to the tool kit that both men had mastered during service with their previous employer. Data gathered from all over England and Wales would be used for robustly evidence-based analysis of any facet of local government. A national report on each topic would present arguments and conclusions based on comparative data from across the country, presented with state-of-the-art graphics to make even the most esoteric statistics accessible to all. Guides would then be prepared for use by auditors at a local level, enabling them to pursue improvements with the audited bodies under their charge, relating the local performance back to the national picture. Nothing quite like this had ever happened before in the public sector. By instilling quasi-commercial disciplines and positioning public services as goods in a quasi-marketplace with citizens as the end-purchasers, it caught the essence of what came to be known as New Public Management. The impact achieved by the Commission between 1983 and 1989 was sufficiently impressive for its role then to be extended into the NHS, much to the displeasure of officials within the Department of Health. This, perversely, was a fair measure of the way that it was by then starting to change the landscape of government.
Through twenty-five years, the Commission has issued more than 250 national reports. These have been at the forefront of a prodigious publishing programme that has also extended to a wide range of Occasional Papers, Police Papers, Management Papers and Handbooks, Executive Briefings and Bulletins (for a list of most of the titles, see Appendix 1). The early reports usually examined the management of specific tasks, some of them gloriously mundane like the purchase of council supplies or the collection of ratepayers’ rubbish bins. By the end of the 1980s, many were exploring far more complex topics such as the management of the probation service or the handling of community care for elderly people. By the end of the 1990s, with many of the easiest targets gone and the bar pitched ever higher — not least because councils were responding to their recommendations — the reports were increasingly looking into systemic weaknesses impeding the delivery of public services. There is probably much in the resulting bibliography that can still be read profitably today — and the publication of national reports remains a vital dimension of the Commission’s activities.
Capturing the essence of these reports within the confines of a narrative history has been a challenge. It was never going to be possible to do their contents real justice: many responded to complex situations, and the background alone would in most cases have required a long digression from the narrative. On the other hand, any proper appreciation of the Commission would be impossible without some grasp of what the best reports contained. With no claim to consistency, the text therefore refers fleetingly to many while lingering for longer on some of more intrinsic interest. These include, for example, reports on the state of the social services in 1986 (Making a Reality of Community Care), on the case in 1990 for a revolution in the use of day surgery for NHS patients (A Short Cut to Better Services), on the need in 1996 to rethink the basic strategy behind the deployment of police officers in the community (Streetwise: Effective Police Patrol) and evidence that same year of disturbing trends across the whole of the youth justice system (Misspent Youth). All report references are noted in their own index.
Over the years the Commission has managed to make a happy habit of publishing timely reports on topics of acute interest to politicians as well as professional interest groups, and many have had an enormous influence that is also an important part of the story. At the most obvious level, reports and papers from the Commission have helped to trigger direct changes in whatever unlit corner of the public sector they exposed to an unaccustomed light. Within a few years of the publication of A Short Cut, for example, some minor operations hitherto involving overnight stays in hospital were being routinely handled by the NHS as day-surgery cases. More broadly, a string of reports within one sector could help elicit a seminal change in working practices across the whole of that sector. By the early 1990s, it was already apparent that a succession of papers on the need for more professionalism in the running of local councils had made a telling impact on the way that local government in general conducted its affairs. By the end of the 1990s, ten years after a cagey start, many senior police officers and their counterparts in the Home Office were happy to acknowledge that they owed a similar debt to the Commission, for the work it had contributed to the modernization of the police service. (Another institution that was arguably changed in a radical fashion by the Commission, though it was most certainly never the subject of a Commission publication, was the National Audit Office. Set up in the same year as the Commission, but reporting directly to Parliament — its head, the comptroller and auditor-general, is an officer of the House of Commons — the NAO was given a broadly parallel remit for auditing the departments and agencies of central government. From the start, it too produced national reports — but their style and content in the 1980s were soon made to look almost Ruritanian by comparison with the Commission’s and a rapid catch-up was required.)
Auditors and analysts, with sharp pencils and disarmingly basic questions, were at first no more welcome inside hospitals than in police stations. But close cooperation with the Royal Colleges and the leaders of the medical profession produced a small library of powerful reports in the decade after the NHS was added to the Commission’s audit franchise in 1990. These helped to change attitudes within the NHS to dozens of topics, from the prevention of coronary heart disease to the management of beds, records and medicines. In 2002, the government transferred one of the Commission’s central duties within the health sector, the preparation of national reports, to a new body. After years of steady expansion, it was a rare diminution of the Commission’s franchise and marked another stage in the restructuring of public services that has been such a constant feature of life under New Labour since 1997. But, in recent years, the influence of the Commission in the health service has grown again, as attention has come to focus increasingly on standards of financial management within the NHS.
At the most general level, the Commission’s approach could fairly be said to have contributed significantly to the way that the analysis and management of the public sector were brought into line with their counterparts in the business world. Of course this was a wider phenomenon. By the end of the 1980s, with new technology transforming the presentational arts, it was increasingly evident that a powerfully written national report could change people’s views of the world. Privately funded think-tanks and publicly funded inspectorates sprouted up on all sides. It would be absurd to attribute their proliferation to the Commission alone. But they first emerged in response to an appetite that the Commission’s publications had undoubtedly done much to nurture. And by the time that Whitehall came to review ways of appraising its own performance, after 2005, the ‘Comprehensive Performance Assessment’ designed by the Commission for local government was openly acknowledged as a model for the appraisal of state departments. (The notion that they might be answerable to anyone other than their ministers and Parliament would have struck an odd note twenty years earlier — another sign of broad changes for which the Commission has arguably been a potent catalyst.)
The second of the Commission’s mould-breaking powers has enabled it to pull off the implementation of an auditing concept generally regarded until 1983 as an ideal too perfect for attainment. The principles of both public and private audit decree that the power to hire and fire auditors should be independent of the bodies — municipal or corporate — that are the subject of audit. The private sector has had difficulty achieving this, whether in Britain or anywhere else. No stock exchange, for example, has ever tried to impose external auditors on the companies whose shares are traded under its umbrella, though this would be a logical step. Instead, shareholders vote to appoint or dismiss auditors, who formally report to them. In practice, as anyone will know who has ever attended an annual general meeting of a large public company, the auditors are effectively selected for the job by directors of the company and their remuneration is overseen by those directors. In the public sector, similarly, there was not much emphasis before 1983 on the separateness of district auditors. Though their integrity was very widely respected, and seldom if ever seriously impugned, individual auditors often worked so closely with their audited bodies over so many years as to be almost part of the municipal family. Indeed, this was one of the features of the local government system that persuaded Michael Heseltine to go for a new and far more rigorous approach.
Hence Heseltine’s enthusiasm for the Commission, and its statutory responsibility for the appointment of auditors. For twenty-five years, the Commission has called upon a pool of audit firms comprising a number of private sector partnerships as well as the legacy body of district auditors (which has always been a separate conceptual entity, though it was only between 1994 and 2002 that it was actually run as an arm’s-length agency). In financial terms, the Commission has ‘purchased’ audits from this pool and has ‘resold’ the audits to audited bodies — building a mark-up into the process, by which it has generated the cash to pay for its national report work and other publications. In so doing, the Commission has taken the ‘purchaser/provider’ concept that was central to Margaret Thatcher’s reforming agenda all through the 1980s, and fashioned a derivative version for the special circumstances of the world of public audit. And, in the process, it has assured the auditors under its aegis of a robust independence. (The auditors and the central institution are often collectively referred to in this book as ‘the Commission’, where it would be pedantic to keep distinguishing between them and the distinction is not pertinent. Most importantly, though, the auditors have acted always as individuals with their own statutory responsibilities — or as representatives of private firms with their own joint and several liabilities — and they are identified separately from the Commission where the context duly demands it.)
The conspicuous independence of its auditors from their audited bodies has many times underpinned the effectiveness of the Commission as a policeman of the public sector. Sometimes this role has pushed it into the limelight. It had to confront the leaders of the so-called Loony Left in the 1980s, taking a firm stand against them in Liverpool and Lambeth — but treading with great care thereafter, to pre-empt what might arguably have become a full-blown constitutional crisis between central and local government under Margaret Thatcher. It is legitimate to wonder — given the animus against it within the Thatcher cabinet — whether local government in its historical form would have survived such a crisis. How it might have ended for the Thatcher government itself, but for the Commission’s involvement, is probably just as hard to say. (The Commission was given no real opportunity to pull off a second rescue over the poll tax, though it cannot be faulted for not trying.)
Again, later in the 1980s, the Commission and one of its appointed auditors had to deal with the potential financial implosion of a London borough council. Hammersmith and Fulham had entered into a complex web of financial contracts with the international banking community which at one point threatened to expose each citizen of the borough to a bill for several thousand pounds. Without the Commission on hand, to flush out the crisis in the first place and to provide critical legal assistance to the auditor tasked with steering it to a safe resolution, English local government might easily have been lured into a much more extensive engagement with the global derivatives markets in the 1990s that could have had calamitous consequences. And no history of the Commission would be complete without its own account of the scandal at Westminster City Council, where an appointed auditor (again from a private firm, ironically, as at Hammersmith) had to toil through much of the 1990s on an investigation into a blatant misuse of public power by the country’s flagship Tory council under Shirley Porter. In all these cases, and many other less egregious episodes, the Commission came under intense and often hostile scrutiny from all those who would have preferred to see auditors in the public sector taking a less inconvenient line. It stood its ground, as did the auditors, and the campaign medals were earned.
It is not just their stubborn independence, though, that has marked out the Commission’s auditors as a rare species over this quarter-century. In reorganizing the audit function under the direction of the Commission, Michael Heseltine wanted to shake up the district auditors because he was ambitious for them to do much more than watch over the probity of local authorities’ accounts. He also envisaged a much expanded role for private firms. In the event, Heseltine found himself pushing on an open door — and there proved to be less scope than imagined for the private firms to sweep in as new brooms — because the district auditors themselves had been keen for many years to widen their own remit beyond the ‘regularity’ work of checking that the accounts complied with the law. That remit has gone on widening ever since.
Here another striking distinction needs to be flagged between the private and public sectors. Private businesses do something with a view to making money; but public bodies spend money with a view to doing something. The key measure of a private business’s activities is the ‘money’ outcome and the key to this is a set of financial statements — hence the importance attached to all the conventions governing those statements, and to the question that must be answered by the auditor: do they give a ‘true and fair’ view of what has been going on? But the key measure of a public body’s activities must focus on the substance of ‘non-money’ outputs: given the expenditure recorded on, say, social services or highway maintenance or public museums, has the audited body, in the words of the relevant statute, ‘made proper arrangements for securing economy, efficiency and effectiveness in its use of resources’?2 Hence the greater scope for stretching the public auditors’ remit. Even in the policeman role, auditors can be required to go in pursuit of the ‘three Es’ as well as their regularity agenda. But the three Es have always been central to the ‘value-for-money’ (VFM) agenda that has defined their consultant role. And of course the pursuit of VFM has been a constant theme of the Commission’s reports — which is why the business model devised by Banham and his colleagues, built round investigative studies, was such an ingenious response to the divergent statutory duties laid on the Commission at the outset.
Much ink has been spilled over the years on the relative importance of regularity work on the one hand and of VFM on the other. Commentaries from time to time on the performance of the Commission have regularly questioned the balance between the policeman and consultant roles, too, and even their ultimate compatibility. In practice, though — given the importance of the three Es to both — they have always overlapped. Straightening out dubious accounting practices and pointing the way towards a more efficient use of resources have been two sides of one coin from the start. And this has been the common currency of the public auditors’ work. Few of them have found themselves featured in front-page stories, battling against adversaries like the Militant socialists of Liverpool, the financial nonwizards of Hammersmith or the gerrymanderers of Westminster. Most have beavered away unseen — combating, for example, the so-called ‘creative accounting’ phenomenon across local government in the later 1980s, or struggling to oversee the huge expansion of compulsory competitive tendering by councils from the late 1980s to the arrival of New Labour.
In both these episodes, auditors were called upon to exercise their judgement in ways that hardly answered to simple categorization as ‘regularity’ or ‘VFM’. It was plain by the 1990s that, whatever the label placed on it, their work was entailing a steadily more sophisticated assessment of what the audited body was doing. Under Banham and Davies, the Commission almost made a virtue of baulking at any overtly prescriptive messages for individual local authorities. Comparative data from across the country and prescriptive suggestions for the sector as a whole were left on the table, for individual councils to take up or otherwise at their own discretion. (Whether or not they exercised this sensibly would be for their local electorates to decide.) After the arrival of John Major’s Citizen’s Charter, though, this approach began to look almost perversely abstemious. Government looked to the Commission to provide support for a concerted drive to improve public services.
Andrew Foster, a former public servant in the social services and the NHS, embraced this same objective wholeheartedly through his ten years from 1993 as the longest-serving controller of the Commission. Armed with ever more measures of performance, the auditors engaged with their audited bodies in new ways: league tables heralded a readiness to evaluate the performance of councils, relative to their peers across the country, against a set of centrally directed criteria. It was a trend that accelerated rapidly after 1997. In its third decade, the Commission was expected to do more than illuminate the facts as in the 1980s, or point to underlying patterns as in the 1990s. It had to join with government in devising ways to effect real change. (It was an evolution labelled ‘sight–insight–foresight’ by those fond of such rubrics.) As this suggests, New Labour’s continuing quest for a quantum improvement in Britain’s public services posed a huge challenge for the Commission and its auditors — inviting them into a relationship with government that posed delicate questions for the independence of the Commission from government, always a far subtler matter than the statute-protected independence of the auditors from the audited.
This independence from government was the third of Heseltine’s magic powers. There exists today a broad spectrum of ‘independent’ bodies at the apex of British public life. At one end of it are those, like Her Majesty’s Inspectorate of Constabulary (HMIC) or HM Inspectorate of Prisons, led by a ‘designated office-holder’. They sit within a department of state and their leaders must report via a permanent secretary to a government minister. They may generally be regarded as independent, but their leaders can be dismissed — as some have been, in recent years. Then there are those bodies operating outside ministerial departments, such as Ofsted, the education inspectorate. They are free of ministerial control, but they are nonetheless a part of the government machine and must function within government conventions. Much more truly independent are most ‘non-departmental public bodies’, such as the Healthcare Commission launched in 2004. But even they are subject to policy directives, and the government pays the piper. The Audit Commission, like the BBC, sits at the opposite end of the spectrum from HMIC.
The Commission is a public corporation, wholly separate from government. It is manned by part-time commissioners under the leadership of a chairman, and served by an executive comprised of employees excluded by statute from the civil service and led by the controller. While in recent years it has begun to receive some direct fees from government, it remains substantially self-funding: if all government fees fell away tomorrow, the Commission would simply shorten by a fraction its menu of services. But these would remain extensive, including (as they always have done) the preparation of reports assessing the impact of government policy on the audited bodies within its franchise. And if these reports proved a serious irritant to the government? As a final measure of its independence, the secretary of state in its sponsoring department is precluded by law from issuing specific directives to the Commission. A general directive would be permissible, provided that it were issued publicly — but in twenty-five years, this has never once happened.
Perhaps, then, the Commission has been such a tame and toothless critic that directives would have been superfluous? The record suggests otherwise. In its early years while John Banham was controller, driving ahead with the commissioners’ (usually) full support, it asserted its independence — especially in lambasting the government for the counter-productive complexity of local government finances — with a vigour that almost certainly had officials running an eye over the statutory small print governing those directives. This did much to establish the credibility of the Commission with those in the local government world who had initially been quick to dismiss it as a tool of Thatcherism (though its second chairman, David Cooksey, was still being tagged as Maggie’s Hammer in the later 1980s). But the Commission paid a high price: its criticism of civil servants, rather than their masters, alienated many officials whose day-to-day cooperation was a prerequisite if it was to exert any real influence in Whitehall.
Howard Davies, a former civil servant himself, restored a vital balance to the Commission’s relationships with the large spending departments of state while at the same time continuing to strengthen its ties with audited bodies. By the time of his departure in 1992, the Commission had used its power to publish a series of influential ‘Section 27’ reports. These were the papers that examined the impact of government policy on the local government world, and were so called because it was Section 27 of the founding statute that spelt out the extent of the Commission’s remit in this area. They threw a harsh light on Whitehall’s stance towards a range of problems, from environmental health and homelessness to food safety and urban regeneration. It was a disappointment to the Commission that its Section 27 licence to criticize was withheld from the legislation extending its remit into the NHS. While this often seemed to make little difference to the subsequent scope of the Commission’s health work, it probably buttressed a view within the health department that Commission recommendations on policy would be out of place — or ‘unhelpful’, as they say in Whitehall. Independence for the Commission in this context would too often amount to excommunication. Elsewhere in government, though, its reports were usually well received, despite their critical content. This reflected an important truth, illustrated by the Commission’s story on many occasions: for it to remain both independent and effective within the Whitehall village required patient negotiation from day to day. Taking too independent a line would risk exclusion from decision-making circles, with officials fencing the Commission off their territory with signs marked ‘Policy Makers Only’. Too deferential a line would risk entangling the Commission in government initiatives, for which it was all too likely to be expected to act as executor on the ground — a high-risk job at the best of times.
Between one line and the other was a narrow path, and it grew steadily harder to follow as central government became increasingly preoccupied with improving public services from the early 1990s onwards. Through his first three-year term as controller, to 1996, Andrew Foster stepped along it with great care and no little skill. But he was determined to expand the franchise of the Commission wherever opportunities arose. Given the progress of John Major’s Citizen’s Charter and the ambitions of New Labour to go much further in the same direction, they arose soon enough. By 1997 and the arrival of the Blair government, the Commission was busy adapting to partnership arrangements with other bodies and collecting new responsibilities that acknowledged its leadership status: it was at the forefront of one government initiative after another to lift the quality of public services. Once again, though, the Commission had to pay a high price: Labour ministers came to see it almost as an executive agency, ready to act at Labour’s bidding. The timing made this doubly unfortunate, for the late 1990s were marked by government initiatives that were long on aspirations but short on detailed planning. When things went badly awry, as they did in 1999–2001, it almost amounted to nemesis for the Commission. Ministers opted for a sudden change of direction immediately after the June 2001 general election and it was far from certain they would take the Commission with them. Not with standing its statutory independence, it would only have taken a statute to abolish it. For Foster and many of his colleagues, this was an uncomfortable time, made more so by a host of internal difficulties.
The lessons learned from that bumpy passage into the new century were turned to advantage in the aftermath, first under Foster’s own lead and then under that of his successors. Radical thinking about the future of regulation in local government led to new ideas in 2001–02 that revived confidence in the Commission while helping the re-elected Blair government to refine its goals. After the Foster era came a new regime in which the Commission’s independence was starkly, and sometimes brutally, reasserted by a chairman, James Strachan, who briefly assumed what was effectively an executive role. A former investment banker, Strachan sought to impose a fresh tempo. With shrewd timing he pushed hard for a new and simpler approach to the general regulation of local government and public services. (Ironically, this of course distanced the Commission not just from government but from its own philosophy of earlier years.) A new controller with long experience of working in local government himself, Steve Bundred, helped confirm this change of tackby reappraising the nature of the Commission’s work both in theory and in practice. It helped that one of his colleagues in the boardroom of the Commission was Michael Lyons, handling a broadly similar brief as the author of an independent review on the future of local government. By the time a sixth chairman, Michael O’Higgins, came aboard in 2006, the Commission seemed once again to be in the happy position of both accumulating new responsibilities and enjoying respect as an independent entity at one remove from government and beholden to no single department of state.
The Commission’s story has been presented as a narrative on various counts. It is hoped these will make amends for a regular need, inevitably, to take liberties with the precise order of events. In the first place, a narrative was needed to lay down many of the facts as a matter of record. It seems unlikely they will be systematically recorded elsewhere. Politicians have little interest in management issues: it is a rare political memoir of the 1980s or 1990s that even mentions the Commission, let alone devotes a paragraph to it. Few management books, on the other hand, have much interest in it either. Even in academia, where New Public Management has hardly been a neglected topic, the Commission gets short shrift. (A respectable academic history of auditing in Britain, published in 2006, makes no mention of it whatever — though by size the Commission’s own field force has ranked for many years among the top ten audit firms in the country.)3 Nor would future historians have faced an easy task, reconstructing its story at a distance: the surviving archives from the early years are meagre indeed — while e-mails, over-zealous shredding and undated papers from project consultants will as usual have cast a deep shadow across the record since the mid-1990s. The Commission would have been left to flit like a ghost through any history of the public sector under Thatcher and her successors. Like Hamlet’s father, an honest ghost; but unedifying, nonetheless.
Its head office, in London’s Vincent Square until 2004, was launched without the blessings of what is commonly referred to as ‘a structured environment’. It was a loose organization, staffed much as though it were a private business by a few clever men and women who were left to recruit talent as and where they found it. But a shared intellectual excitement and a sense of being out ahead of the rest of the field helped engender a highly creative culture, far removed from the usual image of risk-averse public sector endeavour. As the Commission expanded, and its responsibilities multiplied, the need for more structure grew inexorably — not least, in order to manage the working relationship between Vincent Square and the auditors working for the Commission in the field. No assessment of the record from 1983 to 2008 could truthfully depict a chronicle of uninterrupted triumphs on the management front. The Commission has had its setbacks, like any other institution. It has probably had rather more than its fair share, for example, of personality clashes: perhaps they have come with the territory, given the kind of individuals attracted to working for it. Certainly they posed many difficulties from the early 1990s onwards, especially for Andrew Foster, who in ten years as the controller had to manage the passage of the Commission through a highly political environment.
It was not always apparent at the time, of course, that it would be a journey lasting as far as 2008 and beyond. New Labour’s direction of the public services since 1997 has seen non-departmental public bodies coming and going with sometimes alarming speed. It has been an achievement of sorts for the Commission just to have survived this era in (almost) one piece. Here again a narrative structure seemed essential, if only to capture the drama of the fast footwork needed on many occasions.
Viewed as a business, the Commission grew rapidly in size between 1983 and 2008, both at the centre and in the field (see Appendix 5, for the years up to 2006). Its field force also went through four difficult reorganizations — getting relaunched in 1983; taking on the NHS and adjusting to a simpler hierarchy in 1989–90; being transformed from the District Audit Service of Victorian pedigree to plain DA in 1994– 95; and being amalgamated with the Commission’s own newly created Best Value inspectorate in 2002. Huge amounts of time had to be devoted by senior managers and Commission members alike to matters of pay-and-rations, pensions, recruitment practices, staff promotions and the like. And running in parallel with all this was the need to administer an often complex relationship with the several private sector accounting firms that were mandated to work alongside the Commission’s own auditors in the field (see Appendix 6). A full portrait of this relationship alone could probably fill another (albeit rather shorter) book. The author hopes enough has been said about them to reflect the importance of the audit contracts that underpinned everything, without dwelling too long on details that would surely be of limited interest to the general reader. Would we be interested, after all, in a history of (say) the East India Company devoted at length to the terms of its trading contracts? It is surely the story of the nabobs and the maharajahs that is worth the telling, and so it is here — not least because of what their activities reveal about the political world in which they had to operate.
As a biography, this book is focused far more on the Life than the Times; given the complexity of local government, let alone the NHS, it could hardly be otherwise. Readers looking for more of a general background will turn to authors such as Christopher Foster, himself once a Commission member, whose British Government in Crisis presents a devastating picture of the turmoil there has been at the centre since 1979; or Simon Jenkins, whose books on the relentless centralization of political life since 1979, Accountable to None and Thatcher… Sons, chart its impact on the public sector far more broadly.4 But a detailed chronicle of the Commission does offer, in effect, an insider’s view of some of the more notable episodes — from Margaret Thatcher’s war against the councils in the 1980s to New Labour’s false start on public service reforms in the late 1990s — while providing another perspective on perennial themes such as the management of Whitehall and the formulation of government policy. Those inclined to see Britain’s public services, for instance, as the product of a deeply flawed political system — one that seeks to deliver a Nordic welfare model on the back of a US-style tax regime — may cite as evidence the Commission’s long struggle to deliver better results on the ground. As an institution charged in part with ensuring the most economic and efficient use of money in public services, it has accomplished a great deal. Whether enough money has been available to make those services truly effective has always been another matter, rather harder for anyone to be sure of.
Nonetheless, as a regulator intent on protecting the public purse while spreading best practice through the public sector in countless ways, the Commission has become indispensable. Conceivably it could be replaced – but probably only by a different body fulfilling the same functions. It has become an integral part of the way that the public sector in Britain works. In the process of growing into this role, the Commission has been a force for change that many people over a quarter of a century have conspicuously felt proud of working for. Old labels have slipped away. ‘District Audit Service’ went in 1994. Its successor, ‘District Audit’, went in 2002. Today the head offices of the Commission sit in a post-modern tower on London’s Embankment, so no one speaks any longer of ‘Vincent Square’, which for twenty-one years – pace all the auditors working in the field — was almost synonymous with the institution itself. And for many who worked there, as the author can attest from dozens of interviews, Vincent Square represented the happiest and most productive period of their working lives. Given their disproportionate impact on so many facets of public life, as recorded here, it might be wondered why the Audit Commission has remained such a singular experiment of its kind. But then, as the record also shows, there was always much about it that assured it of a unique status.