Today we are used to the idea that medical therapies and interventions are based on some form of evidence about patient outcomes. Indeed the new commissioning processes enshrined in the Health and Social Care Act require NHS commissioners to place patient outcome data at the heart of commissioning decisions. However this is not the positive boon it might appear. Much of the discussions about appropriate therapies are reduced down to restricted and binary debates about good or bad levels of efficacy. But debate of any kind is a relatively recent development. Even in the very recent past, showing evidence that a therapy had a favourable outcome was the exception rather than the rule. To understand how this change came about we need to look at the story of tuberculosis (TB) in the twentieth century. It can tell us a lot about the professions and the relationship between professional autonomy and clinical effectiveness.
It has been over 60 years since a group of researchers published their ‘landmark trial’ of the first effective anti-tuberculosis drug, streptomycin. It was not the first randomised control trial (RCT) of a drug but is widely recognised as marking a revolution in the ways in which clinical treatments were evaluated – a move from analysis centred on individual case studies towards routine evaluations of larger groups of patients randomly divided into different treatment groups that were then compared.
TB had been in decline in Britain since the 1850s, but at the beginning of the twentieth century it still killed significant numbers of people (mainly the urban poor). In the first decades of the twentieth century there were two main treatments options: the sanatoria rest ‘cure’ and ‘collapse therapy’.
The sanatoria rest ‘cure’ (or conservative treatment) was based on the idea that inactivity would allow the body to strengthen itself and thus fight infection. In the first decades of the twentieth century there was a massive building programme of private and local authority controlled sanatoria to treat TB patients in Britain, with similar schemes throughout much of Europe and North America.
Collapse therapy or artificial pneumothorax was introduced prior to the start of the roaring 1920s and was often combined with the rest ‘cure’. By the outbreak of World War II it was the most widely used treatment for TB. In its most basic form this treatment consisted of a surgical technique to allow a controlled infusion of gas into the plural cavity to ‘collapse’ an infected lung. This was thought to allow the lung to ‘rest’ (as a cast would allow a broken leg to rest) and thus cure itself.
One might expect that these widely used treatments would be evaluated for their outcomes. Yet most clinical writings in this period tended to solely concentrate on descriptions of the processes and techniques used in the therapies. There were a few isolated exceptions to this. For example, two early studies attempted to develop a technique to measure and compare the sanatoria as an effective treatment of TB. These studies used life insurance statistics to compare the deaths of those discharged from sanatoria to the expected level of death amongst the general population at that time. They found that in one group of discharged patients one would expect 26 deaths in the following five years whereas there were 562 amongst those who had been discharged from sanatoria. This came as something of a shock to the authors:
‘we confess these conclusions are not those we hoped to reach…we hope, however, that our medical friends will enable us to carry our investigations further and will let us have information about the after histories of their patients’.
These studies were not subsequently repeated and their findings were not widely cited in future publications – it appeared that the hope for future collaborations with their ‘medical friends’ were not to be forthcoming.
The evaluation of collapse therapy was similarly under examined. An analysis of 2,100 journal articles and special reports between 1929 and 1939 revealed that only 5 per cent attempted to report any results of treatment. In the 1930s there were some isolated attempts at a more systematic evaluation of collapse therapies using statistical techniques. These found that, at best, the therapy only offered a very small advantage in the fight against TB. This conclusion also appeared to meet with some resistance, as a commentary in The Lancet revealed:
‘Few if any clinicians… doubt its value (collapse therapy) in suitable cases, and it is impossible to avoid a feeling of disappointment that… painstaking and comprehensive work should have failed to establish a statistical basis for this conviction’.
In this period there was a more general debate about the applicability of statistics to clinical problems. Commentators, like Bradford Hill, who would later play a key role in the streptomycin RCT, had argued that applying simple statistical techniques to clinical problems would have a positive value. But other more conservative physicians followed a more individualistic ethos and thought that treatment was a matter of individual decision making by physicians about specific patients. Patients, and their therapies, should be treated as individuals and doctors’ individual authority to control, judge and be sole arbiters of treatment decisions were seen as paramount to clinical identity.
The use of statistics in medicine was seen as a threat to this individual authority. By its very nature the use of statistics meant that clinicians had to be willing to merge their individuality to take part in group investigations. They may also have had to accept the authority of an independent research team many of whom did not even hold medical degrees. In short the use of statistics and the randomised control trial hinted at a more collectivised and corporate system of medicine.
How was it that in the space of a few years there could be such a change in the evaluation of treatments not only for TB but for all therapies? Partly it was because the invention of viable drug therapy lent itself to statistical analysis in a way that would have been impossible for the sanatoria cure. But it also relates to notions of professional expertise and regulation. It reflects wider debates, both within and outside the professions, around issues of professional autonomy versus clinical efficacy. The push for clinical standards creates a culture of central regulation, and this tension between clinical regulation and professional autonomy is centre stage in the history of the medical professions. Clinical efficacy won out, but at what price for the proponents of professional autonomy? The sheer volume of clincial and quality metrics – QIPP, CCG outcomes frameworks, NHS Commissioning Board Outcomes Frameworks, patient experience data, patient outcome data, clinical outcome data and current international best evidence – suggest a highly regulated (and performance managed) profession.
