Chapter 9. Experimental studies

The survey designs described in chapters 6 to 8 are all observational. Investigators study people as they find them. Thus, subjects exposed to a risk factor often differ from those who are unexposed in other ways, which independently influence their risk of disease. If such confounding influences are identified in advance then allowing for them in the design and analysis of the study may be possible. There is still, however, a chance of unrecognised confounders.

Experimental studies are less susceptible to confounding because the investigator determines who is exposed and who is unexposed. In particular, if exposure is allocated randomly and the number of groups or individuals randomised is large then even unrecognised confounding effects become statistically unlikely.

There are, of course, ethical constraints on experimental research in humans, and it is not acceptable to expose subjects deliberately to potentially serious hazards. This limits the application of experimental methods in the investigation of disease aetiology, although it may be possible to evaluate preventive strategies experimentally. For example, factories participating in a coronary heart disease prevention project were assigned to two groups, one receiving a programme of screening for coronary risk factors and health education, and the other being left alone. Subsequent disease incidence was then compared between the two groups. The main application of experimental studies, however, is in evaluating therapeutic interventions by randomised controlled trials.

At the outset of a randomised controlled trial the criteria for entry to the study sample must be specified (for example, in terms of age, sex, diagnosis, etc). As in other epidemiological investigations, the subjects studied should be representative of the target population in whom it is hoped to apply the results. Comparison of two treatments for rheumatoid arthritis in a series of hospital patients may not provide a reliable guide to managing the less severe range of the disease seen in general practice. Subjects who satisfy the entry criteria are asked to consent to participation. When refusal rates are high a judgement must be made as to how far the volunteers that remain can be considered representative of the target population. They might, for example, be younger on average than the refusers. Is this important in relation to the study question?

Those subjects who agree to participate are then randomised to the treatments under comparison. This can be achieved using published tables of random numbers, or with random numbers generated by computer. When subjects enter the study sequentially (for instance, as they are admitted to hospital) then randomisation is often carried out in blocks. Thus in a study comparing two treatments, A and B, patients might be randomised in blocks of six. Of the first six patients entering the trial, three would be allocated to treatment A and three to treatment B – which patient received which treatment being determined randomly. A similar technique would be used to allocate treatments in each successive set of six patients. The advantage of this method is that it prevents large imbalances in the numbers of patients assigned to different treatments, which otherwise could occasionally occur by chance. It also ensures that the balance between the different treatments is roughly constant throughout the course of the study, thus reducing the opportunity for confounding by extraneous variables that change over time.

Sometimes major determinants of outcome can be identified at the time when subjects enter the study. For example, in a trial of treatment for acute myocardial infarction the presence of certain dysrythmias on admission to hospital might be an important index of prognosis. The use of randomisation means that such prognostic markers will tend to be evenly distributed between the different treatment groups. However, as further insurance against inadvertent confounding, there is the option to stratify subjects at entry according to the prognostic variable (for example, separating patients with and without dysrythmias) and then randomise separately within each stratum in blocks.

When outcome is influenced by other aspects of a patient’s management, as well as by the treatments under comparison, it may be desirable for those responsible for management to be “blinded” to which treatment has been allocated. Arrangements must be made, however, to permit rapid unblinding should possible complications of treatment develop. As far as possible, the criteria for withdrawing a patient from treatment should be specified in advance, although final responsibility must rest with the clinical team caring for the patient. Even if a patient is withdrawn from a treatment under investigation, follow up and assessment of outcome should continue.

The end points of trials vary from objective outcomes, such as haemoglobin concentration or birth weight, to more subjective symptoms and physical signs. Bias in the evaluation of subjective outcomes can be avoided by blinding the assessor to the treatment given. For example, if a new analgesic for migraine is being evaluated on the basis of reported levels of symptoms, it may help to use a pharmacologically inactive placebo for comparison. Otherwise, there is a danger that patients will perceive a benefit simply because they are getting something new. Similarly, if the end point is a subjective physical sign (such as severity of a skin rash) then the examiner is best kept ignorant about which patient received which treatment. It is important to measure not only the outcomes that the treatments are intended to improve, but also possible adverse effects. In a trial of the cholesterol lowering drug, clofibrate, the treated group showed a reduced incidence of non-fatal myocardial infarction, but their overall mortality was more than in untreated controls. This excess mortality could not be attributed to any single cause of death, but may have reflected unsuspected side effects of treatment.

The statistical analysis of randomised controlled trials is too complex to cover in a book of this length, and readers who wish to learn about the methods used should consult a more advanced text. Whatever analytical technique is adopted, it is important always to compare subjects according to the treatment to which they were randomised, even if this treatment was not completed. (In some cases it may not even have been started.) Otherwise, the effects of selective withdrawal from treatment may be overlooked. For example, in a trial to compare a  blocker with placebo in an attempt to reduce mortality after myocardial infarction, patients were withdrawn from treatment if they developed severe heart failure – a potential complication of ß blockers. The patients most likely to be precipitated into heart failure by the trial drug were those with more severe infarcts and therefore a worse prognosis. Fewer of such patients would be expected to develop heart failure while taking placebo. Thus if the withdrawals had been excluded from the analysis any benefits from the ß blocker would have tended to be spuriously exaggerated.

At the same time, it is also helpful to examine outcomes according to treatments actually received. One would be suspicious if the benefits from randomisation to a treatment were confined to those who did not go through with it!

The size of a randomised controlled trial may be decided in advance on the basis of calculated statistical power. Such calculations require specification of the expected distribution of outcome measures, and of the difference in outcomes between treatments that is worth detecting, and are best carried out in collaboration with a medical statistician. A problem with this approach, however, is that the trial may continue long after sufficient data have been accumulated to show that one treatment is clearly superior. Thus some patients would be exposed unnecessarily to suboptimal treatment. A way of avoiding this difficulty is to monitor the results of the trial at intervals, with preset criteria for calling a halt if one treatment appears to be clearly better.

Another problem with randomised controlled trials lies in the need to obtain properly informed consent from participants. Some patients find it hard to understand why a doctor should allocate treatment at random rather than according to his best judgement. This difficulty has prompted an alternative design, which may be applicable when comparing a new treatment with conventional management. Randomisation is carried out for all patients who satisfy the entry criteria, and those who are allocated to conventional treatment are treated in the standard manner. Those assigned to the new treatment are asked to consent to this, but if they refuse are treated conventionally. The need to explain randomisation is thus avoided. Against this, however, must be set two weaknesses. Firstly, as in any randomised experiment, the prime analysis is according to randomisation. If a substantial proportion of patients refuse the new treatment, then differences in outcome may be obscured. Secondly, neither the clinical team nor the patient can be made blind to the treatment received. The importance of this limitation will depend on the nature of the study and the end points being measured

Another modification of the randomised controlled trial is the crossover design. This is particularly useful when outcome is measured by reports of subjective symptoms, but it can only be applied when the effects of treatment are short lived (for example, pain relief from an analgesic).

In a crossover study, eligible patients who have consented to participate receive each treatment sequentially, often with a “wash out” period between treatments to eliminate any carry over effects. However, the order in which treatments are given is randomised so that different patients receive them in different sequence. Outcome is monitored during each period of treatment, and in this way each patient can serve as his own control.

Most experimental studies allocate and compare treatments between individual subjects, but it is also possible to carry out experimental interventions at the level of populations. We have already cited a coronary heart disease prevention project in which the units of study were the workforces of different factories.

As in studies of individuals, interventions in populations can be randomly allocated. However, if the number of populations under comparison is small then randomisation may not be of much value. Instead, it may be better to assign interventions in a deliberately planned way to ensure maximum comparability between different intervention groups. Control of residual confounding can be strengthened by comparing study and control populations before and after the intervention is introduced.

Like longitudinal studies, experimental investigations tend to be time consuming and expensive. They should not, therefore, be undertaken without good reason. However, if well designed and conducted, they do provide the most compelling evidence of cause and effect.