A first
illustrative analogy
Mediaeval
kings usually employed food tasters to detect poison.
This was a form of direct hazard assessment. It involved
a form of experimentation (the feeding of food to a
'volunteer'), and the observation of an outcome (whether
or not the 'volunteer' died in writhing agony within a
time frame).
An alternative approach might have been a risk assessment. The king's security advisers might have noted, for example, that corn had been grown by farmer Giles and that there had not been any reports of outbreaks of rust disease in his fields. It had been collected and stored by miller Jones, who was generally rather trustworthy. It had been cooked by Mr Smith - and was probably safe to eat.
However, the king would have been unlikely to abandon his particular variety of practical human toxicological testing in favour of such a risk assessment, as a food taster would (correctly) be seen as a more foolproof method.
Nowadays we could (and do) perform hazard assessments by animal experiment and direct chemical analysis.
Several points arise from this parable:
A second
illustrative analogy
Risk
assessments of the likelihood of the failure of
engineering structures, such as bridges, are generally
reliable. Relevant data about the strength of materials
used in construction and their rates of fatiguing are
usually available, and our understanding of the
mechanical physics of the design lead to accurate
predictions of performance under various loading
conditions. Thus it is reasonable to model a variety of
predictable scenarios such as overloading, high winds,
collisions with the structure or earthquakes and be able
to make realistic predictions of the likelihood of
structural failure.
This type of approach has been used to build safety margins into the design of many features of modern life. Bridges are generally overdesigned by a factor of five while aircraft, where weight is at a premium, are overdesigned by between 1.1 and 1.2 times. The tighter the margin of this overdesign the more precise and accurate the hazard assessment data need to be and therefore the higher the development costs.
However, even with such clearly identifiable hazards, risk assessment is far from perfect. History is littered with examples of 'teething troubles' associated with new designs. The Comet airliner had three disastrous tragedies before the design fault of square windows was found and rectified. Unpredicted and unidentified hazards are the most common reason for unrealistic risk prediction.
The message from this analogy is that hazards need to be identifiable, quantifiable and relevant to the risk under consideration to lead to reliable risk assessment. In the case of engineering structures, there is at least the prospect of recompense through legal action because it is usually straightforward to identify where blame lies.
The use of
risk assessment in complex systems when hazards are
poorly characterised
A whole
industry of experts in risk analysis has emerged
alongside the plentiful availability of cheap computing
power. Increasingly complex models have been developed,
including weather patterns. Although much improved over
the years, these are clearly probabilistic rather than
deterministic in nature and only of usable accuracy over
a relatively short time scale. And yet, weather modelling
is the result of the collection of information in what is
probably the largest database in the world from sources
such as satellites, ground stations, weather balloons and
oceanographic data. The limitation to the model is the
unfathomable complexity of the global weather system and
its stochastic nature.
So weather forecasting from computerised weather models can be of great value but the limitations are clear for all to see and are acknowledged.
An example
of the inappropriate application of risk assessment to a
class of air pollutants
For
substances that remain in the body, even after the
environmental level has fallen, the approach of simply
setting emission limits is unlikely to succeed in
protecting human health. A typical risk assessment for an
aerial pollutant from a plant, for example, a pesticide
manufacturing plant, will take into account the maximal
concentration of a persistent organic pollutant in the
plume. A terrain dispersion model is used to estimate the
maximum ground level concentration. At this stage in the
risk assessment process not a single piece of real data
will have been used. In environmental risk assessments
there is often an accompanying calculation of the
theoretical amount of the pollutant an adult, standing at
the point of maximum ground level concentration, for 20
years, might expect to assimilate, while eating food
produce grown at that spot. This approach is always
accompanies by an unstated model assumption, namely that
the said adult is free from pollutants in his body at the
start of the exercise, and that there are no other
sources.
Therefore the use of empirical dispersion models, presented as an assurance of lack of health risk cannot suffice with bioaccumulative persistent toxic substances. Essential additional information is required, namely a knowledge of the degree of intoxication of the target population before adding a new source. In toxicology, the only absolute determinant that will dictate whether a toxic effect occurs is the body tissue concentration.
The limitations of the approach are numerous. A confounding factor in the assessment of the effects of some groups of bioaccumlative substances is that their toxic effects can be subtle, and furthermore have their maximal effect on the next generation while in the womb. The results of such exposure may not become apparent until adulthood therefore making it extremely difficult to link to the cause.
The classical approach to risk assessment when predicting damage from pollutants is to consider each chemical in isolation, perform animal experiments and try to decide if there is a no effect level and therefore an environmental loading that can be considered safe. This is in spite of the fact that it is generally recognised that there is no threshold dose for cancer and allergenic effects.
With a number of organic pollutants which cause effects such as immunosuppression and hormone disruption, research is showing that mixtures can have an additive or synergistic effect. It appears that the foetus is most at risk to the effects of these types of pollutant. Under these circumstances classical risk assessment ceases to be of use and it becomes difficult or impossible to predict effects.
Risk assessment procedures are currently poorly served by ecotoxicological and toxicological data derived through chemical hazard assessment and exposure assessment. The important deficiences include:
What are the
consequences of this improper use of risk assessment
Risk
assessments should be viewed with deep scepticism unless
all the areas of uncertainty are defined and the process
is carried out with periodic, penetrating reality checks.
Recent studies on decreasing male fertility in developed
countries are a case in point. The effect seems to be
real and has been reproduced in a number of studies. The
range of putative estrogenic chemicals that might be
responsible is large. Society does not really possess the
intellectual, regulatory or legal instruments to approach
such a problem. None of this was predicted by risk
assessment even though test protocols for fertility in
animal models have been available for many years.
Conclusions
The
regulation of environmental safety has moved, over the
years, from a 'reactionary' to an 'anticipatory' mode.
Nowadays, developers who employ pervasive technologies
have to perform environmental impact assessments based on
risk assessment.
However, risk assessment can only be of use in predicting harm, when it is based on solid hazard assessment data. Frequently, however, the studies that could underpin realistic risk assessments have not been carried out. The reasons for such omissions can be cost or simply that such studies would be impossible, because of the complexity of the system under consideration. Under such circumstances, the assumptions (often termed 'fact-free models') replace physical data in the risk assessment. These assumptions are rarely stated explicitly, despite the fact that they can reduce the whole document to nothing more than an opinion.
For the non-expert in a particular technological field, and this can include politicians and regulators, the existence of a highly technical document containing lots of 'science' can lead to the confident pronouncement that there has been 'rigorous testing' and therefore complete safety, for example in the recent debate on GMO foods. This facilitates the circumvention of the use of the precautionary principle, one valid outcome of which can be to avoid development of potentially harmful products.
Recent public experiences with BSE have introduced a high degree of public scepticism concerning the power of scientists to predict outcome in complex systems. While the valid use of risk assessment to protect the environment and the public is to be supported, in order to avoid further erosion of public confidence there must be a tightening in the rigour of risk assessment procedures. All assumptions must be explicitly stated and reasons given for not having performed the relevant hazard assessments. Decision makers should receive training in the interpretation of risk assessments, especially with a view to developing a suitable level of scepticism when assumptions are being made. It is only by these measures that public support will be regained.
[As published in PEX
Newsletter No.3, June 1999]
(First published in Environment &
Health News, the newsletter of UNED-UK)