"Actuarial" methods of risk assessment

So-called "actuarial" methods of risk assessment have come into vogue in recent times. I have placed the word "actuarial" in quotation marks because it is rather misapplied. Strictly speaking, the word comes from the practices of insurance companies, and they have methods which are rather different from ours.

The principle behind actuarial methods is that we measure a range of variables and see which ones correlate with the outcome (for insurance companies, this may be car crashes and so on; for us it will usually be reconviction). Using purely statistical criteria, all those factors which do not correlate with reconviction are dropped, so avoiding irrelevancies and prejudices as in clinical judgement. Equally, those factors that do correlate with the outcome are not forgotten!

Thus, the use of objectively-defined risk factors should ensure that the inaccuracies of unstructured methods are avoided.

There are four main flaws in this approach too:
  1. Insurance companies use samples of hundreds of thousands, whilst we can rarely rustle up 1,000; they wouldn’t recognise our methods as actuarial at all.
  2. The development of the prediction scale has to be done retrospectively, so sample is not of  present-day offenders, who may be different. For example, the Risk Matrix 2000 prediction scheme for sex offenders was developed using a sample of offenders released in the late 1970s and followed up over 20 years to see who reconvicted. However, starting in the late 90s, a large number of sex offenders were convicted who had committed intrafamilial offences against children many years before. These people pose a very low risk, and the applicability of  the Risk Matrix 2000 criteria to today's rather different sex offenders is questionable.
  3. The prediction is really about a group; applicability to individuals is very much less accurate (Hart, 2005) though (marginally) better than chance for typical individuals.
  4. Actuarial schemes only use “static” (unchangeable) factors and cannot therefore reflect change in people. This means, for example, that they cannot be used in assessing the impact of treatment on an offender. Furthermore, in the case of offenders who have been in prison for a long time, they can only assess risk at the time of entry to prison. Such prisoners are older than average, and this reduces their risk very considerably. No actuarial scheme takes a risk reduction due to age sufficiently into account (Barbaree, Blanchard and Langton, 2003; Wollert, 2006).
As well as these flaws in the rationale, actuarial schemes also have practical disadvantages:
  1. Statistical “shrinkage”: that is, predictors work well with the research sample, but prediction is less good subsequently in the field.
  2. Risk factors may not be as objectively measured as they appear, and this introduces a source of error. Harris and Rice (2003) showed that the accuracy of their Sex Offender Risk Assessment Guide (SORAG) could be greatly affected by differences in scoring of the risk factors by assessors.
  3. Prediction is less good with sub-groups of offenders (incest, older offenders) than with the “general run” (the “average offender”, of whom there are none).
  4. The result is numerical, which conveys a spurious air of scientific accuracy; however, numbers are only as good as the facts they represent.
Despite these problems, actuarial matters do have some advantages. There is at least an attempt to use objective measures, which do avoid most problems of clinical subjectivity, and the result is that different assessors normally arrive at the same answer (more or less!). The resulting risk estimate places a man in an objectively-defined risk category, which gives at least some idea of his relative risk when compared with others. This matters, because even where the absolute risk is low (eg, in older offenders), there is evidence that the relative risk is reflected by actuarials (an old man with a high score still poses a higher risk than an old man with a low score).

However, many practitioners oppose the use of actuarial measures because they do not adequately reflect the risk posed by any individual, and it is individuals about whom we have to take decisions, such as whether to grant parole. There is also no doubt whatever that actuarial schemes do not adequately reflect the reduction in risk due to age. The Risk Matrix 2000, for example, ceases to take age into account beyond the age of 35. The otherwise rather similar Static-99 ceases at 25! This means that older offenders are being assessed as if they were much younger, and their risk overestimated as a result. This is particularly important for lifers, who are often much older than the average prisoner. Wollert (2006) concluded that actuarial predictors for sex offenders were "useless" for all except the youngest adults. Blunt words for an academic journal!