Saturday, 16 August 2014

Bayesian probability analysis of the guilty verdict against Megrahi for the Lockerbie bombing

In my first post about the Lockerbie bombing, I discussed Morag Kerr's book reconstructing the commission of the Lockerbie bombing and demonstrating the innocence of the convicted man, Abdelbaset al-Megrahi.  In common with most humanistic reasoning, neither the verdict that condemned him nor Kerr's argument for his exoneration deployed any arithmetic of probability in analysing the evidence.  I think the widespread lack of arithmetical analysis of evidence is a serious weakness in fields like criminal law and history.

In this, I am following Richard Carrier in his book Proving History.  I am persuaded by him that we ought not just to use adjectives like "possible" and "probable" when we debate which theories best explain the evidence before us on a contentious historical or forensic question.  Additionally, we should use Bayes' Theorem: using numbers to express our opinions, and multiplying and dividing them according to Bayes' formula in order to calculate our reckoning of which theory explains the evidence the best.  The three main virtues of Bayes' Theorem are that it forces the analyst of evidence to specify clearly how good they think a theory is at explaining the evidence; it enables them to put all the evidence together in a mathematically sound way; and, above all, it forces them to look for evidence that supports their theory better than alternative theories, thus helping them to overcome the common failure to give alternatives due consideration.  Of course, different people can have different opinions about probabilities: the virtue of Bayes is that it brings out exactly what people agree and disagree about, and thus focuses their debate productively on crucial areas of disagreement.

Here is a simple account of how to use Bayes' Theorem:
  • We shall test the evidence for theory T (i.e. Megrahi committed the crime) by comparison with theory Not-T or ~T (i.e. Megrahi did not commit the crime).
  • Firstly, we determine the prior probability of T being true.  This is the probability that T is true based simply on our background beliefs; that is, based on general knowledge we have before we start to consider the specific details of the case.  E.g. in general, how probable is it that a bomb in a suitcase could be loaded at Malta?  How probable is it that a bomb could evade detection while undergoing x-ray checks at Frankfurt and Hamburg?
  • Next, we take each piece of specific evidence in turn and consider for each piece its conditional probability of existing if, firstly, T is true, and if, secondly, ~T is true.  E.g. how probable is it that Megrahi would be present at Luqa airport on the day of the bombing if he were not there to load the bomb into the baggage system?  How probable is it that a mysterious piece of unaccompanied baggage would have passed through Frankfurt airport's baggage transfer system, if it were not the bomb-case hypothetically loaded in Malta?
  • For each of T and ~T, we multiply the probabilities of each piece of evidence existing, to produce the conditional probability of the whole set of evidence in existence, assuming the truth of each theory in turn.
  • Finally, we calculate the product of the prior and conditional probabilities of the background and specific evidence, assuming the truth of T, and then assuming the truth of ~T, and then we put these two consequent probabilities into a ratio, giving us the relative probability of theory T.  Since in this case T and ~T exhaust all the possible theories, we can multiply the consequent probability of T by 100 to produce a percentage.

If this does not make sense, please read try the explanations on these pages.

I want to perform a Bayesian analysis on the evidence that convicted Megrahi of the Lockerbie bombing, both out of interest to see the result, and in order to show up clearly which errors of reasoning, if any, the judges fell into.

Now, let's determine the prior probability that the narrative of the crime constructed in the verdict was a true reflection of events, considering only such general background knowledge as we have and was included in the verdict.  An expert in airline logistics and criminology would obviously be able to give more expert opinions, but I will do the best I can according to my own lights.  I will assume that if the bomb was loaded at Malta, then Megrahi was the guilty party.  We can express the relative probabilities of each piece of evidence, background or specific, on each theory T and ~T as a ratio: 1:1 when the evidence is equally probable on either theory; 10:1 if it is 10 times more probable assuming the truth of T than ~T.

Background knowledge leading to a prior probability
  • How probable was it that a bomb could be introduced onto a plane at Luqa airport?
As per the verdict:
The evidence of the responsible officials at the airport, particularly Wilfred Borg, the Air Malta general manager for ground operations at the time, was that it was impossible or highly unlikely that a bag could be introduced undetected at the check-in desks or in the baggage area, or by approaching the loaders, in view of the restricted areas in which the operations proceeded and the presence of Air Malta, Customs and military personnel. Mr Borg conceded that it might not be impossible that a bag could be introduced undetected but said that whether it was probable was another matter. ... the Crown accepted that they could not point to any specific route by which the primary suitcase could have been loaded ... The absence of any explanation of the method by which the primary suitcase might have been placed on board KM180 is a major difficulty for the Crown case... (p. 41-2)
I would normally rate the probability of a bomb in a suitcase getting onto a plane at Luqa airport at lower than 1%, based on the expert evidence that it was "impossible or highly unlikely".  I shall rely on this evidence, even though it could potentially be biased against the possibility by Mr Borg's care to safeguard his reputation.  But, for the sake of making Megrahi's guilt as probable as we can reasonably make it, in order to find the maximum probability that he was guilty, I will go ahead and allow a 10% probability.  This is way higher than the real probability, but if we use maximum probabilities then we will at least know by the final calculation that Megrahi's guilt cannot be any more probable than the number we calculate in the end.

  • How probable was it that a bomb such as destroyed Pan Am 103 could have evaded x-ray detection at Frankfurt?
The bomb was hidden in a Toshiba radio.  Not only were the normal x-ray checks done on the luggage that transferred at Frankfurt onto the plane to Heathrow that was set to go on to New York City, Pan Am 103, but at Frankfurt the x-ray operators were under special warning to watch out for bombs hidden in radios.  The judges were aware of this:
It was submitted that the x-ray would, in all probability, have detected any explosive device in a case, particularly as the staff at Frankfurt were aware of warnings to look out for explosive devices hidden in radio cassette players. One such warning was issued after the Autumn Leaves operation in October 1988. Another, more limited, warning was issued because there was understood to be a threat that a woman from Helsinki would attempt to smuggle a device on board an aircraft. It was submitted that that examination would have revealed the presence of the radio cassette player and its contents, particularly in view of the fact that there had been a warning to look out for explosive devices hidden in radio sets. (p. 37)
 However, with regard to Mr Maier, the relevant x-ray operator, whose written statements were entered as evidence since he was too ill to attend the trial:
Mr Maier explained that he had had some limited training in the use of the machine, but said that in the course of using it he had taught himself to distinguish various sorts of electrical equipment, and that he knew how to tell if explosives were present, from their appearance. Neither statement directly dealt with the question whether, and if so how, Mr Maier would detect explosives hidden in a radio cassette player. What he said was that the approach in dealing with electrical equipment was to see whether it presented a normal appearance, for example whether it had a plug. Other evidence, however, particularly the witness Oliver Koch, Alert’s trainee manager at the time, shows that the standard of training given to Alert employees was poor. That was also the view of the FAA investigators who visited Frankfurt in 1989. Mr Maier’s description of what he looked for does not suggest that he would necessarily have claimed to be able to detect explosives hidden in a radio cassette player. There was no expert evidence as to the ease or difficulty of detecting such hidden devices. The x-ray examination is one of the factors to be taken into account but it is only one factor to be weighed along with the others.  (p. 38)
It is difficult as a non-expert to know the probability of the bomb being detected.  The existence of the warning argues in favour of it, but the low quality of training argues against it.  It is frustrating that such a low standard of evidence was left unimproved due to the illness of a man.  Surely courts should have a means of asking for important evidence that was not presented at trial before reaching a verdict!  Anyway, I will allow an outside 50% chance that the bomb could have got through this dubious security system.

We need to multiply together the probabilities of the bombs being loaded at Luqa and evading detection at Frankfurt.  10% multiplied by 50% gives a 5% outside chance.  So, based on these elements of background knowledge, it was at best highly improbable, 19-to-1 against, that the crime could have been committed as the verdict said.

It was a highly unlikely crime to come off.  That does not mean Megrahi did not beat the odds and commit the crime in this way.  If the specific evidence of his involvement is sufficiently strong, in that it demands decisively to be explained by his guilt as per theory T, then the conditional probability may be high enough to overcome the inherent improbability of the crime.

Specific evidence leading to a conditional probability

Here is the concluding paragraph of the verdict listing the main points of evidence adduced against Megrahi.  I have coloured the separate points that I will deal with so as to bring out how I will deal categorise and consider the evidence:
We are aware that in relation to certain aspects of the case there are a number of uncertainties and qualifications. We are also aware that there is a danger that by selecting parts of the evidence which seem to fit together and ignoring parts which might not fit, it is possible to read into a mass of conflicting evidence a pattern or conclusion which is not really justified. However, having considered the whole evidence in the case, including the uncertainties and qualifications, and the submissions of counsel, we are satisfied that the evidence as to the purchase of clothing in Malta, the presence of that clothing in the primary suitcase, the transmission of an item of baggage from Malta to London, the identification of the first accused (albeit not absolute), his movements under a false name at or around the material time, and the other background circumstances such as his association with Mr Bollier and with members of the JSO or Libyan military who purchased MST-13 timers, does fit together to form a real and convincing pattern. There is nothing in the evidence which leaves us with any reasonable doubt as to the guilt of the first accused, and accordingly we find him guilty of the remaining charge in the Indictment as amended. (p. 82)
I shall take the evidence against Megrahi point by point.  Let us see if they really do "fit together to form a real and convincing pattern".

  •  The bomb-suitcase containing clothes from Gauci's shop in Malta, and Gauci's "identification of the first accused (albeit not absolute)" (there was a second accused who was found no guilty).
I will assume that whoever bought the clothes in Gauci's shop, that later went into the bomb-suitcase, was the guilty party.  Now, the fact is that Mr. Gauci did not really identify Megrahi as the purchaser of the clothes:
Mr Gauci picked out the first accused at an identification parade on 13 August 1999, using the words as written in the parade report “Not exactly the man I saw in the shop. Ten years ago I saw him, but the man who look a little bit like exactly is the number 5”. Number 5 in the parade was the first accused. He also identified him in Court, saying “He is the man on this side. He resembles him a lot”.  (p. 55)
Mr Gauci’s initial description to DCI Bell would not in a number of respects fit the first accused.  (p. 65)
Gauci actually mischaracterised Megrahi to the police in terms of his age and height.

Let us consider this evidence in Bayesian style.  That means determining the probability that Gauci would say the purchaser of the clothes resembled Megrahi, and tell the police at first that the customer was of the wrong age and height, first assuming Megrahi's guilt, then assuming his innocence.  If Gauci had indeed sold the clothes to Megrahi, I would expect him to pick him out fairly definitively.  If Megrahi was not the man, then I would expect Gauci to say at best that Megrahi only looked "a little bit like exactly" the man and fail to give police an exact description.

What is bizarre is how the judges reasoned about this uncertain semi-identification:
Unlike many witnesses who express confidence in their identification when there is little justification for it, he was always careful to express any reservations he had and gave reasons why he thought that there was a resemblance. There are situations where a careful witness who will not commit himself beyond saying that there is a close resemblance can be regarded as more reliable and convincing in his identification than a witness who maintains that his identification is 100% certain. From his general demeanour and his approach to the difficult problem of identification, we formed the view that when he picked out the first accused at the identification parade and in Court, he was doing so not just because it was comparatively easy to do so but because he genuinely felt that he was correct in picking him out as having a close resemblance to the purchaser, and we did regard him as a careful witness who would not commit himself to an absolutely positive identification when a substantial period had elapsed. We accept of course that he never made what could be described as an absolutely positive identification, but having regard to the lapse of time it would have been surprising if he had been able to do so. (p. 66)
That's right!  The tentativeness of Gauci's identification was taken not as an indicator of its weakness, but explained away as care to be accurate!  This is completely the wrong way to handle uncertain evidence.  Gauci's identification should be held highly suspect because it was so uncertain, not lauded as the best that a careful witness would allow himself to say.

I would rate the probability of Gauci failing to identify Megrahi properly if he were the purchaser of the clothes as 50%.  If Megrahi were not the purchaser, I would rate Gauci's semi-identification of Megrahi as perhaps 90%, given that on this theory Megrahi would merely have the misfortune of somewhat resembling the real purchaser.

So Gauci's semi-identification evidence is actually a blow for the theory of Megrahi's guilt by a ratio of 9:5.  Percentage-wise, the probability of Megrahi's guilt, given Gauci's evidence, would be 5 in 14, thus 36%.

You can start to see what mistake the judges made: they chained together a series of improbabilities as if they added up to a probability.  Rather, they multiply up to a greater improbability!

Note that Kerr in her book totally undermines the validity of Gauci's evidence.  Among other things, Gauci had seen a photograph of Megrahi linking him with the Lockerbie bombing before he ever spoke with the police.  Furthermore, Gauci also told police that a picture of an Egyptian terrorist named Abu Talb resembled the customer, and this suspect is known to have visited Malta before the bombing, giving him the opportunity to buy clothes at Gauci's shop.

  • The transmission of an item of baggage from Malta to Heathrow.
The issue of whether a piece of unaccompanied luggage passed from Malta to Heathrow via Frankfurt is very complicated.  Suffice to say that Kerr shows the system was basically a mess, and the mystery bag supposed to be in the system could just as easily have come from Warsaw as Malta.

An example of how Kerr describes the transfer-baggage mess:
Not only could no bag from Malta be discovered, at least thirteen more items remained unaccounted for. Given the sheer number of untraceable baggage trays there was really no reason to believe any particular one of them was the bomb, and the German police gave up. They appear to have been so thrown by the impenetrability of the worksheets they decided the whole thing was a complete waste of time. ... [The German investigator] doggedly hunted for some legitimate item of luggage that might have been in tray 8849 [supposedly transferred from the flight from Malta]. He failed – but then, he also failed to trace anything that might have been in at least five of the other trays on the printout, which puts that into a bit of perspective.
Kerr quotes from the summary of the German police investigation of the Frankfurt element of the case:
This item of luggage cannot be said with certainty to originate from KM180 [from Malta] for the following reasons. [Many reasons given.]  Throughout the inquiries into the baggage for PA103A there was no evidence that the bomb suitcase had been transferred with the luggage either from or via Frankfurt Main to London.
 She goes on:
The Scottish branch of the Lockerbie investigation, despite having found nothing on Malta to contradict that position, simply ignored him.
 So it is strange to read this from the judges:
The evidence in regard to what happened at Frankfurt Airport, although of crucial importance, is only part of the evidence in the case and has to be considered along with all the other evidence before a conclusion can be reached as to where the primary suitcase originated and how it reached PA103. It can, however, be said at this stage that if the Frankfurt evidence is considered entirely by itself and without reference to any other evidence, none of the points made by the defence seems to us to cast doubt on the inference from the documents and other evidence that an unaccompanied bag from KM180 was transferred to and loaded onto PA103A.  (p. 38)
The judges thought they had good documentary evidence that an unaccompanied bag transferred at Frankfurt from Malta.  They did however consider some issues with the evidence, including the issue about an unaccompanied bag from Warsaw:
Reference was also made to another item in production 1060. ... this bears to show that an item coded at a station in HM at 1544 on 21 December also was sent to PA103A, and reference to the coders’ records bears to show that baggage from flight LH1071 from Warsaw was being encoded at that station at that time. It was agreed that no passenger from that flight transferred to PA103A, so that the records seem to show the presence of another unaccompanied bag on that flight. (p. 36)
After mentioning this issue, however, the judges do not go on to provide any analysis of it.  It is incredible!  They had evidence that the supposed bomb-suitcase could have come equally well from Warsaw as Malta.  Yet they mentioned this evidence only to ignore it completely!

If we include the evidence of a bag transferring from Malta, and say it was 10 times more likely that an unaccompanied bag transferred from Malta via Frankfurt on the hypothesis of guilt than on the hypothesis of innocence, then this would be strong evidence against Megrahi.  So we can also say that the judges' misuse of the baggage evidence, if they were thinking in Bayesian terms, would have been a very significant mark against Megrahi (it would make my final calculation 72%, so positive but not really enough to exclude reasonable doubt).  However, apart from Kerr's demonstration that the evidence did not necessarily mean what the judges thought it meant, the judges seriously misused even the evidence as they saw it.  They completely ignored the evidence that the purportedly unaccompanied bomb-suitcase might have come as easily from Warsaw as Malta.

Let's see what would have happened if the judges had included in a Bayesian calculation the evidence about the Warsaw bag.  This is actually an interesting bit of probability work.  Assume that an unaccompanied bag has a high probability of containing a bomb, as is required by the judges' reasoning about the evidence of the bag that they thought came from Malta.  What then is the probability that an unaccompanied bag would get onto the Frankfurt-Heathrow feeder flight from Warsaw on the hypothesis of Megrahi's guilt?  Call that number x.  It has to be a low number, since this bag cannot be the bomb, yet unaccompanied bags, ex hypothesi, are likely to contain bombs.  It is therefore an improbable sort of bag.  Now, what is the probability of such a bag feeding onto the flight from Warsaw on the hypothesis of Megrahi's innocence?  Surely it is a number much greater than x.  This is because, if an unaccompanied piece of luggage is regarded as likely to contain a bomb, and a bomb was fed onto the Heathrow flight, but that bomb did not come from Malta, then it must have come from Warsaw (or even somewhere else, since there were other dubious transferred bags).  Therefore, assuming unaccompanied bags are likely to be bombs, which the judges must believe, the correct Bayesian interpretation of the existence of a piece of unaccompanied luggage feeding in from Warsaw is that this fact is much more likely on the hypothesis of Megrahi's innocence.  Alternatively, if unaccompanied bags are regarded as not likely to contain bombs, then the evidence that an unaccompanied bag came from Malta is no more probable on the hypothesis of guilt that on innocence.

Since the judges included as a fact the existence of a suitcase from Malta, and might have rated that as, say, 10-to-1 evidence against Megrahi, by the same token they should really have rated the existence of a suitcase from Warsaw, equally likely to have existed, at the same 10-to-1 in Megrahi's favour.  This is because, if the existence of a bag transferring from Malta makes it more likely that the bomb came from Malta, then the existence of a bag transferring from Warsaw does the same for the Warsaw origin.  So the net effect of the evidence the judges had about luggage transferring at Frankfurt should have been neutral: 1:1

Kerr gets much closer to a correct understanding of the importance of the mysterious bag that seems to have come from Warsaw, even though no legitimate transfer baggage was supposed to come from that flight:
Both [the Malta and Warsaw mystery bags] were single items apparently coded at a time and a place where no legitimate luggage for PA103A was being handled. We’re completely out of legitimate luggage to reconcile to either of them. More lost, misrouted or rush-tagged cases being shuttled to Heathrow? Who knows. The fact is, though, that it has happened twice, and in the context of a significant number of other items being less than conclusively reconciled. Tray 8849 is anything but an isolated anomaly shrieking “bomb here!”

  • Megrahi's association with Libyan intelligence and with people who sold and handled the MEBO timers, one of which set off the Lockerbie bomb.
Kerr's investigation shows that this evidence was flawed as well, since the timer was of a type not used by the Libyans (it was made with pure tin rather than alloy, so it does not match the items made for Libya).  But I will disregard this and focus on the judges' reasoning.  Let us accept for sake of argument that Megrahi had access to the timers, one of which set off the bomb.  The judges believed the timer was of a kind only supplied in an order of 20 to the Libyan military.  What follows?

It follows that Megrahi would have had the opportunity to commit the crime.  But it does not follow that it is probable that he did it.  There is a huge difference between possible and probable.  Only if Megrahi's access to the timers makes it probable that he committed the crime should the judges draw a positive conclusion from this piece of evidence.

Let's look at this point in Bayesian style.  What is the probability that Megrahi would turn out to have access to the MEBO timers if he committed the crime?  Obviously 100%.  What is the probability that Megrahi would turn out to have access to the MEBO timers if he did not commit the crime?  Obviously still 100%!  The access and association are certain on either theory because they refer to the time before the crime.  What access and association Megrahi had before the crime is not evidence of whether he not he committed it, except in the sense that they made it possible for him to have committed it.

This was a crucial mathematical error by the judges.  The (purported) fact of Megrahi's access to the right sort of timers only raises the probability that he committed the crime in the sense that it makes it possible rather than impossible.  It does not make it probable, since even if he did not commit the crime, it would still be a fact that he had access to the timers.  Therefore the correct use of this evidence is to regard it as indifferent towards the relative probabilities of theories T and ~T.  It should have no positive effect moving the judges towards a guilty verdict, except in so far as making it possible that Megrahi committed the crime.  There can be no movement above 50% probability however.  The correct ratio to draw from his access to the timers is 1:1.

It is not as if only Megrahi had access to these specific timers, and it is not as if the Libyans could not have sold or supplied them to somebody else.  The CIA had an equivalent that they had seized in Togo, so who knows what hands the Lockerbie one might have been through?  It is even possible that the Libyan timers were used by Libyans to carry out the bombing, but not by loading the bomb at Malta.  Warsaw, as we saw, was an equally probable origin as per the evidence the judges considered; and in fact Heathrow was a far more likely origin, as I explained in my previous post.

However, we can also ask how probable it was that the bombing would be perpetrated using these specific MEBO timers on the hypothesis that Megrahi did it (on orders from Libyan military or intelligence forces), compared with the probability on the hypothesis that he did not.  It can be argued to be more likely that such timers would be used if Megrahi committed the crime than another, non-Libyan culprit.  But since we could not say that the timers were not used by another Libyan in Warsaw or Heathrow, or supplied to other forces by the Libyans, I am not prepared to grant a significant imbalance in the probability in favour of Megrahi's guilt.  There are lots of other possibilities that also involve the use of the same timer.

Furthermore, Kerr explains another alternative possibility:
The Blinkbonny fragment almost certainly originates from the same template as the Thüring boards [those supplied by MEBO]. That doesn’t, however, mean that it must have been made directly from that template. Printed circuit boards are easy to copy by photographing them, and any such second-generation copy would reproduce the tracking irregularities from the original. Anyone who had an original timer or even just a circuit board could make such a copy. ... had some other group who had got hold of one of the original models decided to copy this nifty little number for themselves?
So the timer would not even necessarily be one supplied to Libya by MEBO; it could be a copy.

As I noted, Kerr's analysis makes this discussion of the timer academic, since she shows that it was not the same as that supplied to Libya after all.  The timer's provenance is in fact rather mysterious.

  • Megrahi's presence in Malta on a false passport at or around the material time.
Megrahi failed to provide an explanation for this.  Were the judges right to draw an adverse inference from this?  Let's consider: would Megrahi be more likely to fail to do so if were guilty of the crime, than if he were innocent?  It is probably fair to say that, were he in Malta on an innocent mission for the intelligence services, he would have explained this to the court to try to save himself from prison.  On the other hand, there may be other reasons why he felt he could not reveal why he was there.  I don't know.  It looks bad for Megrahi, I will say by 10-to-1.

Let's calculate the conditional probability of Megrahi's guilt, based on the specific evidence the judges relied on.  The ratio of probability on T relative to ~T is 5:9 times 1:1 times 1:1 times 10:1 equals 50:9, or an 85% conditional probability of guilt.  As can be easily seen, the only evidence that supports Megrahi's guilt is his failure to explain his presence in Malta on a false passport.  The identification evidence is so weak as to work in favour of his innocence, the evidence of a transfer bag from Warsaw neutralises the evidence of a bag from Malta, and the timer evidence is indifferent between guilt and innocence.  Megrahi should certainly have come up with a reason for his suspicious activity.

Consequent probability

In order to find the probability of guilt that the judges should have arrived at by applying correct, Bayesian reasoning to the evidence they cited, we plug our numbers into the formula using Richard Carrier's online Bayesian calculator:

That is, the highest probability of guilt that the judges should have arrived at was 23%.

To explain the arithmetic, the prior probability of guilt (5%) is multiplied by the conditional probability of the specific evidence on the hypothesis of guilt (85%).  The product is divided by the sum of the product of the prior probability (5%) and the conditional probability of guilt (85%) and the product of the prior probability of innocence (95%) and the conditional probability of the specific evidence on the hypothesis of innocence (15%):

(0.05 x 0.85) / (0.05 x 0.85)+(0.95 x 0.15) = 0.0475 / 0.185 = 23%.

Megrahi should have been found not-guilty (or alternatively not-proven in the Scottish legal system).  This result is based on the evidence the judges had, except for the misused evidence about the baggage transfer which I excluded because its use contradicted the findings of the police report.

The crime was such an unlikely one that terrifically good specific evidence was needed just to make Megrahi's guilt turn out probable.  If the conditional probabilities were 99:1 for Megrahi's guilt, then the consequent probability would turn out as 84%.

The judges' crucial error is clear and highly important as an example of how forensic and historical reasoning can go badly wrong:

They chained together a series of improbabilities as if they added up to a probability.  Rather, they multiply up to a greater improbability!

The crime was hugely improbable on background knowledge alone.  The identification evidence was so poor as to work in favour of acquittal, whereas the judges used it to help establish guilt.  The evidence of Megrahi's access to the timer used was irrelevant, but the judges used it to help establish guilt.  Of all the evidence, only Megrahi's failure to explain his suspicious behaviour was a real piece of evidence against him.


The judges failed to use Bayesian reasoning, which would have shown them that, far from a series of improbabilities adding up to a proof of Megrahi's guilt, they should have multiplied them out to a much greater sense of doubt.  They failed to appreciate that the crime was such an unlikely one on principle, that iron-clad evidence of Megrahi's guilt was required to overcome the prior improbability: extraordinary claims require extraordinary evidence.  A circumstantial case built on improbabilities does not cut it.

If the judges had applied correct probabilistic reasoning to the facts they did have about an unaccompanied bag from Warsaw, then this would have neutralised the evidence of an unaccompanied bag coming from Malta.

Of course, if we included the evidence explained yesterday, and considered how probable it was, on a hypothesis of Megrahi's guilt, that a mysterious suitcase answering to the description of the bomb-case would be seen by a baggage-handler at Heathrow before the feeder flight from Frankfurt had even arrived, then it would only be fair to divide the 23% we have come to here by maybe 10 times, if not more.  Include all the evidence, and the probability of guilt is minimal.

Moreover, include a more realistic expectation of the probability of getting the bomb into the baggage system at Malta, and the probability drops again to a miniscule number.

In sum, even without the new understanding born of Kerr's investigation, Megrahi should not have been found guilty.  With it, his innocence is proven.

Thus the worst mass-murder in British history, the killing of 270 people, should be regarded as an unsolved crime.

Thank you for reading if you got all this way!


  1. This is beyond me - but who says the IED was contained within a Toshiba radio-cassette recorder? It may well have been but I am unaware of any credible evidence that it was.

  2. The items most affected by blast damage had had bits of a Toshiba radio blasted into them (verdict pp. 9-10).

  3. The difficulty with trying to present something like this as Bayesian reasoning is that some smartarse will simply quip back "argument from incredulity!" Believe me, I've tried. The riposte is merely that you can't disprove something that actually happened by showing that it was very unlikely that it would have happened.

    You're quite right though. Page 187 of the book tries to show more or less this point, without the statistics.

    Morag Kerr.

    1. But, of course, we don't know that it did happen (i.e. the theory that "Megrahi did it"). We can only try to work out from the evidence whether the theory that it did happen is the superior theory. Unless I'm completely wrong, the probability of Megrahi's guilt is miniscule, and something else must have happened. I'm not sure what sort of percentage probability would constitute beyond-a-reasonable-doubt, maybe 99% ?

      Do you know which "location" in the Kindle version you're referring to?!

    2. I'm talking about two-thirds of the way through chapter 10. Paragraph beginning "The investigators didn’t seem to realise quite what an odd plot they were proposing." It's not quite what you're presenting, but it's where I ended up from a similar position.

      When I say "you can't disprove what actually happened by saying it's improbable" I'm of course paraphrasing the smartarses. The problem is of course that sometimes very improbable things really do happen, and they point to these and say, see, what do you say to that? Very improbable, but you can't deny it happened. Maybe I'm couching my counterarguments in the wrong way.

      A case in point is the Selby rail crash.

      What was the probability that Gary Hart would have fallen asleep at precisely the moment when this would cause his Land Rover to run straight on to the railway line? A moment earlier and he wouldn't have gone down the embankment, a moment later and he'd have hit the safety barrier (which was just a metre or two too short).

      What was the probability that a train would be coming along the line where the Land Rover ended up at exactly that moment? A few minutes earlier and it would already have passed, a few minutes later and Mr. Hart might have been able to contact the signal box and alert the driver.

      What was the probability that this passenger train would derail to the right rather than to the left, thus putting it in the path of a train coming the other way?

      What was the probability of another train (a goods train in that instance) coming along the other track at precisely the right moment to hit the passenger train as it derailed into its path? A few seconds earlier and it would have passed, a few minutes later and it might have been stopped.

      And yet all that happened. Without the second train it would probably have been a fairly minor accident, even after the passenger train hit the Land Rover and derailed. With the second train, it was carnage.

      To the nay-sayers, someone arguing Bayesian logic against Megrahi's conviction is in the position of someone arguing that the Selby rail crash didn't happen on the grounds that the multiplication of these probabilities produces a minuscule probability of the actual event. It's a misunderstanding, but it's a difficult one to counter, especially when debating with someone who has a high trust level for authority and a disdain for "conspiracy theories".

    3. Ah, I see. You make a different but similar argument there, arguing that if Megrahi did commit the crime, then it was very improbable that he would have done it in exactly that way. You're right, that works by the same principles.

      I think the crash is a different situation though. We know the outcome occurred. We need to know why. If the theory proposed is inherently improbable, then excellent evidence is needed to prove it.

      I think the way to argue here is to show as I did that each piece of evidence relied on has an innocent alternative explanation. And that the correct way to combine them is to show that since each separate piece was unlikely to have been created by a bomb-plot of Megrahi's, the likelihood of all the evidence together being created by Megrahi is not greater for having so many pieces of evidence but massively smaller for multiplying their unlikelihoods together.

    4. Indeed, I quite agree. I respond merely from the perspective of having been round this one with a bunch of people who believe that any proposal that a convicted person has suffered a miscarriage of justice can be summarily dismissed on the grounds that "that's a conspiracy theory" (and so, a priori, false).

      Their reply merely boils down to the assertion that simply showing that something is unlikely doesn't disprove it. If it's even faintly possible, it might have happened, therefore if a court said that's what happened then it must have done.

      So, I managed to prove that it didn't happen.

      It's funny, really. These are the same people who will say, if you hear hoofbeats, think horses, not zebras - and certainly not unicorns! But if a court declared that it was a herd of zebras, they will accept that unquestioningly, and castigate anyone who suggests there is a much more plausible, pony-shaped explanation.

    5. I suppose such people do not realise how common miscarriages of justice are.

    6. They appear to assume that everyone freed on appeal was really guilty, but got lucky because of a technicality, or because the standard of proof wasn't quite achieved. Many of them seem to suspect that everyone who is charged is guilty!

      Having said that, one Australian chap who read the book then messaged me to apologise for his earlier behaviour in that respect, as seeing it all set out as one complete presentation convinced him.


Please make your comment evidence-based and polite.