The Horizon Post Office scandal. A case study in poor software testing…?

If you haven’t read about the Post Office scandal that has wrecked so many lives and cost the UK taxpayer millions, all whilst the corporation at fault walked away scot free, then check out this great article in the Financial Times ( or Nick Wallis’s book ‘The Great Post Office Scandal’.

Somehow a faulty system designed to manage millions of pounds of post office transactions passed system testing and was deployed across the UK post office network.

So how did it pass testing? What tests were executed? Unit, component, system, functional? (According to Justice Fraser’s court case judgement ( we do know that regression testing was executed albeit with significant gaps.)

We don’t know much around the actual details on testing and very few in our testing community seem to be asking why? Was it in fact an organisational failure rather than a failure to rigorously test?

Was it due to our industries obsession and drive for a shared quality culture resulting in no one taking ownership? Was everyone responsible and no one?

Or perhaps all the various phases of testing were in fact executed perfectly – the test automation detected issues and the regression suites resulted in the logging of multiple high severity defects. A number of whistle blowers seemed to suggest that whilst the project was subject to the usual plethora of unclear requirements, slack development practices and general demoralising, high pressure work patterns the testing itself was in fact raising copious numbers of high severity and high priority defects.

Maybe despite this no one felt they had the authority to speak up? Or perhaps they did but they were ignored or overruled.

Testing as a practice has for too long been weighed down by an inferiority complex when in fact it should be the loudest voice in the room. It’s no good knowing the ins and outs of api testing or Cypress if as a software tester you not able to present a firm and cohesive case for NOT deploying a system riddle with defects.

Perhaps all of the testing was in house or the teams had ‘embedded testers’ leading to confirmation bias or were simply under pressure to deliver. Would an independent test team have helped prevent this tragedy?

Beyond Nick Wallis’s view that the dev and test teams were under enormous pressure we simply don’t know enough of the detail – it’s pure supposition and that is what is so frustrating. So why is our industry so quiet on this high profile issue? Do a quick search and from a software testing perspective you will only find a post from James Christie, a software tester and ex-auditor (which you can read here

Where are the discussion forums? Where are the explanations from the Fujitsu testers who worked on the Horizon system? Understandably they may fear for their jobs but surely some of them have moved on by now.

If we don’t know what happened then how can we learn from the mistakes made? The Post Office Scandal should be a software testing case study up there with the 1986 Space Shuttle tragedy (if you can get hold of the manuscript you will be amazed how the test engineers who warned about the defective Space Shuttle ‘O’ rings were effectively shouted down – ‘the weather is good, the pressure is on, we’re launching on Thursday’ – sounds familiar?).

We should be studying this scandal as part of our software testing certifications. It should be compulsory reading at university.

Perhaps the long and the short of it was there was no incentive to care about the results of testing. After all Fujitsu have never said sorry or been liable to pay any compensation. In fact, the company has since won a further £3.1 billion in government contracts. And that is a broader economic and societal failing that no amount of software testing can fix.