Hancock admits data error on test and trace programme
Mistakes happen of course, particularly when you are dealing with an evolving global pandemic, but we were surprised to hear the Department of Health was relying on Excel software to migrate public data to its mainframe system.
Government departments like most organisations suffer from information systems that were never designed to match seamlessly with other software or platforms.
Edward Tufte, the leading statistician and professor of computer science at Yale, once noted that errors were usually the result of unrealistic expectations on behalf of the technicians rather than attributes of the data itself.
A decent data scientist will tell you what they can achieve with the data in its native format without expensive, or in this case, unsuitable refits. Even elementary machine learning tools, which gather data from multiple layers through keyword searches, time records and other techniques, will produce reliable information you can use instantly. It’s a mystery why Excel was thought to be an option in this context.