This graphic on research related to machine learning algorithms to track or treat Covid-19 shows the pitfalls of relying on AI. (Source: MedRxiv)

UK Report Shows Lack of Impact from Array of AI Tools Deployed to Catch COVID Cases

The conclusion of multiple studies published in the past couple of months was capped off by one from the Turing Institute, the UK’s national center for data science and AI. In summary, the tools hardly made an impact in fighting against Covid.

Laure Wynants, an epidemiologist at Maastricht University in the Netherlands, who studies predictive tools, wrote a review in the British Medical Journal that is still being updated as new tools are released and existing ones tested. Of 232 algorithms used for diagnosing patients or predicting how sick those with the disease might get, none worked for clinical use, and only two are seen as having enough potential for future testing.

“It’s shocking,” says Wynants. “I went into it with some worries, but this exceeded my fears.”

Derek Driggs, a machine-learning researcher at the University of Cambridge, and his colleagues, and published in Nature Machine Intelligence about how deep-learning models for diagnosing covid and predicting patient risk from medical images, such as chest x-rays and chest computer tomography (CT) scans, also failed. They examined 415 published tools.

“This pandemic was a big test for AI and medicine,” says Driggs, who is himself working on a machine-learning tool to help doctors during the pandemic. “It would have gone a long way to getting the public on our side,” he says. “But I don’t think we passed that test.”

The failures came from using poor quality data, according to all of the reports.

“Information about covid patients, including medical scans, was collected and shared in the middle of a global pandemic, often by the doctors struggling to treat those patients. Researchers wanted to help quickly, and these were the only public data sets available. But this meant that many tools were built using mislabeled data or data from unknown sources.”

The World Health Organization is considering an emergency data-sharing contract that would activate during international health crises, letting researchers move data across borders more easily, says Mateen. Before the G7 summit in the UK in June, leading scientific groups from participating nations also called for “data readiness” in preparation for future health emergencies.

read more at technologyreview.com