Artificial Intelligence Failure at IBM `Watson for Oncology`

Artificial Intelligence Failure at IBM `Watson for Oncology`
Case Code: ITSY126
Case Length: 24 Pages
Period: 2012-2021
Pub Date: 2022
Teaching Note: Available
Price: Rs.400
Organization: International Business Machines Corporation
Industry: Technology & Communications
Countries: United States
Themes: Artificial Intelligence, Machine Learning, Technology Development,Technology in Healthcare
Artificial Intelligence Failure at IBM `Watson for Oncology`
Abstract Case Intro 1 Case Intro 2 Excerpts

Abstract

The case discusses the artificial intelligence (AI) failure at American multinational technology company International Business Machines’ (IBM) AI software Watson for Oncology (Watson). In 2012, IBM partnered with New York-based cancer treatment center and research organization Memorial Sloan Kettering Cancer Center (MSK) to develop Watson as AI software that could provide medical professionals with improved access to up-to-date and comprehensive cancer data and practices regardless of where a patient was living or the physician was practising. In addition to this, in 2013, IBM partnered with University of Texas MD Anderson Cancer Center (MD Anderson) to develop an AI tool called Oncology Expert Advisor (OEA) with the stated mission of eradicating cancer.

IBM touted Watson as a project that could be better at diagnosing cancer and giving treatment recommendations to patients than human doctors due to its cognitive computing ability through which it could stay current with the huge volume of new findings published every day on cancer research and care. Despite Watson’s Natural Language Processing (NLP) ability that enabled it to read and gain insights from unstructured data such as doctors’ notes, clinical summaries, etc., it was not able to interpret data as human doctors could. A 2017 investigation carried out on Watson by news website STAT revealed how IBM’s AI software could not live up to the hype created around it by the company. STAT reported that internal documents from IBM revealed that Watson had recommended ‘unsafe and incorrect’ cancer treatments. There were no reports of patients being harmed since the recommendations were not used on real cancer patients. But the documents obtained by STAT showed that doctors who tried using Watson to help them design treatment complained that the AI software was not ready for practicing medicine.

To add to IBM’s troubles, a 2017 audit carried out by the University of Texas showed that MD Anderson was using old data to train its OEA. The same year, i.e. in 2017, the cancer center closed down its project with IBM after spending US$62 million, attributing the closure to delays, cost overruns, procurement problems, and IBM’s Research Team shifting the focus of the project from one form of cancer to another. Analysts at American market research company Forrester Research Inc. (Forrester) revealed that data quality was among the biggest challenges faced by AI projects, which often led to failures.

In February 2021, American business-focused international newspaper the Wall Street Journal (WSJ) reported that IBM was considering selling Watson Health since the business was not profitable. Experts felt that AI tools in healthcare lacked the ability to diagnose a patient like a physician could and it would require huge amounts of information to treat patients especially cancer, which was complex and unique to every patient.

With data quality and domain expertise being some of the major reasons for the failure of AI projects, is it worth it for companies such as IBM to invest in AI and Machine Learning (ML) tools in healthcare like Watson for Oncology without proper data preparation? What are the roadblocks standing in the way of IBM finding success in using AI and ML in a bid to drive clinical research and drug discovery? Going forward, how should technology companies make AI and ML a truly transformative force in healthcare?

Issues

The case is structured to achieve the following teaching objectives:

  • Understand how companies can use AI tools in the field of healthcare.
  • Employ strategies to enable companies to make AI and ML a truly transformative force in healthcare.
  • Understand the limitations of AI and analyze the challenges that organizations have to confront when they integrate AI software in their businesses.
  • Understand the causes that led to AI failure at IBM’s Watson for Oncology
  • Discuss why Watson was successful in verticals other than healthcare
  • Explore strategies on how IBM can successfully use AI in healthcare

Contents

Keywords

International Business Machines Corporation; IBM Watson Health; Memorial Sloan Kettering Cancer Center; University of Texas MD Anderson Cancer Center; Jeopardy!; DeepBlue; DeepQA; Oncology Expert Advisor; advanced analytics; cloud computing infrastructure; clinical research; drug discovery; genomic sequencing capabilities; data quality; bioinformatics tool

Buy this case study (Please select any one of the payment options)

Price: Rs.400
 Price: Rs.400
 PayPal (9 USD)

Custom Search