According to a seminal 2015 report on diagnostic error by the Institute of Medicine,
“It is likely that most of us will experience at least one diagnostic error in our lifetime, sometimes with devastating consequences.”
Indeed, studies on diagnostic error in the U.S. have suggested overall misdiagnosis rates range from 5% to 15%, and are sometimes as high as 97% for certain diseases like celiac disease.
Misdiagnosis results in poor patient outcomes and excessive healthcare cost. A Johns Hopkins study concluded it is also the single largest cause of medical malpractice lawsuits, accounting for 35.2% of payouts and an estimated 80,000 to 160,000 potentially avoidable deaths and significant permanent injuries each year. The misdiagnoses stem from delays or failure to treat an underlying disease, or from treating diseases that are not actually present. That would make diagnostic error the third leading cause of death in the U.S. after heart disease and cancer. A study published in BMJ reports that approximately half of all diagnostic errors with adult outpatients are potentially harmful.
An inconvenient truth
If diagnostic error is such a big deal, why don’t we hear more about it?
“Overall, diagnostic errors have been underappreciated and under-recognized because they’re difficult to measure and keep track of owing to the frequent gap between the time the error occurs and when it’s detected,”
says Dr. David Newman-Toker, leader of the Johns Hopkins study. He goes on to say, “These are frequent problems that have played second fiddle to medical and surgical errors, which are evident more immediately.”
Indeed, accurate data on diagnostic error rates is exceedingly hard to come by, so no one knows the actual number for sure. Even the definition of diagnostic error has been the subject of much discussion amongst those who follow the topic. Nevertheless, physicians I’ve spoken with agree that it is much more common than the public realizes.
In addition, Newman-Toker suggests that experts have downplayed the scope of diagnostic errors because they are afraid to open a can of worms they can’t close. He says,
“Progress has been made confronting other types of patient harm, but there’s probably not going to be a magic-bullet solution for diagnostic errors because they are more complex and diverse than other patient safety issues.”
It should also be noted that, traditionally, physicians and other clinicians have been reluctant to report their colleagues’ diagnostic errors when noticed due to a culture of mutual support. They’ve also been reluctant to self-report mistakes in fear of malpractice liability. Similarly, hospitals and other provider organizations have an incentive to protect their reputation and finances by not drawing attention to diagnostic errors. It’s also natural to believe that while such errors may be more common than generally appreciated, the problem lies with others.
What is diagnostic error?
Diagnostic error can be thought of as a failure to accurately identify and communicate on a timely basis the cause of a patient’s health problems. Not surprisingly, effective treatment plans—from both patient outcome and cost perspectives—depend largely on accurate and timely diagnoses.
The Society to Improve Diagnosis in Medicine suggests that there are three subcategories of diagnostic error:
- Wrong diagnosis. An example would be when a patient who is having a heart attack is diagnosed as having acid indigestion. The accurate diagnosis may be identified from a post-mortem study. This, of course, isn’t always done. And when it is, the data isn’t necessarily flagged and reported as diagnostic error, making actual data hard to quantify.
- Delayed diagnosis. The most common is cancer. If it isn’t detected early, it usually becomes much harder to treat. However, many illnesses aren’t suspected until symptoms persist or worsen.
- Missed diagnosis. Sometimes, the causes of patients’ complaints are simply not identified. Examples might include chronic headaches or fatigue or loss of appetite.
Why do we have so many diagnostic errors?
The causes of diagnostic error are many, and can be attributed to cognitive issues driven by the complexity of diagnosis and to system issues driven by the way healthcare is delivered.
With almost 70,000 diseases represented by ICD-10-CM diagnosis codes, but less than 200 presenting symptoms, it is impossible for any physician to know about every disease and its associated signs and symptoms. Even with clinical decision support tools and access to expert databases, getting to the right diagnosis requires a strong foundation in the diagnostic reasoning process—what many would describe as the “Sherlock Holmes part of medicine.”
It’s critical to understand why one would ask certain patient history questions, conduct certain physical exams, consider certain diseases (“differential diagnoses”), and order certain tests from thousands available to efficiently evaluate the most likely or most harmful diagnoses. Of course, one must accurately interpret those findings to get to the right diagnosis so the right treatment can be provided.
To err is human, and the potential for cognitive error is significant when time and cost pressures make it difficult to be as comprehensive as one would like. A physician may resort to pattern matching—perhaps, he saw this combination of symptoms in a patient recently and quickly concludes it is, therefore, the same disease. While efficient in time, the diagnosis may, in fact, not be the same. He must, therefore, know when it’s critical to investigate other possibilities. Or he may have failed to ask questions, conduct physical exams, or order tests that might have been pivotal in narrowing the differential diagnoses. Adding to the challenge, the patient may not have accurately described her symptoms. Any of these issues can lead to premature—and incorrect—closure.
Less experienced physicians may exclude an important hypothesis simply because the patient doesn’t exhibit all the classical symptoms. Yet, patients seldom exhibit all “classic” signs and symptoms. Learning time and cost naturally limit the variety of patient encounters and presentation during apprenticeship training and residency.
System infrastructure and process issues also contribute to the problem. Critical diagnostic tests must be available and affordable. Data must be accurate and results must be communicated on a timely basis. Operational, communications, or collaboration breakdowns might get in the way. Specialists must be available when consultations are necessary.
As clinician shortages and value-based reimbursement put pressure on the length of time available for appointments, adequate patient-provider face time for the critical patient history and physical exam becomes increasingly difficult to obtain. Inadequacies in these areas are present in 56% and 47% of missed diagnoses, respectively. While electronic medical records provide an important repository of information that helps make collaboration and follow-up more effective, the data entry and other paperwork, which can take 50% of a physician’s time, makes it even harder to get adequate patient-provider face time.
While any one of the cognitive or systems issues can lead to diagnostic error, research has shown that oftentimes multiple issues come into play.
Outlook
The first step in addressing any problem is to acknowledge that the problem exists. While it has been difficult to accurately quantify the scope of our nation’s diagnostic error rate, there is growing recognition that it is a large issue that must be dealt with. Patient safety, of course, is paramount. And, it is core to the Hippocratic oath.
Leading researchers on diagnostic accuracy like Dr. David Newman-Toker of Johns Hopkins, Dr. Hardeep Singh of Houston Veterans Affairs Center for Innovations in Quality, Effectiveness, and Safety, Dr. Mark Graber of the Society to Improve Diagnosis in Medicine, and many others have long reported on the estimated magnitude and causes of diagnostic error. The Institute of Medicine report was an important validation of their work and has brought even greater national attention to the issue. More recently, the Agency for Healthcare Research and Quality announced the availability of grant funding to understand and improve diagnostic safety and Johns Hopkins announced a new Center for Diagnostic Excellence.
But perhaps the greatest catalyst for addressing diagnostic error on a broad scale will be the rapid shift in healthcare from a volume driven fee-for-service reimbursement system to an outcomes and cost-effectiveness driven value-based reimbursement system. With this new payment environment, diagnostic error rates will directly impact provider revenue and financial stability.
No provider organization wants to waste money on treating the wrong disease while the real disease escalates to the point where it is much costlier to deal with or the patient dies—both of which are bad under value-based reimbursement. And provider organizations are looking at ways to leverage more nurse practitioners and physician assistants, who may be more available and less expensive than physicians, but also have less training.
Thus, health systems are getting serious about diagnostic accuracy. They will experiment with real-time clinical decision support tools and artificial intelligence systems that prompt the clinician to consider certain diagnoses. And they will make sure that they utilize advances in broadly scalable online simulation technology to evaluate their new and experienced clinical staff on diagnostic reasoning, and to remediate or provide continuing education as appropriate.