There’s an old Jerry Seinfeld stand-up bit where the comedian tries to puzzle through why patients seem to always assume that their physician is the best: “Every time somebody recommends a doctor, he’s always the best. ‘Oh, is he good?’ ‘Oh, he’s the best. This guy’s the best.’“ The crowd laughs knowingly; they’ve probably given that exact endorsement to friends and family. But Mr Seinfeld senses that something is askew: “They can’t all be the best,” he insists, his voice rising in comic exasperation. “There can’t be this many bests. Someone’s graduating at the bottom of these classes. Where are these doctors?”
Mr Seinfeld has a point: Not every physician can be above average, much less the best. On the contrary, common sense dictates that for every physician with a legitimate claim to superlative performance, there’s another who has fallen behind the curve. But, just because we have an intuitive sense that some physicians are better than others doesn’t mean that it’s straightforward to rank them from best to worst.
Still, that shouldn’t stop us from trying. Ranking or otherwise gauging physician quality isn’t just a way to assign bragging rights or allay Mr Seinfeld’s skepticism. Rather, performance metrics can — and should, as long as we’re suitably convinced of their validity — be used to set reimbursement levels, tailor insurance schemes, and of course, identify physicians who are falling behind.
One simple way to spot those substandard performers is to order physicians by the number of malpractice suits to which they’ve been subject. The results of this exercise are fairly dramatic. It turns out that there is a group making up 1% of all physicians who, together, account for 32% of all paid malpractice claims.1 The 1%ers are more than 46 times as likely to pay out a malpractice claim compared with everyone else. It doesn’t seem like these physicians are just having a run of bad luck, either. Not only were those data collected over the course of nearly a decade, but researchers also found that, as physicians accumulated claims, they became more and more likely to be subject to further separate lawsuits. Physicians with at least 3 paid claims were more than 3 times as likely to incur another claim compared with physicians who had only 1 claim. Some states have found these data so compelling that they have threatened to revoke the license of any physician with 3 or more claims on the books.2 Looks like we’ve found Mr Seinfeld’s elusive “bad doctors.”
Using malpractice data to single out underperforming physicians seems reasonable. Not only are paid claims data easy to assemble, but we automatically think of malpractice defendants, especially those who end up paying out settlements, as negligent and incompetent — or worse. Malpractice claims stand as prima facie evidence that a patient has been done dirty.
But neither patients nor their problems are created equal. When the federal government first sought to publicize hospital-level mortality rates in the mid1980s, it had to contend with pushback centered on the idea that the data largely failed to account for the initial severity of the patient’s illness. Risk adjustment is a crucial consideration: A poor outcome following a routine procedure on a young and healthy patient almost certainly implies a different level of culpability than if the patient had been hanging on by a thread before the surgery. Even today, claims of inadequate risk adjustment are inevitably trotted out to undercut any attempt to reduce physician performance to a single number.3
Risk adjustment is important in the malpractice arena, too. In theory, courts consider the patient’s preprocedure risks and other contextual factors before declaring a verdict. In practice, the specter of jury nullification hangs over the proceedings like the sword of Damocles.4 Even in cases where the judge and jury follow the letter of the law, attempts at risk stratification are often ad hoc rather than systematic and frequently boil down to a battle of the experts. And that’s only if the claim goes to court at all. Quite often, the physician defendant will rationally conclude that the cost of going to trial is too daunting and will decide to settle instead.
Regardless of whether the decision to settle is fiscally prudent or reflective of actual culpability, it will still be recorded as a paid malpractice claim. The dominoes fall from there: Insurance companies might be expected to use it as a pretense to raise policy rates, or maybe a state review board will conclude that it’s grounds for license revocation. Anything is possible. Faced with those consequences, the rational response might be to avoid patients with challenging risk classifications or those who profile as otherwise potentially litigious at all costs5 — up to and including patients’ lives.
The crowd is rolling, but Mr Seinfeld isn’t done. “Is somewhere, someone saying to their friend, ‘You should see my doctor, he’s the worst’?” he wonders. “‘He’s the absolute worst there is. Whatever you’ve got, it’ll be worse after you see him.’“ Mr Seinfeld breaks into a grin and shakes his head in dismissal of the obviously preposterous idea. The absurdity is palpable.
He’s right again. Because as long as we’re relying on non-risk adjusted malpractice numbers, how can we possibly know who the worst is?
- Studdert DM, Bismark MM, Mello MM, Singh H, Spittal MJ. Prevalence and characteristics of physicians prone to malpractice claims. N Engl J Med. 2016;374(4):354-362.
- Associated Press. Florida Passes Three-Strikes Malpractice Law. New York Times. https://www.nytimes.com/2004/11/26/us/florida-passes-threestrikes-malpractice-law.html. November 26, 2004. Accessed May 25, 2018.
- Friedberg MW, Pronovost PJ, Shahian DM, et al. A methodological critique of the ProPublica surgeon scorecard. Rand Health Q. 2016;5(4):1.
- Marder NS. The Medical Malpractice Debate: The Jury as Scapegoat. Loy LA L Rev. 2005;38:1267.
- Shahian DM, He X, Jacobs JP, et al. Issues in quality measurement: target population, risk adjustment, and ratings. Ann Thorac Surg. 2013;96(2):718-726.