We must interpret hospital mortality rates with caution, whilst acknowledging their potential importance in catalyzing care transformation, writes Dr. Padhraig Ryan, Trinity College Dublin & Harvard University.
Poor quality healthcare imposes a vast burden of avoidable morbidity and death in health systems around the world. In the United States, for example, around 90,000 patients die from avoidable mistakes in hospitals each year. Key reasons include medication errors, infection, preventable blood clots, and injury during surgery. In Irish hospitals, death rates from mistakes are probably significant, but we have yet to systematically assess the scale of harm.
The Hospital Standardized Mortality Ratio (HSMR) compares death rates between hospitals, using statistical techniques to account for patients’ varying risk of death. HSMR can ideally detect poor quality, trigger investigation of the underlying cause (known as root cause analysis), and catalyze care redesign. This should foster clinicians’ will to refine care processes and thereby safeguard patients.
Public reporting of HSMR is an intuitively appealing proposal for policy makers. Don’t patients have the right to know which hospitals offer safer care? And isn’t transparency a key ingredient for improvement? Yes, however the HSMR has limited ability to achieve these goals. This article explores challenges associated with the HSMR, and proposes some general steps for monitoring clinical performance.
False reassurance, false alarms
HSMR may fail to detect 10 of every 11 poorly performing hospitals. Only around 6% of hospital deaths are preventable, hence the key “signal” of preventable deaths is easily drowned by background variation in unavoidable deaths. Therefore HSMR may offer false reassurance about patient safety.
Quality lapses may be pervasive in Irish healthcare, due to historic under-investment in the intellectual and technical infrastructure for quality improvement, as well as recent cutbacks to hospital services.
Conversely, around 10 of 11 alarms triggered by HSMR may be false. Hospital death rates depend on the characteristics of patients, for example elderly and seriously ill patients are more likely to die. Statistical models are only partly successful in adjusting for this. Numerous proprietary HSMR tools are available, and hospital rankings fluctuate depending on which tool is used.
Variation in coding practices can dramatically affect HSMR rankings and obscure quality defects. Poor quality resulted in many tragic deaths in Mid Staffordshire Trust in England, and although HSMR initially illuminated abnormally high death rates, its ranking surpassed the UK average following changes in coding.
The volatility of HSMR rankings is attributable to multiple factors. First, Mid Staffs recorded only 1% of admissions as palliative care compared to a UK average of 9%, but took explicit steps to increase this after a review of coding practices. Second, the hospital’s “coding depth” (diagnoses per episode) was below the UK average, but the novel coding practices increased this. The availability of nursing homes and hospice beds to enable discharge can also influence hospital mortality rates and distort HSMR rankings, although apparently not in this instance. To avert the latter problem, the HSMR can include deaths in any setting within a defined period after hospital admission.
So what next?
Quality lapses may be pervasive in Irish healthcare, due to historic under-investment in the intellectual and technical infrastructure for quality improvement, as well as recent cutbacks to hospital services. Despite HSMR’s limitations it makes sense to analyze mortality rates, and to galvanize clinicians for care redesign. But the wellbeing of Irish patients urgently demands more sophisticated clinical performance measures. HSMR should form part of a broader measurement strategy covering the spectrum of outcomes that matter to patients, including functional status and quality of life.
We must interpret hospital mortality rates with caution, whilst acknowledging their potential importance in catalyzing care transformation. If the benefits of greater focus on quality outweigh the harm of false reassurance and false alarms, then public reporting of HSMR is worthwhile.
References available on request.
