|
|
ADF Archive Files : For Browse View Click Here
Stephen Milgate
Executive Director
Australian Doctors' Fund
25 February 2003
"In medical research, the real news is the evidence, not the public claim."
"The idea that every time there's an injury we write a rule, that just makes the world so hopelessly complex, it would probably increase injury rates."
On 1 June 1995, Australians awoke to sensational headlines declaring that up to 18,000 people were dying each year because of medical mistakes. The media quoted the authoritative source as being the Quality in Australia Health Care Study (QAHCS)1 funded by a Federal Government grant.
The QAHCS study first appeared in the newspapers and the Federal parliament before being published in a peer review journal. The strategy of those who released it was to bypass the traditional medical research publishing pathways and create a media event. This tactic upset many, but it meant that critical analysis or explanation of the study was virtually impossible since few, if any, had read it prior to the media splash.
"The high profile public release of the QAHCS preliminary information was accompanied by widespread and sensational coverage by the mass media and the medical media. The Minister's abridged version of the QAHCS data was viewed with disquiet, scepticism, frustration and even anger by a medical profession accustomed to the orderly progression of research information through peer review to comprehensive publication."
Politics and publishing: the Quality in Australian Health Care Study.
The Medical Journal of Australia, Vol 163, 6 November 1995.
Criticism of the QAHCS remained largely in house.
There were also claims that QACHS made limited references to: co-morbidities in the study's questionnaire, the problems associated with retrospectivity and the fact that the preventability assessment "was mainly based on question (4e) and was subjective" and that "the information released has not been stratified for Casemix and Illness Severity".2
Such criticisms were ignored by the media if at all presented. The study's assertions that 18,000 would have died as a result of their health care was all that mattered. After all, it was a claim being made by some very reputable and well qualified people.
A year after the QAHCS was released a virtually identical US study, Utah-Colorado Study (UTCOS) with dramatically different results for the same base year, was published by the very reputable Harvard School of Public Health.
This forced the Federal Government to commission the Harvard School of Public Health to investigate why QAHCS and UTCOS had produced such a wide discrepancy in results using identical methodology.
In response, the Harvard School of Public Health and others producedtwo papers, A comparison of iatrogenic injury studies in Australia and the United States 1: Context, methods, casemix, population, patient and hospital characteristics and A comparison of iatrogenic injury studies in Australia and America 11: Reviewer behaviour and quality of care.
These studies were embargoed and not published until 1999 when they finally appeared in the International Journal for Quality and Health Care.3
What both these studies show is that reviewer behaviour and perceptions can dramatically affect the results of any study relying on such processes to categorise adverse events.
In particular, the differences between the high adverse event rate in the QAHCS and the much lower rate in the UTCOS was in fact largely due to specific bias. It seems that the reviewers modified their reviewer behaviour to meet the objects of the study.
The publication of new information has not stopped the sensational claims of 18,000 dying each year made by QAHCS, becoming dogma in any discussion of the performance of the health care system and particularly the Australian hospital system and the Australian medical profession.
The ABC's 7.30 Report in March 2002, introduced a story on medical errors as follows:
"Last year, the national road toll was 1,756 fatalities."
"But according to the Federal Government study, so called "adverse events" in hospitals contribute to nine times that number of deaths."
"The study came up with 18,000 deaths a year and about 30,000 permanently inured from medical mistakes."
Lorrain Long, Medical Error Action Group, The Danger in Australian Hospitals,
7.30 Report, ABC TV, 4 March 2002
In February 2003, in the Australian Doctor magazine:
"There are 18,000 deaths in Australian hospitals each year because of the wrong diagnosis, and wrong treatment, and 12,000 serious adverse side effects from pharmaceuticals-and these are Commonwealth Government figures," Dr Brighthope says.
Australian Doctor, 14 February 2003, pg 26
The Australian Doctors' Fund has extensively investigated the claims made by the QAHCS. We leave it to the reader to judge for themselves where the truth lies. We can report, but only you can decide.
Stephen Milgate
Executive Director
Australian Doctors' Fund
25 February 2003
| Year | Country | Study |
| 1977 | US | Report on the Medical Insurance Feasibility Mills DH |
| 1991 | US | Harvard Medical Practice Study (HMPS) |
| 1995 | AUST | Quality in Australia Health Care Study (QAHCS) |
| 1999 | US | Cost of Medical Injuries with Utah and Colorado (UTCOS) |
| 1999 | US/AUST | A Comparison of Iatrogenic Injuries in Australia and America |
| 1999 | US/AUST | A Review of Behaviour and Quality of Care |
| 2001 | AUST | Iatrogenic Injury in Australia |
An adverse event (incident or omission) occurring up to 4 years before admission and well after discharge was included in the QAHCS.
"The Australian study has included in its calculation of frequency of adverse events all admissions associated with an adverse event, including those in which an adverse event occurred more than 4 years before admission or was not discovered until some time after discharge."
Taskforce on Quality in Australian Health Care – Interim Report, September 1995, Pg 9
Although it used almost identical methodology to the US HMPS, the QAHCS produced dramatically different results.
"It has been estimated by the Consortium that exclusion of adverse events that would have been excluded by the Harvard Study would reduce the calculated frequency of adverse events in the Australian study by 20% to 25%."
Taskforce on Quality in Australian Health Care – Interim Report, September 1995, pg 9.
"QAHCS and HMPS used virtually identical methodology in their data collections, namely retrospective sampling from hospital records, with and initial screening of records followed up by physicians checking marked records for evidence of AEs according to a clearly defined set of criteria."
"1. The definition of adverse event was virtually identical across studies. Both the HMPS and QAHCS research teams used the same six-point scale to indicate confidence that the cause of the injury was medical (or health care) management rather than disease process. However, whereas HMPS used a confidence score of greater than or equal to 4 as the criterion for an AE, QAHCS accepted a score of greater than or equal to 2. Thus, events that would not have been considered adverse by the Harvard Group would have been considered so by the Australian Group."
A comparison of the Harvard Medical Practice Study (HMPS) and the Quality in Australian Health Care Study (QAHCS): Final Report to the Royal Australasian College of Surgeons. Charles M et al, Department of Psychology, University of Sydney. December 2000.
Many of the deaths and disabilities in the QAHCS were not caused by adverse events and would have occurred anyway.
"A qualification to these disturbing figures is that, in a proportion of these cases, the seriousness of the patient's underlying condition means they may have died or become severely disabled during the admission, even if the adverse event had not occurred."
Dr Carmen Lawrence, Federal Minister for Human Services and Health,
Commonwealth Hansard, 1 June 1995.
In 1996 an American study (UTCOS) using the same base year (1992) to QAHCS and similar methodology estimated the rate of adverse events at being five-times less than the Australia QAHCS.
"By mid-1996 questions had been raised with respect to the validity of the findings of the QAHCS, as the rate of adverse events in Australia was apparently five-times that estimated by the Utah-Colorado Study (UTCOS) in the United States of America, a study from the same year, 1992, using similar methodology."
Iatrogenic Injury in Australia, October 2001, pg 4.
"By coincidence, a study using ostensibly the same methodology and sample size as the QAHCS has been carried out in Colorado and Utah (nurse screening of 15,000 records followed by medical review) (UTCOS), and had found an AE rate of 2.9% (3.4% unweighted) Both QACHS and UTCOS had defined an adverse event as any event leading to unintended harm or suffering which had resulted in admission to hospital, prolongation of hospital stay, significant disability at discharge or death."
"In any event, when the methodologies of the two studies were aligned, the QAHCS adverse event rate remained about three times greater than the US rate (10.6 versus 3.2%, or 16.6% versus 5.4%, a ratio of 3.3:1)."
A comparison of iatrogenic injury studies in Australia and America 11:
Reviewer behaviours and quality of care.
In 1996 the Commonwealth Department of Health commenced dialogue with the Harvard School of Public Health to have the discrepancies between the UTCOS study and the QAHCS investigated and explained. Two confidential studies were subsequently commissioned by the Australian Federal Government, namely:
"A comparison of iatrogenic injury studies in Australia and the United States 1: Context, methods, casemix, population, patient and hospital characteristics." and
"A comparison of iatrogenic injury studies in Australia and America 11: Reviewer behaviour and quality of care."
Both these studies (which are still not widely referred to in Australia) were subsequently published in 1999 in the International Journal for Quality in Health Care4, 4 years after the QAHCS had appeared.
In 1997 Dr Ross Wilson, an author of the QAHCS, admitted on ABC Radio that many of the patients who died as a result of the adverse event were generally elderly or frail or had advanced disease and that there death may have well be inevitable.
"Well for half of them, it was relatively minor disability in that it was fully resolved within 30 days. But for a small percentage, that health care did prematurely cause those patients to die. Now that group of patients were generally elderly and frail, or who had otherwise advance disease. But it is fair to say that their health care on that particular day did accelerate their death. Their death may well have been inevitable, but some of those were preventable."
Dr Ross Wilson, Quality in Australian Health Care, The Health Report,
Radio National, 7 July 1997.
Dr Wilson's statement stands in marked contrast to the findings published in the QAHCS:
"18,000 would have died as a result of their health care,"
QAHCS, MJA Vol 163, 6 November 1995
In 1999 with the publishing of the two studies commissioned by the Australian Federal Government, and undertaken by the Harvard School of Public Health and others, concluded that the very high incidents of adverse events in the QAHCS study, compared with a much lower incidents of the UTCOS study was primarily due to specific biases.
"However, through analyses using natural categories of adverse events and of indicators of severity, it has been possible to detect specific biases consistent with the disparate underlying objectives of the two studies." [UTCOS and QAHCS]
"The differences between the studies identified about are consistent with the contrasting goals of each study. QAHCS sought to measure the impact of medical injury on the health care system by estimating the prevalence of patient injury and total impact of adverse events on admissions and costs. It also aimed to generate information to support quality improvements through the prevention of adverse events."
"Also, Australian reviewers knew that the study was intended to identify the frequency and nature of all adverse events, in order to estimate the burden thy placed on the system, whereas US reviewers knew that exposure to litigation and claims for compensation were being assessed, which may have led to different perception as to what was considered to be an "adverse event" in the two studies."
A comparison of iatrogenic injury studies in Australia and America 11:
Reviewer behaviours and quality of care.
The existence of co-morbidities was given little, if any, consideration in the QAHCS.
"There was limited reference to co-morbidities in the questionnaire."
Brian T Collopy, Commentary on Australian Hospital Care Study and the Ministers Early Release of Information, 8 June 1995.
"In the QAHCS nearly 40% of all adverse outcomes, and over three-quarters of all potentially preventable deaths occurred among patients over 65 years old. The chance of an adverse event resulting in death in this age group was over ten-fold that of the under 45 age group."
WB Runciman, Iatrogenic Injury in Australia. A report prepared by the Australian Patient Safety Foundation, October 2001, pg 17.
"The extent to which a problem may be attributed to iatrogenic injury is often debatable. In the case of the 14,000 deaths for example, it is known that the patient died and that there was an iatrogenic event. The extent to which the death was caused by the iatrogenic event compared with the contribution of their causes such as the critical nature of the patient's condition has not been determined and is the subject of further study."
WB Runciman, Iatrogenic Injury in Australia. A report prepared by the Australian Patient Safety Foundation, October 2001, pg 23.
Most adverse events referred to are systems problems and not the failings of an individual clinician.
"Brent James: Injuries, Those were injuries, those 3,996. The fascinating thing was the overlap. Among 3,996 confirmed injuries, 138 or 3.5% resulted because of a human error."
Minimising Harm to Patients in Hospital. Broadcast Monday 1 October 2001. Radio National. With Dr Brent James, Executive Director of Intermountain Health Care in Salt Lake City, Utah.
"Most problems result from a sequence of system failures rather than a single mistake by an individual."
Data for Action, A key to safer health care, Safety and Quality Council, 1/8/01.
"The causes of iatrogenic injury appear to be systemic. The remarkable constancy of pattern across the Australian and US health care systems for serious injuries bears witness to the fact that despite all of the differences in structure, training and practice, similar patterns of iatrogenic injury are observed."
Iatrogenic Injury in Australia. A report prepared by the Australian Patient Safety Foundation, WB Runciman, October 2001, p 106
"I believe the system is much more often responsible for problems than individual practitioners."
Dr Ross Wilson, Radio National ABC, 7 July 1997.
Most doctors do not come into contact with the majority of incidents classified as adverse events.
"It is evident that only half of the adverse events in such a hospital would be encountered more than 10 times per year, with individual doctors being exposed to a very small number in each category; furthermore a claims file would be opened for only about 2% of these events. This low individual exposure may go some way to explaining why most clinicians were very surprised at the scope and cost of adverse events when the figures were extrapolated to provide national estimates."
"…only one-quarter of all adverse events occur with sufficient frequency to be amendable to tracking, even in large teaching hospitals."
"About one in ten admissions to acute-care hospitals in Australia is associated with a preventable adverse event-half of these occur before and half during the admission."
"Only one-quarter of all events fall into categories which occur sufficiently frequently to be tracked, even in large hospitals-to gain a proper understanding of the nature of the remainder, a national database is required."
WB Runciman, Iatrogenic Injury in Australia. A report prepared by the Australian Patient Safety Foundation, October 2001, pg 18-19.
If a three-month prognosis is included in a study of adverse events the results change dramatically.
"However, after considering 3-month prognosis and adjusting for the variability and skewness of reviewers' ratings, clinicians estimated that only 0.5% (95% CI, 0.3%-0.7%) of patients who died would have lived 3 months or more in good cognitive health if care had been optimal, representing roughly 1 patient per 10,000 admissions to the study hospitals."
"Conclusions: Medical errors are a major concern regardless of patients' life expectancies, but our study suggests that previous interpretations of medical error statistics are probably misleading. Our data place the estimates of preventable deaths in context, pointing out the limitations of this means of identifying medical errors and assessing their potential implications for patient outcomes."
Estimating Hospital Deaths Due to Medical Errors, Preventability is in the Eye of the Reviewer, Australian Medical Journal, 25 July 2001.
The distinction between whether an adverse event co-existed, contributed or caused the death of a patient has been conveniently blurred.
"Some data sources identify deaths associated with an adverse event, while others only record deaths where the adverse event was the cause of death or contributed to the death."
Media Release. Data for Action: A key to safer health care.
Safety and Quality Council. 8 August 2001.
The number of deaths from adverse events are in the eye of the beholder.
"A medical victims group claims that between 1990 and 1995 there were 90,000 deaths from medical mistakes and that the medical profession only acknowledged 346 of them, whilst a US patient safety group claims that 105 million Americans (40% of US population) have suffered or know someone who has suffered an adverse event."
Sunday Telegraph, 24 June 2001
"ABS data suggest 88.5 deaths per year can be attributed to adverse events as a direct underlying cause of death, but this increases to 2,678 deaths per year if you count where an adverse event may have contributed to their death. On the other hand, extrapolation of coronial data suggests approximately 700 patients may suffer an adverse event that contributes to their death each year, while the results of the Quality in Australian Health Care Study suggest a range between 8,600 and 18,000 deaths per year."
Media Release. Data for Action: A key to safer health care.
Safety and Quality Council. 8 August 2001.
"The figure most often quoted by the media is from the Quality of Australian Health Care Study, which reported and adverse event rate of 16.6 per cent associated with hospital admissions. However, reanalysis of the study following the methods of a similar study in the US found that the Australian and US studies had a virtually identical rate of serious adverse events – about 2 per cent of cases (1.7 per cent leading to serious disability and 0.3 per cent to death). It is thought that overall, about 10 per cent of hospitals admissions in Australia and other developed countries are likely to be associated with an adverse event. Most of these are simple problems."
First National Report on Patient Safety. Safety & Quality Council. August 2001.
Claimed Deaths from Adverse Events in Australia per annum
| ABS (a) |
Coroner (b) |
Medical Error Action Group (c) |
QAHCS (AUS) (d) |
NQSC Report (AUS) (e) |
| 88.5 | 53 (1994) |
90,000 (1990-1995) 18,000 pa |
18,000 (1992) |
177 (1997-98) 85.5 pa |
An examination of real time adverse events in hospitals in the US reveals the likely source of the underlying patterns of adverse events which exist almost uniformly across the system.
Minimising Harm to Patients in Hospital. Broadcast Monday 1 October 2001. Radio National. With Dr Brent James, Executive Director of Intermountain Health Care in Salt Lake City, Utah.
"Brent James: One of the first things you have to do is you have to prioritise. It turns out that some sources of injury are much, much more common than other sources of injury. For example, my current Big Six list, this is based upon expert opinion, so it's probably going to get changed. Top of the list is adverse drug events, drug reactions. Second is hospital acquired infections. Third is pressure sores. Fourth is something called Venous thromboembolism, or blood in the body spontaneously clots and can cause damage when it blocks blood vessels. Fifth on the list is patient falls and injuries, usually associated with the use of restraints, either physical restraints, tying a patient in a bed, or chemical restraints where you sedate them. Number six on my list is blood product transfusions, inappropriate blood product transfusions."
Minimising Harm to Patients in Hospital. Broadcast Monday 1 October 2001. Radio National. With Dr Brent James, Executive Director of Intermountain Health Care in Salt Lake City, Utah.
In the eight years since the QAHCS was first published, the priorities in improving the quality of health care and making medical treatment safer are now just being heard above the headlines of "18,000 people killed each year from medical mistakes".
Remarkably the identified problem areas have been known for many years.
Any one of them could have been nominated by any active experienced medical practitioners over the last 20 years.
There is a strong desire among all professionals in the health care system to strive for greater quality and safer care.
However, a desire and good will is not enough.
There are currently 230 million transactions between the medical system and patients each year in Australia.
Health care systems are expensive and medical intervention, particularly in the frail and elderly, is high risk and becoming riskier.
A safer health care system will certainly add costs to health care and those costs eventually have to be born by those who demand a safer and better system.
Footnotes:
1 Wilson et al.
2 Brian T Collopy, Commentary on Australian Hospital Care Study and the Ministers Early Release Information, 8 June 1995
3 The International Journal for Quality and Health Care, 1999, 12(5): 379-388.
4 International Journal of Quality and Health Care, 1999, 12(5): 379-388.
|
Australian Doctors Fund |