American Heart Journal
Volume 152, Issue 3 , Pages 410-413, September 2006

Reporting for provider performance: Should we punish the bad, or try to make them all good?

  • T. Bruce Ferguson Jr, MD

      Affiliations

    • Corresponding Author InformationReprint requests: T. Bruce Ferguson Jr, MD, East Carolina Heart Institute, Division of CT & Vascular Surgery, 600 Moye Blvd., Greenville, NC 27858-4354.

Department of Surgery, LSU Health Sciences Center, New Orleans, LA

Department of Physiology, LSU Health Sciences Center, New Orleans, LA

LSU Cardiovascular Outcomes Research Group, LSU Health Sciences Center, New Orleans, LA

Received 12 December 2005; accepted 22 January 2006. published online 24 June 2006.

Article Outline

 

Typically, the term quality in medicine has been used with a qualifying adjective that has in part reflected the degree to which stakeholders understood quality and its root causes in care delivery. Thus, “quality assurance” and “quality measurement” assumed a level of quality of care that was dispelled with the 1999 IOM report “To err is human.”1 This led to developments in “quality improvement” and then “continuous quality improvement,” where important successes in improving quality in medicine have in fact been achieved over the past decade.2, 3, 4, 5 However, much work remains to be done.6

The definition for quality in health care has evolved as well, to the Agency for Healthcare Research and Quality's (AHRQ) “…doing the right thing, in the right patient, at the right time in the patient's disease process, with the best possible outcome, and which strikes the right balance of services.”7 This operational definition sets the current quality standard against which delivered care should be evaluated medically and financially going forward.

The newest terminology step in this quality evolution is “quality reporting,” recently reembraced with enthusiasm by a variety of stakeholders in both the public and private sectors. The most recent and visible example is the Centers for Medicare and Medicaid Services (CMS) initiative to begin in January, 2006 titled “Physician Voluntary Reporting Program” (PVRP). This initiative is built upon the success of the Medicare Hospital Quality initiative, where a modest set of hospital-based quality metrics for certain disease conditions were identified, and hospitals had the option of reporting the results of these metrics to CMS; only 2% of hospitals failed to participate because they understand that reporting of these metrics will become the basis for modifications to reimbursement at the hospital level going forward.8 In the PVRP Program, beginning in 2006, the reporting of a set of 36 metrics (“G-codes”) to CMS at the provider level will be in place as a voluntary program for all providers.9 In 2007, PVRP will be linked to reimbursement at the provider level, and these two efforts will firmly establish Pay for Performance based on quality reporting in the Medicare population in the United States.

To be consistent with this AHRQ quality definition as the standard for evaluation, the operative principles for quality reporting should include defining the agenda, tools, and format for this reporting (Figure 1). The tools and format metrics are generally agenda-dependent; the current agenda options are to use this reporting mechanism to either (1) eliminate the poor quality providers, thus increasing the likelihood that patients will encounter a “better quality provider;” or (2) establish the infrastructure for CQI to take place in a sustainable manner and influence all providers. We understand much about the strengths and weaknesses of these agenda options and these tools and formats as “quality reporting” comes into prime time.2, 3, 4, 5, 6, 10, 11, 12, 13

  • View full-size image.
  • Figure 1. 

    Schematic for reporting for provider performance and evaluation. Underlined tools for reporting are proposed as becoming increasingly important. Solid lines are principal associations for agenda 1; dashed lines are principal associations for agenda 2.

The article by Guru et al14 addresses in a timely manner an important component of quality reporting. They assessed the impact of changing from a private, internal mechanism for sharing coronary artery bypass graft (CABG) performance data to the public reporting (via Web site) of this same information on risk-adjusted CABG mortality at the hospital level. Importantly, the initial establishment of a provider-endorsed, private feedback reporting mechanism resulted in a significant and sustainable improvement in CABG mortality outcomes in Ontario. However, there was no incremental improvement in outcomes as a result of making these same data public via a public reporting mechanism.

One interesting question that Guru's analysis does not address is whether public reporting initially would have resulted in the same degree of quality improvement. A report from the New York State Cardiac Surgery Registry suggests that it might,15 but there has been a well-documented adverse selection consequence of this mandatory and public reporting process. Surgeons and institutions avoided some high-risk patients and denied therapy to patients who might have derived great benefit from CABG because of the public reporting of results.16 With this private feedback in a voluntarily elected program, using a combination of administrative and clinical content (containing processes and risk-adjusted administrative data), it does not appear that this adverse selection effect occurred in the Ontario experience.

The results from this analysis parallel similar improvements in quality demonstrated in the United States over the same time frame in regional, voluntary clinically driven programs such as the NNE,17 in national, voluntary clinically driven programs such as the STS National Cardiac Database,18 in the government-sponsored Quality Improvement Program in Cardiac Surgery in the VA system,19 and in a number of regional efforts based upon the infrastructure of the STS.20, 21 With the use of these same basic approaches, the mortality for adult cardiac surgery in Great Britain, Europe, and Japan have declined as well over similar time frames.22, 23 In the broader picture, are these improvements in cardiac surgery outcomes linked principally to the procedures themselves and/or to the specialty and, therefore, of little relevance to the rest of medicine?

Several observations from this study shed some light on this question. First, their experience confirms that the most critical step to be taken is to generate data (content and evidence from Figure 1) to feed back. Second, provider buy-in to the reporting and report analysis process was a seminal ingredient for improving quality. This buy-in, in turn, may depend upon the underlying Reporting Agenda24; quality improvement (in this case, reduction in CABG mortality) was the driver for this project. Thus, public reporting is not a requisite for quality improvement if a sustainable CQI infrastructure can be developed; the international experience in cardiac surgery suggests that the infrastructure is the key across a variety of health care delivery systems. In terms of content and evidence, Guru used a combination of administrative data linked to a small set of clinical variables; in addition, it is possible to take issue with the risk analysis in this study because it was based on primarily administrative data techniques.25 Some stakeholders argue that administrative data are adequate because they extend across all provider groups, whereas providers maintain that only clinical data contain the care processes and risk-adjusted outcomes necessary to identify the mechanisms whereby improvement has occurred and to target new areas for sustainable CQI. In the United States, the National Quality Forum developed and endorsed a set of cardiac surgery performance metrics based on clinical process and outcomes data26; unfortunately they are not likely to be used by either the public or private sector as they develop reporting and performance evaluation systems (including the PVRP effort), based primarily upon administrative data. This issue of data content and evidence is an unsettled and volatile one, affected by factors such as who owns/controls the data, cost of data acquisition and analysis, data quality, and the ability to risk-adjust the information and the underlying reporting agenda to which these tools are linked (Figure 1). One of the more intriguing aspects to this analysis is the degree to which this combination of administrative and clinical data was effective.

If a mix of reporting methods, incentives, content, and evidence have been associated with improvements in quality, is there an optimal combination of these factors that might be considered as the PVRP and other “pay for performance” programs in the United States gain momentum? This study from Ontario suggests that significant quality improvement can be accrued by adoption of an approach that is linked to a sustainable CQI reporting agenda. QI can be accomplished without resorting to a “competition” or “elimination” paradigm that is much less likely to engage the providers and may not provide for an ongoing mechanism to improve quality going forward. Certainly, a common thread among these issues is a significant lack of trust across and among stakeholders as this quality reporting agenda unfolds.

What are the new directions for quality reporting efforts? Because the recent significant increase in health care costs is the fundamental driver behind these new reporting initiatives, linking clinical and administrative data to financial data is a key next step; this is also necessary to meet the appropriateness and efficiency criterion of the AHRQ quality definition.8, 27 If sustainable QI is to remain an important part of this reporting process, then clinical data to which providers can claim ownership will be necessary as a concession to keep them engaged. A second concession will be an effective solution to the real and significant financial burden this reporting agenda demands from providers to keep it linked to improvements in quality. Finally, reestablishment of an appropriate level of trust between stakeholders so that the best data available are used to address specific components of the longitudinal care delivery processes is critical. For example, inhospital cardiac surgery may be best benchmarked clinically and financially with nationally benchmarked clinical data, which are then coupled with longitudinal administrative outpatient and inpatient data from government and third-party payers already contained in their administrative and financial data sets. This level of cooperation and combination of data sources will be necessary to truly benchmark the medical and financial effectiveness of care encompassed in the AHRQ definition of quality. At that point, we as a health care delivery system would be back at the point of meaningful quality assurance.

Back to Article Outline

References 

  1. Institute of Medicine . To err is human. Building a safer health system. Accessed at http://www.iom.edu/object.file/master/4/117/0.pdf[Accessed 12/11/05]
  2. Ferguson TB, Peterson ED, Coombs LP, et al. Use of continuous quality improvement to increase use of process measures in patients undergoing coronary artery bypass graft surgery: a randomized controlled trial. JAMA. 2003;290:49–56
  3. Bradley EH, Holmboe ES, Mettera JA, et al. A qualitative study of increasing beta-blocker use after myocardial infarction: why do some hospitals succeed?. JAMA. 2001;285:2604–2611
  4. Kiefe CI, Allison JJ, Williams OD, et al. Improving quality improvement using achievable benchmarks for physician feedback: a randomized controlled trial. JAMA. 2001;285:2871–2879
  5. Roe MT, Peterson ED. Improvements in quality improvement. Am Heart J. 2005;149:751–752
  6. McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348:2635–2645
  7. Agency for Healthcare Research and Quality. Department of Health and Human Services. Cited at http://www.ahrq.gov/consumer/guidetoq/guidetoq4.htm. Accessed 12/11/05.
  8. Centers for Medicare and Medicaid Services, Department of Health and Human Services. Cited at http://www.hospitalcompare.hhs.gov. Accessed 12/11/05.
  9. Pacific Business Group on Health. Advancing physician performance measurement: using administrative data to assess physician quality and efficiency. Available at http://www.pbgh.org/news/eletters/documents/PBGHP3Report_09-01-05final.pdf[Accessed 12/11/05]
  10. Dranove D, Kessler D, McClellan M, et al. Is more information better? The effects of “Report Cards” on Health Care Providers. J Polit Econ. 2003;111:555–588
  11. Marshall MN, Shekelle PG, Leatherman S, et al. The public release of performance data: what do we expect to gain? A review of the evidence. JAMA. 2000;283:1866–1874
  12. Bentley JM, Nash DM. How Pennsylvania hospitals have responded to publicly released reports on coronary artery bypass graft surgery. Joint Comm J Qual Improv. 1998;24:40–49
  13. Chassin MR. Achieving and sustaining improved quality: lesson learned from New York State and cardiac surgery. Health Aff (Millwood). 2002;21:40–51
  14. Guru V, Fremes SE, Naylor CD, et al. Public vs. Private institutional performance reporting: what is mandatory for quality improvement?. Am Heart J. 2006;152:573–578
  15. Hannan EL, Kilburn H, Racz M, et al. Improving the outcomes of coronary artery bypass surgery in New York State. JAMA. 1994;271:761–766
  16. Peterson ED, et al. The effects of New York's bypass surgery provider profiling on access to care and patient outcomes in the elderly. J Am Coll Cardiol. 1998;32:993–999
  17. O'Connor GT, Plume SK, Olmstead EM, et al. A regional intervention to improve the hospital mortality associated with coronary artery bypass graft surgery: the Northern New England Cardiovascular Disease Study Group. JAMA. 1996;275:841–846
  18. Ferguson TB, Hammill B, Peterson ED, et al. A decade of change: risk profiles and outcomes for isolated CABG procedures, 1990-99: a report from the STS National Database Committee and the Duke Clinical Research Institute. Ann Thorac Surg. 2002;73:480–490
  19. Grover FL, Shroyer AWL, Hammermeister KE, et al. A decade's experience with quality improvement in cardiac surgery using the Veterans Affairs and Society of Thoracic Surgeons National Databases. Ann Surg. 2001;234:464–474
  20. Holman WL, Allman RM, Sansom M, et al. Alabama CABG Study Group: Alabama coronary artery bypass grafting (CABG) project: results of a statewide quality improvement initiative. JAMA. 2001;285:3003–3010
  21. Arom KV, Peterson RJ, Orszulak TA, et al. Establishing and using a local/regional Cardiac Surgery Database. Ann Thorac Surg. 1997;64:1245–1249
  22. Kawachi Y, Nakashima A, Toshima Y, et al. Risk stratification analysis of operative mortality in heart and thoracic aorta surgery: comparison between Parsonnet and EuroSCORE additive model. Eur J Cardiothorac Surg. 2001;20:961–966
  23. Gogbashian A, Sedrakyan A, Treasure T. EuroSCORE: a systematic review of international performance. Eur J Cardiothorac Surg. 2004;25:695–700
  24. Grol R. Improving the quality of medical care: building bridges among professional pride, payer profit, and patient satisfaction. JAMA. 2001;286:2578–2585
  25. Grunkemeier G, Zerr KJ, Jin R. Cardiac surgery report cards: making the grade. Ann Thorac Surg. 2001;72:1845–1848
  26. The National Quality Forum Cardiac Surgery Performance Measures Project. Cited at http://www.qualityforum.org/docs/cardiac_care/webcardiacPUBLIC02-07-05.pdf. Accessed 12/11/05.
  27. Rosenthal MB, Fernandopulie R, RyuSong H, et al. Paying for quality: providers' incentives for quality improvement. Health Aff. 2004;23:127–144

PII: S0002-8703(06)00120-7

doi:10.1016/j.ahj.2006.01.017

American Heart Journal
Volume 152, Issue 3 , Pages 410-413, September 2006

Article Tools

* Image Usage & Resolution