Appendix II

THE NEW YORK STATE CARDIAC HOSPITAL RATING SYSTEM

The Rating System. New York State first adopted an aggressive regulatory posture toward cardiac surgery in the 1980s, a time when revenue-hungry hospitals were proliferating new heart surgery centers, and surgical mortality rates were rising sharply. Since there was good evidence that related high volumes to good outcomes, both for centers and surgeons, a new certification process focused on closing down low-volume centers and limiting the number of new ones coming on line.

To support its certification program, in 1989 the State created a system for tracking CABG outcomes, defined as risk-adjusted mortality rates, for both centers and individual surgeons. The system was extended to valves in 1998, but because of lower volumes, reported outcomes are in three-year moving averages, so the first valve report was issued only with the 2000 data.

The risk adjustment was based on a statistical model of the entire State database that related mortality to objective case characteristics, like age, a reop, ejection fraction, renal failure, and the like. The hospitals rate their own cases, subject to selective State audits that concentrate on risk ratings that look too high. The ratings and rankings were originally supposed to be confidential, but they eventually became public, partly due to a Freedom of Information suit.

The system must be counted an almost unalloyed success, in great part due to the consistent, persistent management of Edward Hannan, a professor at the State University School of Public Health in Albany, who has been its primary designer and overseer since the beginning. It has been replicated in five other states as well as in a reporting network for hospitals in New England. All but two of the other networks also report their results publicly. A similar system was being debated in Great Britain as of March 2007.

The implementation of the report coincided with a sharp drop in New York State mortality rates. Using Medicare’s risk-adjusted CABG outcome data, New York State’s CABG mortality rate dropped 28 percent from 1989 to 1993, compared to 13 percent in the rest of the country. That advantage was maintained in subsequent years. In the 1994–1999 period, the last one for which Medicare data are available, New York State’s CABG mortality rate averaged 67 percent of the nationwide rate, a statistically significant difference.

It’s reasonable to believe that the reporting system has something to do with New York State’s excellent performance. But it’s harder to point to exactly why. It’s certainly not, as free-market theorists hoped, because consumers shifted their business to hospitals with the best ratings, for low ratings had no effect on subsequent volumes. (And consumers don’t choose their heart hospitals anyway; their cardiologists do.) Nor is there any support in the data for complaints from within the profession that ratings-wary surgeons were avoiding difficult cases, or referring them out of state. The most plausible, if probably not the entire, explanation is that published surgical ratings forced the worst performers out of the system, either through embarrassment or from quiet pressure from colleagues.

Columbia-Presbyterian and the Ratings. In Chapter 6, I recounted the sudden spike in Columbia-Presbyterian’s one-year mortality rankings in 2001. They were reflected two years later in the statewide compilations, just at the time of the Clinton surgery, drawing much critical comment from the New York Times. It clearly did not have any affect on cardiologist referrals—as it probably shouldn’t have—since it does appear to have been a one-year (actually, seven-month), so-far-unexplained blip.

But I found it interesting that Columbia-Presbyterian almost certainly understates their performance for State reporting purposes (although possible underscoring had little to do the 2001 results). I take that as another example of the academic mind-set; I can’t believe that a for-profit hospital wouldn’t pay very close attention to their numbers.

I show the CABG data from ten years of reports below. For a hospital, three different numbers are tracked. The first is the OMR, or observed (actual) mortality rate. The second is the EMR, or expected mortality rate, based on the State’s risk-adjustment formula, and the third is RAMR, or risk-adjusted mortality rate. The hospital RAMR is then compared to the statewide mortality rate.*

The formula for a hospital’s risk-adjusted mortality rate is OMR/EMR times the statewide mortality rate. For example, if the statewide rate is 4 deaths per 100 patients, the hospital’s observed mortality rate is also 4, and its expected rate is 2, the formula produces (4/2) times 4 = 8. The hospital’s risk-adjusted rate is twice as bad as the statewide average, reflecting the fact that its average case was only half as risky. On the other hand, if the hospital’s expected mortality rate is high—assume it’s 8, meaning it handles very difficult cases—then the formula would be (4/8) times 4 = 2: the hospital’s risk-adjusted outcome now shows that its performance is twice as good as the statewide average, even though its observed mortality rate is the same.

Here are the ten-year data on Columbia-Presbyterian’s CABG mortality.

In every year but two, Columbia-Presbyterian’s observed mortality was lower than the state average. But in every year except 1996, the EMR, or the difficulty of Columbia-Presbyterian’s cases, was also lower than the statewide average. In other words, Columbia-Presbyterian consistently rated itself as having easier cases than the state’s other hospitals, thereby inflating its risk-adjusted mortality rate, in some years by a substantial amount. Even with the low EMRs, however, Columbia-Presbyterian risk-adjusted mortality rate exceeded the state rate only three times.

Columbia-Presbyterian’s low-risk ratings do not seem plausible. The hospital, after all, is one where other hospitals send their hard cases, and the surgeons pride themselves, as Allan Stewart put it, in taking “the high-risk surgery, the case that’s been turned down by two other surgeons.” I looked closely at the data for 2003, which were released in 2005 when I was doing the research. In both CABGs and valves, Columbia-Presbyterian’s case difficulty rating put it in the bottom quintile of all state hospitals. Interestingly, in CABGs, all three of its New York affiliates, including one that was in a start-up mode and had restrictions on the cases it could accept, had higher difficulty rankings than Columbia-Presbyterian did. When I ran those numbers by Ed Hannan, he asked “Why would they shoot themselves in the foot like that? We don’t audit for underscoring.” I have no explanations for the low-risk ratings, except as another example of academic behavior.