In September 2013, we published “Your Safer-Surgery Survival Guide,” including our first-ever surgery Ratings, with findings on nearly 2,500 hospitals in all 50 states, Puerto Rico, and Washington, D.C.
“Our goal was not only to help patients make better choices when deciding where to go for surgery, but also to stimulate discussion in the health care community about hospital quality and the importance of making information public,” John Santa, M.D., M.P.H, medical director of Consumer Reports Health, said.
To develop our Ratings, we used the best and most comprehensive data publicly available: the billing claims hospitals submit to Medicare. “That allowed us to compare hospitals across the country on the same measure, using the same method,” Santa said.
Our report clearly hit a nerve: Readers wrote in, some praising Consumer Reports for helping them make decisions about where to get surgery. Others were concerned that hospitals that treated them weren’t given high enough Ratings. We also received e-mails and phone calls from more than 100 hospitals in 38 states—more than for any hospital report we’ve ever published. As with readers, their responses varied widely.
We heard from many grateful hospital officials, including several at community hospitals, such as Kootenai Medical Center in Coeur d'Alene, Idaho, and Enloe Medical Center in Chico, Calif. “There is a lot of great patient care being provided at community hospitals,” said Marcia Nelson, M.D., vice-president of medical affairs at Enloe. “It is very gratifying to have the recognition of an organization such as Consumer Reports, which allows hospitals to be recognized for how well they do, not on their name or how much they advertise.”
We also heard from administrators at a few high-profile academic medical centers, such as Baylor Health Care System in Texas and Hospital for Special Surgery in New York City. Some of them took issue with the method we used to rate hospitals.
Praising the results
Our surgery Ratings were based on the percentage of a hospital’s Medicare patients undergoing scheduled surgery who stay longer than expected for their procedure or die in the hospital.
“Some have criticized this as an imperfect method,” said Lisa McGiffert, director of the Consumer Report Safe Patient Project. She says that Consumer Reports has to rely on billing data, rather than clinical data, simply because that’s what’s publicly available. “We could opt to do nothing and leave patients in the dark, but we believe this data is strong and can help patients,” she said.
Apparently, many hospitals agree. Some already use the same method to track quality. And more than 50 hospitals that contacted us asked to use our Ratings in their ads. As is our policy, we had to say “no.” (Unlike other organizations that rate hospitals—or cars, toasters, or insurance plans, for that matter—we have a no-commercial-use policy. Under it, we don’t grant permission to companies to use our name or Ratings in advertisements.)
Several independent hospital safety experts also see value in our methods.
“The approach Consumer Reports uses, based on length of stay, meets the sniff test, since research clearly shows that’s related to surgical complications,” said Peter Pronovost, M.D., director of the Armstrong Institute for Patient Safety and Quality at Johns Hopkins Medicine, in Baltimore.
And while he has some concerns about our use of billing data rather than clinical data, Pronovost said he thought that when it comes to rating hospitals, “Consumer Reports may be uniquely qualified, because you don’t accept advertising and because you have been willing to make your method transparent.”
Questioning the methods
About 30 hospitals that contacted us questioned our methods and wanted more information. Because we believe in transparency, we sent them a link to our 44-page technical report (which is also posted on our website) that explains in detail what’s behind our surgery Ratings, including specific data we used to rate their institutions.
But some remained concerned. Louis A. Shapiro, president and chief executive officer of Hospital for Special Surgery, wrote that our surgery Ratings “distort the truth about the safety of specialized referral centers whose patients are often refused by other hospitals because they present with more complex conditions, require more difficult surgical procedures, and warrant longer recovery periods to be discharged safely.”
But Santa says that the method is designed to level the playing field. For example, it only includes more common, straightforward cases, not complex ones, and leaves out patients who were transferred to or from other hospitals. And it is adjusted based on the characteristics of each hospital’s patients, such as their age and sex and whether they have chronic conditions such as diabetes.
Some other administrators told us they were open to using the information to think about safety in a new way. “There is always a tendency when you get fresh data to lash out if you think they are publishing something that may or may not reflect how well you are doing,” said Robert Bayly, M.D., vice president for quality and safety at JFK Medical Center in Edison, N.J. “But we decided to start talking with the people doing the measurements, like Consumer Reports, and figure out how to do it collaboratively.”
The patient perspective
Jeffrey D. Selberg, executive vice president and chief operating officer of the nonprofit organization Institute for Healthcare Improvement, emphasized the value of involving patients in hospital safety. “Information like your ratings needs to be in front of consumers, even if it is not perfect,” he said. That, he said, can help motivate hospitals to make changes. And he says it helps patients, too. “The more patients know, the more they, along with their family members, can be vigilant when they enter any hospital,” Selberg said. “Information can lead to asking questions and also to noticing things that are either being done well or could be handled better.”
Reactions from our readers support Selberg’s viewpoint.
Michael Hayes, a Consumer Reports reader in Westminster, Colo., was pleased with our surgery Ratings, but wanted more information. “I would expect that good and bad results are heavily influenced by the surgeon as well as the hospital,” he said. “How do we find this out?”
And a subscriber, Brenda Smith of El Paso, Texas, wondered whether we “adjusted the hospital ratings for those physician hospitals who ‘cherry pick’ their patients?”
“Those are excellent questions,” Santa said. “But we know even less about individual surgeon performance than we do about hospital performance.”
Santa says that Consumer Reports has begun to rate some doctors, when we have enough reliable data to do it effectively. For example, we now rate heart surgery groups on bypass surgery, using clinical data we get from the Society of Thoracic Surgeons.
The bottom line, Santa says, is that, in our ongoing efforts to improve our hospital Ratings, we want to hear from patients—and hospitals, too. “We want to hear what the hospitals have to say," he said, "so we can work together to make our information even better.”
If you used our hospital Ratings, we want to know what you think of them and encourage you to share your experience with us.
Consumer Reports has no relationship with any advertisers or sponsors on this website. Copyright © 2007-2013 Consumers Union of U.S.
