Since U.S. News & World Report started its Best Hospitals rankings in 1990, the classification has become the industry's gold standard. If hospitals were to shun the rankings like law and medical schools, big questions loom for what it means to be the "best."
The law schools started it. In November, top-ranked Yale Law was first to say it would no longer cooperate with the U.S. News rankings of law schools and would stop sharing data with the publication. Within three months, more than 40 law schools — about 20 percent of the programs that U.S. News ranked — did the same, including 12 of the top 14.
The medical schools followed. No. 1 Harvard Medical School took the plunge in January. Within a month, a dozen medical schools followed, including Stanford (Calif.) School of Medicine, Columbia University, University of Pennsylvania Perelman School of Medicine, University of Chicago Pritzker School of Medicine and Johns Hopkins University School of Medicine. All said they would end their cooperation and no longer share data with the publication.
The medical and law school tumult raised questions about whether hospitals would follow suit and begin to question or downplay their rankings, which are released in the summer. Unlike medical schools, however, the hospitals are ranked by publicly available data from CMS as well as the American Hospital Association, medical associations and physician surveys. Children's hospitals are invited to participate in their evaluation.
"At this point, I can say definitively that neither top-ranked hospitals nor other hospitals will opt out of our survey this year," Ben Harder of U.S. News told Becker's May 2. "We've completed our data collection survey of children's hospitals, and hospital participation was as high as last year. In fact, every hospital that was ranked last year submitted data again this year."
Mr. Harder joined U.S. News in 2007 and oversees its portfolio of health rankings, including the Best Hospitals and Best Children's Hospitals, as managing editor and chief of health analysis. He is in talks with hospitals and health system executives, clinicians, researchers and specialty societies on an ongoing basis. No sudden movement from hospital stakeholders has been detected since the law school and medical school shakeups.
"Those conversations have continued for the last six months, for sure, but I wouldn't say the tenor has changed in any way," he said. "I can't recall a single conversation or change in which anyone has turned away from the rankings or not given us data."
While no hospitals have signaled a chilling effect, the attention paid to top-ranked institutions' involvement in the prestigious ranking system raises important questions about what the status of "best" means today about hospitals.
* * *
Unlike the rankings of law and medical schools, U.S. News & World Report intends for its hospital rankings to be used by patients, and ranked hospitals heavily cite their U.S. News ratings in direct-to-patient advertising. But as visible as hospitals' rank and standing may be, hospital leaders are oddly reluctant to talk about it openly and instead prefer to leave the matter for quiet hallway conversations — or such was the case at one recent industry event.
One hospital insider who did speak to Becker's on the record about U.S. News is Steve Klasko, MD. Dr. Klasko was the CEO of Philadelphia-based Jefferson Health for nearly a decade before stepping down to join venture capital firm General Catalyst as its executive-in-residence. In his time with Jefferson, the system's flagship hospital clinched a spot in the U.S. News Honor Roll, landing at No. 16 in the nation in 2017.
He has many thoughts on hospital rankings.
"Let me start out with something positive," Dr. Klasko said. "Transparency is great. Having some form of transparency is really good — an objective model that sort of says, 'Here's quality, here's access, here's cost, here's experience.' What's interesting is those measures are already out there through CMS Hospital Compare, Leapfrog. People can already get that."
Now, the critical from Dr. Klasko: "We, in 2023, don't understand how to measure quality when it comes to value and equity. There is no validated outcome composite measure for hospital performance and value."
Earlier this year, U.S. News announced it would assign more weight to clinical outcomes and other objective measures of quality in its hospital rankings, reducing the weight of physician opinion. The change was informed by hospital leaders and medical experts, it said.
In the methodology for the upcoming hospital rankings, outcome measures derived from federal data will account for 45 percent of the methodology in 11 specialty rankings, up from 37.5 percent the year prior. Structural indicators of quality will account for 35 percent of methodology. The weight of physician opinion fell from 25 percent to 12 percent for four specialties; from 27.5 percent to 15 percent for another seven specialties and from 50 percent to 30 percent for the specialty of rehabilitation.
"Sometimes present in the C-suite is the impression that physician opinion is what drives the results," said Mr. Harder. "We've looked at this extensively, with maybe one exception I'll mention, but it's not what drives which hospitals are best. We can remove that factor from the methodology and re-run the results and the top 50 in every specialty is virtually unchanged."
The one exception is that within the top 10, the order of the rankings "might change a bit" if physician opinion were removed. This brought me to my next question — how fixated do hospital leaders get not on the performance that results in a top 10 standing, but in their precise placement within the top 10? Is there greater focus on the result than the process?
"I think there is, in some quarters, an overemphasis on numerical rankings," Mr. Harder said. "If you are one of the top 50, you're doing really well. Whether you're 48 this year or 42, we'd be among the first to acknowledge there is often no statistical difference — numerical difference is within a margin of error. We do continue to publish numerical rankings because we believe there is a difference between No. 1 and No. 50. We don't see patients using the rankings driven by numerical placement."
If a hospital or health system board is obsessed with moving up one spot on the U.S. News Honor Roll, Mr. Harder said he would hope leaders instead look closely at priorities and align their focus with what matters for patients, staff and clinicians.
* * *
The tumult U.S. News faced in participation with its law and medical school rankings was a long time coming, with institutional stakeholders noting they had raised concerns about data strategy and bias in many forums, from written letters to in-person meetings. Mr. Harder is quick to note that for hospital rankings, he is receptive to input.
"We pioneered hospital rankings," he said. "We continue to be arguably the most influential. It's natural our method will be very closely followed and, at times, critiqued and scrutinized. It's something we welcome. We take feedback in stride. Some of that feedback will help us do our job better. Every time we take critical feedback we look for what's the kernel of truth that should motivate us to go back to the data with fresh eyes or an outcome we can incorporate into our methods."
One challenge the outlet faces is accounting for the many different patients it aims to equip with its rankings. Given hospital and health systems' intensified focus on health equity, any data collection or analysis that sees patients as one-size-fits-all is called into question and in need of reexamination. Are the "best" hospitals still the "best" when accounting for access, insurance status, language barriers or other barriers that can obstruct or limit patients' healthcare experience?
"If I'm a Medicaid patient and go to the five top systems, can I get an appointment right away?" Dr. Klasko asked. "If you're in any major city and you take the six health systems that are in a major city and there is a 25 year difference in life expectancy – in essence those health systems as a composite have failed. If you really see health equity as a measure, the outcome would be: 'We're going to penalize all of you because you haven't gotten together to solve those outcomes.'"
Mr. Harder said he has been involved in ongoing dialogue with participants about health equity as it shows up in the rankings. The outlet's last hospital rankings evaluated organizations for racial disparities in unplanned readmission, charity care provision for uninsured patients, community residents who accessed the hospital for care, and preventive care for Black residents in the community, noting that "additional measures of equity may be added over time." The outlet is also trying to better account for the shift from inpatient to outpatient care settings.
"We really appreciate the feedback," Mr. Harder said. "We could never do what we do for as long as we have if we weren't talking to hospitals."