Blogs

Share

Hospital Researchers Find Important Patient Experience Insights By Mining Twitter

By The SHSMD Team posted 05-16-2016 11:06 AM

  

When you think of using social media to conduct groundbreaking customer research, a hospital probably isn’t the first place you’d look. But the Computational Epidemiology Group in the Boston Children’s Hospital’s Computational Health Informatics Program (CHIP) is a leader in mining social channels for health-related insights.

They’re now looking to sentiment analysis of publicly shared tweets to complement more traditional research methods and help hospital administrators better track patient experience metrics.

The team at Boston Children’s Hospital first attracted attention to their work a few years ago when they successfully mined reviews on the popular rating website Yelp. Their analysis showed the ingredients people described in their reviews mentioning food poisoning matched closely with ingredients public health officials later reported were involved in food-borne illness outbreaks for that time.

It was an important finding since it showed that socially shared information potentially carried useful health insights. Dr. John Brownstein, BCH’s Chief Innovation Officer and a leader of the research team, notes, “It’s hard to make people come to you. People aren’t engaged necessarily in public health.” But if you can tap into their online voices, he says, “You can actually get a huge amount of information that would not come from another vehicle.”

Faster and more useful tool for gauging patient experience?
Jared Hawkins, a faculty member at Boston Children’s Hospital and a researcher on Brownstein’s team, actively develops new tools and methods that extend the usefulness of sentiment analysis. We recently spoke with Hawkins about research he’s published using Twitter sentiment as a proxy for hospital quality metrics.

As Hawkins explains, quality metrics like the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey are extremely important to hospital administrators. And yet, these reports often lag today’s events by 12 or more months because of the need to survey patients and tabulate findings. Many administrators also complain about low response rates and insufficient granularity in the data.

What if, Hawkins wondered, commentary on Twitter could serve as a proxy for these official quality scores, or serve as a new satisfaction metric? Even if potentially less accurate, could the timeliness and granularity of a Twitter-powered sentiment score still make it useful for reporting and decision-making?

Sifting through thousands of tweets to understand sentiment
With Twitter’s support, Hawkins and his colleagues began by developing a collection of hundreds of thousands of tweets that explicitly mentioned hospitals in the 12-month period between late 2012 and late 2013.

They first weeded out tweets that had nothing to do with patient experience. Hawkins is proud of their efforts here to identify and isolate specific topics. They pushed the envelope of artificial intelligence and machine learning to automatically eliminate tweets related to, say, fundraising, open jobs, or clinical research. It’s easier said than done at scale.

The team then trained a computer algorithm to parse the remaining tweets and watch for expressions of anger, happiness and most every emotion in between. Reading each tweet to determine meaning and sentiment would never have worked given the size of their dataset. Hawkins’ team instead relied again upon sophisticated algorithms to do the work for them.

Importantly, Hawkins’ experience had taught him that sentiment analysis can be tricky since people communicate in confusing and inconsistent ways. Consider the difficulty of teaching an algorithm to differentiate between, “They would not let my dog stay in this hospital.” vs. “I would not let my dog stay in this hospital.”

Slang, abbreviations, emojis, foreign languages, and spelling mistakes added to the challenges Hawkins’ team faced. Both open-source and internally developed tools helped get the job done with surprising accuracy.

Imperfect but still helpful in gauging hospital quality metrics
Ultimately, Hawkins’ research helped him demonstrate that his hypothesis was mostly right. Eliminating hospitals that didn’t attract enough tweets to offer reliable data, he showed an interesting correlation between some important hospital quality outcomes and his Twitter sentiment scores.

Importantly, though, they did not see a relationship between tweet sentiment and the gold-standard HCAHPS experience data. “This is a brand-new way of using Twitter data,” Hawkins notes. “It may be that we have to be cautious about using tweet sentiment to understand quality.”

But obsessing about perfection misses the point, he argues, if the speed of getting a “good enough” insight can be substantially improved. And research continues that could further improve the accuracy of his findings. He’s already underway with that effort.

Hawkins went on to rate hospitals based on his findings naming Seattle Children’s Hospital the Mayo Clinic, and Boston Children’s Hospital as the early leaders in Twitter-based public sentiment. They’re all highly regarded institutions so it shouldn’t be a surprise that their patients praise them on Twitter.

Your patients are talking about you and their experience with your hospital. If you’re ready to listen to these conversations and learn from them, sentiment analytics could be just the tool for you.

By James A. Gardner | May 16, 2016
SHSMD Digital Engagement Task Force Member
Digital Strategist & Client Development
ConnectiveDX
Boston, MA
855.700.2105
jgardner@connectivedx.com

The author thanks S. Cannon of Nationwide Children’s Hospital, T. Gilkey and J. Hawkins of Boston Children’s Hospital, and D. Hinmon of the Mayo Clinic’s Social Media Network for their comments and suggestions. No endorsement is implied.
0 comments
738 views

Permalink