The Adoption of an Online Journal Club to Improve Research Dissemination and Social Media Engagement Among Hospitalists
BACKGROUND: Twitter-based journal clubs are intended to connect clinicians, educators, and researchers to discuss recent research and aid in dissemination of results. The Journal of Hospital Medicine (JHM) began producing a Twitter-based journal club, #JHMChat, in 2015.
OBJECTIVE: To describe the implementation and assess the impact of a journal-sponsored, Twitter-based journal club on Twitter and journal metrics.
INTERVENTION: Each #JHMChat focused on a recently published JHM article, was moderated by a social media editor, and included one study author or guest.
MEASUREMENTS: The total number of participants, tweets, tweets/participant, impressions, page views, and change in the Altmetric score were assessed after each session. Thematic analysis of each article was conducted, and post-chat surveys of participating authors and participant responses to continuing medical education surveys were reviewed.
RESULTS: Seventeen Twitter-based chats were held: 7 (47%) focused on value, 6 (40%) targeted clinical issues, and 4 (27%) focused on education. On average, we found 2.17 (±0.583 SD) million impressions/session, 499 (±129 SD) total tweets/session, and 73 (±24 SD) participants/session. Value-based care articles had the greatest number of impressions (2.61 ± 0.55 million) and participants (90 ± 12). The mean increase in the Altmetric score was 14 points (±12), with medical education-themed articles garnering the greatest change (mean increase of 32). Page views were noted to have increased similarly to levels of electronic Table of Content releases. Authors and participants believed #JHMChat was a valuable experience and rated it highly on post-chat evaluations.
CONCLUSION: Online journal clubs appear to increase awareness and uptake of journal article results and are considered a useful tool by participants.
© 2018 Society of Hospital Medicine
Twitter Metrics
After each session, the following Twitter-based engagement metrics were obtained using the Symplur© Healthcare Hashtag project;8 total number of participants and tweets, tweets/participant, and total impressions (calculated as the number of tweets from each participant multiplied by the number of followers that participant currently had then summed up for all participants). Simply put, impressions can also be thought of as the number of times a single Tweet makes it into someone else’s Twitter feed. So as to avoid artificially inflated metrics, all were obtained 2 hours after the end of the journal club. Participants were defined as anyone who posted an original tweet or retweeted during the session and were encouraged to tag their tweets with the hashtag #JHMChat for post-discussion indexing and measurement. Because authors’ or guests’ popularity on Twitter may influence participation rates, we also assessed the number of followers for each participating author. Spearman’s rank correlation was calculated (Microsoft ExcelTM) where appropriate.
Altmetrics and Page Views
As a means to measure exposure and dissemination external to Twitter, we assessed the change (“Delta”) in the each article’s Altmetric score9, a digital-based metric that quantifies the attention received by a scientific publication on various online platforms including news, blogs, and social media. Delta Altmetric scores were calculated as the difference between the scores on the day of the session and 2 weeks after the respective session, with higher scores indicating greater global online discussion. By measuring the Altmetric score on the day of the discussion, we established a baseline score for comparison purposes. Additionally, this allowed us to better attribute any changes that may have occurred to the discussion itself.
Additionally, using information provided by the journal publisher (John Wiley & Sons Publishing) in 2016, we assessed the effect of #JHMChat on the number of article page views on the JHM website relative to the release of the electronic Table of Contents (eTOC). The eTOC release was chosen as it is historically associated with a high number of page views. In order to isolate the effect of #JHMChat, we only reviewed months in which #JHMChat was not held within 3 days of the eTOC release. Because JHM changed publishers in January 2017, we only assessed page view data on 2016 sessions, as the new publisher lacked enhanced search optimization to obtain these data.
Thematic Analysis
In addition to the above measurements, a thematic analysis of each article was conducted to assess any common themes that would influence our chosen metrics. Themes were assessed and ascribed by one author (C.M.W.) and verified by another (V.M.A.).
Participant and Author Experience
To assess the participant experience, responses to a post-session CME questionnaire that assessed (1) overall quality, (2) comprehensiveness of the discussion, (3) whether the participant would recommend the chat to a colleague, and (4) whether participation would lead to practice-changing measures were reviewed. Registration of each session for CME was also quantified. Finally, each participating author was asked to fill out an electronic post-chat survey (SurveyMonkey®) meant to assess the authors’ experience with the journal club (Appendix).
