Introduction

Researchers in some countries need to demonstrate the societal impact of their research (Wilsdon et al. 2015), despite this being impossible to directly measure. Social media mention counts (also known as “altmetrics”) have been proposed as a solution to this problem (Priem et al. 2010), on the basis that evidence of research being mentioned may serve as evidence that has been noticed, or as a proxy for evidence that has been found to be useful. The original definition of altmetrics has been expanded to encompass similar indicators derived from the wider web, such as online news, syllabus or policy document mentions. Evidence of societal impact is especially needed in the arts and humanities because these often claim to address the cultural needs of wide sections of the public (McCarthy et al. 2001; Nussbaum 2010). In addition, humanities research is often characterised by the need to explain the purpose of the research in detail, requiring greater engagement by the author with potential readers (Fry & Talja 2007; Whitley 2000). The social sciences underpin many numerically large professions as well as generating public interest for insights into the human condition and may expect to have a large non-academic audience (Lynd 2015). It therefore seems reasonable to expect that some specialization in these areas might generate substantial interest online. This article investigates the social sciences, arts and humanities together as relatively neglected areas even though they encompass widely varying research topics and practices (e.g. from Visual Arts & Performing Arts to Public Administration).

Although there is no detailed analysis of altmetrics for the social sciences, arts and humanities, one previous study has analysed half a million Web of Science (WoS) articles and reviews with Digital Object Identifiers (DOIs) published between July until December 2011. It used altmetric data collected via the Altmetric.com API on or before November 2014 (Costas, Zahedi, & Wouters 2015). It mapped the prevalence of various types of altmetrics in 250 WoS subject categories, and reported mean and median values by broad area for the social reference sharing site Mendeley. It analysed the proportion of fields within three broad areas (Science, Social Science, Arts & Humanities) that had average altmetrics above or below the full data set median (Table IV of Costas, Zahedi, & Wouters 2015), finding almost all Arts and Humanities subjects (81%–96%) to have below-average altmetric scores for Mendeley, Twitter, Facebook, Google Plus, blogs and news sources, whereas most Social Sciences subjects (63%–88%) had above-average scores. It also found Twitter to be most prevalent for medicine, psychology and the social sciences. This study used relatively old data, however it gives relatively coarse-grained information (other than the mappings) and does not analyse these subjects in detail. A comparison of Mendeley reader counts in social sciences and humanities found reader counts to be twice as high in the social sciences (Mohammadi & Thelwall 2014). Books are important in some social sciences, arts and humanities, but these seem to have lower altmetric scores than articles (Torres-Salinas, Robinson-Garcia, & Gorraiz 2017). A study of articles with PubMed IDs in multiple fields found that the proportion of articles that had been tweeted was relatively high in professional fields (17%) and psychology (15%) compared to other social sciences (9%) and humanities (7%) (Haustein et al. 2014), but more recent studies have reported much higher proportions tweeted. An analysis of Swedish humanities journal articles from 2012 found that 21% had been tweeted, compared to 3% mentioned in Facebook and 2% blogged (Hammarfelt 2014). Alternative indicators have also been developed for humanities books, such as library holding counts (Torres-Salinas & Moed 2009; White et al. 2009; White & Zuccala 2018), Goodreads reviews (Zuccala et al. 2015) and syllabus mentions (Kousha & Thelwall 2016).

Some content analyses of altmetrics have given insights into why articles can be cited in the social web. Most tweets citing academic papers just tweet a title or short summary and few tweet an opinion (Thelwall et al. 2013). Tweets from a set of researchers were often about research, although with disciplinary differences in the extent of tweeting (Holmberg & Thelwall 2014; see also: Haustein et al. 2014; Priem & Costello 2010). An investigation into citations in health-related science blogs found that these were a valuable means of translating academic research for the public, even sometimes including practical health advice (Shema, Bar-Ilan, & Thelwall 2015). A survey of people that had tweeted a link to academic research found that a substantial minority were non-academics, providing useful evidence that tweets may reflect public interest in scholarship to a moderate extent (Mohammadi et al. 2018).

Given the lack of current information about the prevalence of altmetrics across fields in the social sciences, arts and humanities, the current paper investigates this using data provided by the main current altmetric gatherer, Altmetric.com. Mendeley reader counts are excluded because they have been extensively researched and shown to be a citation-like indicator (Mohammadi et al. 2016; Thelwall 2017c). The following research questions drive the investigation, which focuses on the ten Altmetric.com indicators shared via their Applications Programming Interface (API) (blogs, news, tweets, Reddit, Facebook, Pinterest, Wikipedia, reviews, questions, Google Plus):

• RQ1: Are there substantial differences between Social Sciences, Arts and Humanities narrow fields in the prevalence of altmetrics?
• RQ2: Do any Social Sciences, Arts and Humanities narrow fields attract high proportions of non-zero scores for any altmetric?
• RQ3: Do individual Social Sciences, Arts and Humanities narrow fields that attract high proportions of non-zero scores for one altmetric also tend to attract high proportions of non-zero scores for other altmetrics?

Methods

The articles were taken from Scopus rather than WoS because of its greater coverage of non-English sources (Falagas et al. 2008; Mongeon, & Paul-Hus 2016), which is important for the Arts and Humanities because these often publish in local languages. Thus, a Web of Science dataset would be likely to overrepresent articles from English-speaking nations more than Scopus. Google Scholar has greater coverage (Halevi, Moed, & Bar-Ilan 2017; Harzing & Alakangas 2016) but lacks a classification scheme or automated access for fields. Both Microsoft Academic (Harzing & Alakangas 2017; Thelwall 2017a) and Dimensions (Thelwall 2018) also probably have greater coverage but both lack reliable fine-grained classification schemes.

All documents of type journal article that were published in 2013 and classified in Scopus Social Sciences (22 fields), Arts and Humanities (13) narrow fields were collected 15–17 July 2018 from the Scopus API. The year 2013 was chosen to give enough time (5 years) for articles to accrue mature citation counts in addition to altmetrics. Arts and humanities citations can be slow to accrue and so a relatively long citation window is appropriate here. Altmetrics for each article in this set with a DOI were collected 5–7 August 2018 from the free Altmetric.com API using a DOI query. Articles were analysed according to the Scopus classification of the journal in which they were published. This is not an ideal approach because articles within multidisciplinary journals can be allocated to inappropriate categories. Alternative classification methods, such as those based on keywords, article/title text, citations and references (Ruiz-Castillo & Waltman 2015) might largely solve this problem but would give less transparent and less practically useful conclusions for the current article, since there is no agreed procedure for algorithmic article-level classification.

The geometric mean and proportion cited was calculated for citation counts and each of the ten Altmetric.com indicators (Blogs, News, Tweets, Reddit, Facebook, Pinterest, Wikipedia, Reviews, Questions, G+) separately for each field. Geometric means are more appropriate than arithmetic means for skewed data (Fairclough & Thelwall 2015) and more fine-grained than medians. This is important when there are many zeros, as for all the indicators in all fields (all fields had a median of zero). Wilson Score Intervals (Wilson 1927) were used for the proportion cited calculations because these are reasonable estimates for 95% confidence intervals (Agresti & Coull 1998). Although the data is not a sample from a population in the conventional statistical sense, the data can be thought of as a sample of the papers that could have been written under similar circumstances (i.e. the apparent population: Berk, Western, & Weiss 1995; Bollen 1995; Leahey 2005) and confidence intervals cover the underlying likelihood of papers from each field attracting altmetrics.

To investigate fields with relatively high proportions of non-zero altmetric scores (RQ2), narrow fields with the highest proportions were investigated for each altmetric with outliers. This investigation took the form of tracing the source of the mentions or citations to find patterns in their creation.

For the third research question, the proportions of articles with non-zero scores on each altmetric for each field were correlated. Pearson correlations were used instead of Spearman because the proportion data was not highly skewed.

Results

Although citation counts for journal articles are low in the arts and humanities, in all fields they were more common than the commonest altmetric, Tweeters (Table 1). The rest of the results focus on the proportion of articles with a non-zero altmetric score. This is more informative than average scores because most articles have a score of 0 on all altmetrics.

The main results are displayed for two indicators per graph, ignoring the three indicators (Questions, Reviews and highlights, Pinners) with less than one citation per thousand articles. The Twitter altmetric counts the number of different Twitter users that have tweeted a link to an article for which a DOI could be extracted, using data from the Twitter firehose. Although there are variations and exceptions, on average, 35% of articles in social sciences fields are tweeted and 15% of articles in Arts and Humanities fields (Figure 1). The most tweeted field is unsurprisingly health-related (e.g. see the Health circles in Figure 1B of Haustein et al. 2014; see also: Costas, Zahedi, & Wouters 2015). The least tweeted, Classics, is not a natural fit with modern social media in the sense that it deals with communication in an age that pre-dated the web. No fields have an anomalously high rate of tweeting. No investigations were conducted for Twitter since there have been many previous content analyses of Twitter and no field has unusually many tweeted articles. Overall, there is a substantial difference in the prevalence of tweeting between fields, varying from 6% to 41% of articles, a factor of seven.

The Facebook altmetric covers links from public Facebook wall posts, excluding all private content within the site. Google Plus content also originates only from public posts. Since Twitter is public by default and Facebook is private by default, it is not clear whether articles are more prevalent in Twitter than Facebook, despite the lower scores for the latter. The proportion of articles with non-zero scores for Facebook and Google Plus are low, with only three fields having over 10% of their articles posted about publicly on Facebook (Figure 2). There is a substantial difference between fields in the prevalence of Facebook posts, varying from 2% to 11% of articles, a factor of five. For Google Plus, the range is from 0% to 1.7%. The three most posted fields were investigated for insights into why articles were discussed on Facebook. The most important conclusions are highlighted in bold.

• For Sociology and Political Science, 750 out of 7132 articles were posted on Facebook. The journal Critical Sociology in this category had its own Facebook page and posted about 41 out of 43 of its articles. The same was true for East European Politics and Societies (posting 31 out of 34 of its articles). Removing these two journals would reduce the percentage of posted articles from 10.5% to 9.6%.
• For Archeology, 274 out of 2626 articles were posted on Facebook. No journal had close to comprehensive Facebook coverage. The most influential was Journal of Archaeological Science, with 98 of its 416 articles on Facebook (23.6%), which is more than double the field average (10.4%). This journal seemed to be popular on Facebook for its articles having a general interest value for the public. For example, ‘Cacao consumption during the 8th century at Alkali Ridge, Southeastern Utah’ was posted by an educational research centre Facebook page because it had been mentioned in Science magazine as the earliest evidence of chocolate in North America. The less promising, ‘Statistical means for identifying hunter-gatherer residential features in a lithic landscape’ was posted by a religious organisation’s Facebook page as part of a hunter-gatherer event. Similarly, ‘A new approach to tracking connections between the Indus Valley and Mesopotamia: Initial results of strontium isotope analyses from Harappa and Ur’ was posted by a man in India, presumably for its local relevance. Thus, some Archeology seems to generate substantial enough non-academic interest for it to be broadcast independently on Facebook.
• For Gender Studies, 256 out of 2557 articles were posted on Facebook. No journals had close to complete coverage. The most influential journal was Psychology of Women Quarterly, with 21 out of 35 articles posted to Facebook (60%), although mostly from the journal Facebook page. As for Archeology, this journal’s articles seemed to have general interest on Facebook. For example, ‘Gender and emotion: What we think we know, what we need to know, and why it matters’ was discussed on Facebook.

Ignoring the general category, the field with the highest proportion of articles with Google Plus citations was investigated for causes.

• For History and Philosophy of Science, 46 out of 2961 articles were cited on public Google Plus pages. No journal systematically posted its articles in this site. Posts were typically simply announcements that an article had been published, accompanied by a title and abstract or other information. The articles posted mainly related to biomedical science (‘Parents’ and professionals’ perceptions of family-centered care for children with autism spectrum disorder across service sectors’ from the journal Social Science and Medicine and eight articles from Studies in History and Philosophy of Science Part C: Studies in History and Philosophy of Biological and Biomedical Sciences). Thus, the relatively high coverage for the Arts & Humanities narrow field History and Philosophy of Science seems to be due to its partial coverage of biomedical and other out of field research.

News counts record links from Altmetric.com’s list of science-related mainstream media online sources that can be tracked to DOIs. It may not cover social science, arts and humanities press content as well, given that it may appear in general sections of newspapers and websites. Similarly, the blog citations are from Altmetric.com’s list of science-related blogs, which may tend to omit arts and humanities blogs as well as occasional coverage of scholarship in other blogs (e.g. literary blogs). The prevalence of News varies from 0% to 5% of articles, and for Blogs from 0.6% to 7% gives a factor of 12. Ignoring the miscellaneous categories, the fields attracting the most mainstream media news attention (Figure 3) were investigated to find the causes or origins of the attention.

• For Sociology and Political Science, 329 out of 7132 articles were referenced by news sources (4.6%). Some politics journals attracted media attention. Journal of Politics had news stories written about 15 out of 50 articles (30%). In this journal, ‘Counterframing effects’ was written up in Journalist’s Resource, an online intermediary news site that aims to ‘curate scholarship relevant to media practitioners, bloggers, educators, students and general readers’ (journalistsresource.org/about). The article ‘Legislative organization and the second face of power: Evidence from U.S. state legislatures’ was discussed by Vox.com, an explanatory journalism news site. The article, ‘The Return of Old-Fashioned Racism to White Americans’ Partisan Preferences in the Early Obama Era’ was discussed by the Washington Post. Thus, the practical political implications of Journal of Politics articles have attracted the attention of analytic mainstream media sources.
• For Life Span and Life Course Studies, 58 out of 1332 articles were referenced by news sources (4.4%). Child Development Perspectives had nine out of 45 articles written up (20%). ‘The Biological Residue of Childhood Poverty’ was picked up by Vox.com for its story, ‘Why are Americans no longer the tallest people in the world? One theory: inequality’. ‘Infants and toddlers in foster care’ was discussed by liberal news site HuffPost. ‘Pollution and infant health’ was discussed in the Hungarian news site Index.hu. The health and social implications of Child Development Perspectives articles have again attracted the attention of analytic mainstream media sources.
• For Demography, 76 out of 1770 articles were referenced by news sources (4.3%). The biggest contributor, Population and Development Review had 11 out of 53 articles discussed (21%). The most discussed article, ‘The Characteristics Approach to the Measurement of Population Aging’, was picked up by 11 news sources (including HuffPost) for mentioning an alternative way to measure age that readers might apply to themselves. ‘Surplus Chinese men: Demographic determinants of the sex ratio at marriageable ages in China’ was discussed by Scientific American. ‘Demographic Metabolism: A Predictive Theory of Socioeconomic Change’ was cited for its theory about population demographic changes in an Eco-Business article focusing on a different article about climate change. Thus, Demography seems to contribute information of general and topical interest for the mass media.

The fields attracting the most blogging were investigated for possible causes of the interest.

• For History and Philosophy of Science, 203 out of 2961 articles were blogged (6.9%). History of the Human Sciences had eight out of 21 articles blogged (38%), none of which also attracted news stories. ‘Historians in the Archive: Changing Historiographical Practices in the Nineteenth Century’ and six other articles from this journal were blogged about by Campo Historiográfico, from a history professor. In contrast, ‘Oikonomia in the age of empires’ was referenced in a 4,000-word academic essay in the Disorder Of Things group blog for, ‘the critical inquiry of global politics’. Public Opinion Quarterly had seven out of 25 articles blogged (28%). ‘Coding voter turnout responses in the current population survey’ had been blogged by the Washington Post. ‘Extreme voices: Interest groups and the misrepresentation of issue publics’ had been blogged by Journalist’s Resource. ‘The polls’ trends: Americans’ changing views on crime and punishment’ and ‘Does Biology Justify Ideology? The Politics of Genetic Attribution’ were discussed by the same academic blog. ‘Commemoration matters’ was discussed by a different academic blog. Social Studies of Science had 11 out of 41 articles blogged (27%). ‘A history of deep brain stimulation’ was discussed in an academic blog. ‘Designing a market-like entity: Economics in the politics of market formation’ was announced without comment in a commercial blog. ‘The management of non-evidence in guideline development’ was discussed in a BMJ (medical journal) blog. Another article was apparently randomly selected as example of an academic article in a Scientific American blog about peer review and another was an author self-citation. Thus, this category attracted academic blogging as well as individual prolific bloggers.
• For Demography, 114 out of 1770 articles were blogged (6.4%). All 26 International Journal of Refugee Studies articles were announced without discussion by the Refugee Archives academic blog. Thus, systematic content-free blogging from a single source is the cause here.

Wikipedia citations are derived directly from Wikipedia references and Reddit citations are extracted by the Reddit API. Both are dependent on DOIs in references or the cited article pages. The prevalence of Wikipedia references varies by a factor of 8 from 0.5% to 3.8% of articles, and for Reddit it varies from 0% to 0.7%. The field attracting the most Wikipedia citations (Figure 4) was investigated for coverage reasons.

• For Gender Studies, 96 out of 2557 articles were cited by Wikipedia (3.8%). Gender and Language had 11 out of 14 articles cited. ‘Domestic violence and public participation in the media: The case of citizen journalism’ was cited in a page about citizen journalism. ‘Normal straight gays: Lexical collocations and ideologies of masculinity in personal ads of Serbian gay teenagers’ was cited in a page about LGBT rights in Serbia. ‘The construction of ‘tough’ masculinity: Negotiation, alignment and rejection’ was cited in the Masculinity page. All were added by the same Wikipedia editor, who has a feminism focus. Thus, an individual Wikipedian has contributed to effective representation of Gender and Language research in Wikipedia. Sex Roles had 11 out of 102 articles in Wikipedia (11%), referenced by different Wikipedians, and so seems to provide generally well-known research that is useful for topics covered by Wikipedia.

Ignoring the miscellaneous categories, the field attracting the most Reddit posts (Figure 4) was investigated for coverage reasons.

• For History and Philosophy of Science, 18 out of 2961 articles were posted about on Reddit (0.6%). Six were from Pragmatics and Cognition, out of 24 (25%), none of which had been discussed in any social media site. They had all been announced without discussion by a single user to the r/cognitivelinguistics subreddit.

Discussion

An important limitation of this study is that the Altmetic.com scores are likely to be substantial underestimates of public interest in journal articles in all fields. This is because Altmetric.com cannot scan private web pages, such as most of Facebook, and presumably does not have the resources to crawl the entire web for mentions. Moreover, it can only associate a citation with an article when it can be tracked to the cited article’s DOI, loosing informal mentions even formal citations that lack DOIs. Its data may also contain errors or miss citations for technical reasons (Ortega 2018), and automated tweets may also have been collected because they are difficult to detect (Haustein et al. 2016). Another limitation is the implicit assumption that all Scopus categories cover similarly scholarly journals but this is unlikely to be true. For example, the Health (social science) category contains journals publishing articles aimed at a more general or professional audience (e.g. ‘Identifying urinary incontinence in the home setting: Part 2: Treatment and related care of incontinence’ in Home Healthcare Nurse). Language and international differences in the uptake of the social web sites covered by Altmetric.com are also important factor that affect the results. For example, articles with a national focus outside of the countries that extensively use the social web sites covered by Altmetric.com may be discussed elsewhere on the social web.

Since monographs are central to the humanities and books and other outputs/activities can also be central to individual social sciences, arts and humanities fields, the results should not be used as evidence as a lack of public online discussion about the fields analysed. These discussions may instead centre on other outputs or activities of the scholars concerned.

The proportions tweeted in each field are higher than found in previous research for the humanities, (Hammarfelt 2014; Haustein et al. 2014) presumably due to increases over time in the amount of academic tweeting. The results echo, from a different perspective, the greater amount of altmetric data for the social sciences than for the arts and humanities (Costas, Zahedi, & Wouters 2015; Haustein, Larivière et al. 2014).

The analyses of fields with high scores on each individual altmetric identified six cases where a field had generated an above average level of non-academic interest: Archaeology and Gender Studies on Facebook; Sociology and Political Science, Life Span and Life Course Studies, and Demography in News sources; Gender Studies on Wikipedia. These give reassurance that social science and humanities subjects can attract online attention for their subject matter, but the evidence is fragile due to the low proportions involved. Given the apparent widespread interest in social science, arts and humanities topics, it seems strange that evidence of online discussions of articles in these areas is rare. It is likely that there is much more extensive discussion around relevant news stories, magazine articles and popular books for topics in these areas, but this does not seem to translate into interest in journal articles. To give a concrete example, the Life Span & Life Course Studies article, ‘The role of peer rejection in the link between reactive aggression and academic performance’ in Child & Youth Care Forum from 2013 had an Altmetric.com score of 0 and a web search for its title discovered no examples of the article being mentioned other than in academic reference lists (e.g. from 28 Google Scholar citations), publication announcements and CVs. Its finding, ‘high levels of reactive, not proactive, aggression were uniquely associated with low levels of academic performance, and peer rejection accounted for this association’ (Fite et al. 2013), seems to be of immediate practical significance within education despite the lack of public online non-academic mentions. Its full text is online (in ResearchGate) and the publisher website has a clear summary. It might be discussed in private areas of the social web and in subscription magazines for social workers and education professionals. Nevertheless, it is hard to understand why it has not generated any recordable online commentary. More generally, there seems to be a large gap between the provision of potentially useful research and its wider uptake, at least as recorded in the social web.

Conclusions

This article assessed the prevalence of altmetrics for journal articles in social science, arts and humanities fields, comparing the results at the level of whole fields rather than individual articles. Most articles in all fields had a score of zero for all altmetrics. This extends and updates prior research with Altmetric.com data (Table IV of Costas et al. 2015) by giving finer grained results. Excluding the announcement-based Twitter, at most 12% of articles in any field attracted a non-zero altmetric score. Thus, the Altmetric.com data suggests that most social science, arts and humanities papers are not discussed online in all fields. Altmetric.com data cannot be expected to be comprehensive and ignores, for example, mentions in private Facebook pages, and so this conclusion is tentative. From a policy perspective, and particularly in the current era of increasing expenditure on open science, it is worrying that so much research is ignored outside academia despite being presumably high quality, expensive to produce and frequently useful. It is therefore important to assess the extent to which research that is not discussed online is influential offline, but this may be impossible to systematically track. Despite all the altmetrics investigated being zero for most articles in all fields, there are substantial differences between fields in the extent to which articles have nonzero scores in the seven altmetrics examined in more. Thus, altmetrics are much more useful for some fields than others. Overall, they are twice as prevalent in the social sciences than in the arts and humanities.

Positive correlations between the seven main altmetrics and Scopus citation counts suggests that, with the partial exception of Wikipedia (weak correlation of 0.230), more cited fields are also likely to attract attention from all altmetrics (i.e. a higher proportion of articles with Scopus citations indicates a higher proportion of articles with a positive altmetric score). Individual exceptions include the relatively high number of public Facebook posts for Cultural Studies and the relatively low attention for Transportation. Due to these exceptions, the prevalence of relative prevalence of altmetrics must be assessed separately for individual fields. Similarly, and again with the partial exception of Wikipedia, there was a strong tendency for fields in which a higher proportion of articles attracted attention in one altmetric to also receive more attention from all the others. Together with the anomalies found in some of the investigations of outliers, this suggests that the extent to which social science, arts and humanities fields attract social web and news attention is not site-specific. Finally, since all altmetrics examined give a score of zero for most articles in all fields, they are not prevalent enough to be used to routinely compare individual articles for formal or informal evaluations. Nevertheless, they can still be used to compare groups of articles to assess the proportion with a non-zero score between groups (Thelwall 2017b), and to identify individual high impact articles. This may be useful for self-evaluations and to support more formal evaluations (e.g. to compare research groups) when the risk of systematic, bot or accidental manipulation (Haustein et al. 2016; Wouters & Costas 2012) is low.