Become a Reviewer

Reading: Use of Altmetrics to Analyze ScholarWorks in Natural Resource Management


A- A+
Alt. Display


Use of Altmetrics to Analyze ScholarWorks in Natural Resource Management


David L. Kulhavy ,

Arthur Temple College of Forestry and Agriculture, Stephen F. Austin State University, Nacogdoches, Texas, US
X close

R. P. Reynolds,

Ralph W. Steen Library, Stephen F. Austin State University, Nacogdoches, Texas, US
X close

D. R. Unger,

Arthur Temple College of Forestry and Agriculture, Stephen F. Austin State University, Nacogdoches, Texas, US
X close

M. W. McBroom,

Arthur Temple College of Forestry and Agriculture, Stephen F. Austin State University, Nacogdoches, Texas, US
X close

I-Kuai Hung,

Arthur Temple College of Forestry and Agriculture, Stephen F. Austin State University, Nacogdoches, Texas, US
X close

Yanli Zhang

Arthur Temple College of Forestry and Agriculture, Stephen F. Austin State University, Nacogdoches, Texas, US
X close


Digital preservation of library materials has increased the need for methods to access the documents and contents maintained in digital archives. The use of altmetrics to quantify the impact of scholarly works, including PlumX, is increasing readership by listing articles in reference services. The outreach from the digital repository ScholarWorks at Stephen F. Austin State University (SFASU) highlights the impact within the natural resources community from Digital Commons, Forest Sciences Commons; and from the Natural Products Chemistry and Pharmacognosy Commons. The use of PlumX altmetrics was examined to evaluate usage, impact, and digital audience downloads for the Arthur Temple College of Forestry and Agriculture (ATCOFA) at SFASU.
How to Cite: Kulhavy, D.L., Reynolds, R.P., Unger, D.R., McBroom, M.W., Hung, I.-K. and Zhang, Y., 2020. Use of Altmetrics to Analyze ScholarWorks in Natural Resource Management. Journal of Altmetrics, 3(1), p.6. DOI:
  Published on 16 Oct 2020
 Accepted on 21 Sep 2020            Submitted on 13 Aug 2020


Digital preservation of library materials is increasing, and proactive institutional organization and response are critical for developing comprehensive procedures and oversight in both securing and cataloging digital library resources (Wilson 2017). This includes acquiring needed rights and permissions to store and disseminate the materials and to continue adding to the collection. Kulhavy et al. (2017) reviewed digital preservation of natural resource documents from the Arthur Temple College of Forestry and Agriculture (ATCOFA) for ScholarWorks at Stephen F. Austin State University (SFASU) as part of Forest Sciences Commons™, a subset of Life Sciences Commons™ of the Digital Commons Network™ (DCN) for bepress™. Documents archived included research articles, eBooks, digital journals, theses and dissertations, monographs, bulletins, and documents produced by research centers on the SFASU campus. Digital records give greater access to material downloads, as the scanned documents are archived at the online SFASU ScholarWorks. The DCN software is used to build institutional repositories and publish peer-reviewed journals, and increases the visibility of the institution and its research (Erway 2012; Enis 2013).

Once entered into the ScholarWorks database, altmetrics are available to assess the access and usability of the documents archived in the system. A team of library archivists is generally responsible for scanning digital records to ensure accuracy and consistency of records (Lapinski et al. 2013). Choice of the altmetrics for measurement of digital record use is a function of cost and the intended recordkeeping for the institutional records. The use of altmetrics to quantify scholarly communication promotes the impact of the SFASU academic community (Roemer and Borchardt 2013, 2015).

The use of ScholarWorks at SFASU supports the mission of ATCOFA to produce society-ready natural resources managers who deal effectively with complex ecological, economic, and social issues associated with contemporary natural resources challenges. Maintaining excellence in teaching, research, and service to enhance the environment through sustainable management, conservation, and protection of natural resources is ATCOFA’s strategic guiding principle (Bullard et al. 2014). The development of the ScholarWorks natural resources repository at SFASU reflects the faculty research and teaching emphasis (Kulhavy et al. 2017). The research is primarily funded by the McIntire-Stennis Cooperative Forestry Program, which was established during the Kennedy Administration by PL 87–788. McIntire-Stennis addressing the need to acquire new knowledge, understanding, and technologies and apply these to complex, transdisciplinary social and biological issues and problems (Bullard & Straka, 1986; Thompson & Bullard, 2004; Bullard et al., 2011; Bullard et al., 2014). This program was designed to impact society by providing forestry institutions with the capacity funding needed not only to address forestry issues but also to disseminate findings to key audiences, including forestry professionals, the forest industry, and forest landowners, as well as to train new forestry professionals from the undergraduate through doctoral levels. Tools like ScholarWorks are often overlooked as a necessary means of reaching these audiences in the digital age.

PlumX is an aggregator that offers more metrics, including citation and usage metrics (i.e., views and downloads). It covers more than 52.6 million artifacts, being the largest altmetrics aggregator (Ortega 2018a; Plum Analytics 2018). Altmetrics can be used to measure the impact that scientific articles have on society (Holmberg et al. 2019a). PlumX is the primary altmetrics source for analysis of downloads and use of the natural resources materials from the SFASU ScholarWorks. In addition, the downloads and articles available are retrieved from the Digital Open Network from bepress. This information provides a basis for evaluation of the impact in the natural resource digital-retrieval user community including top academic scholars with archives stratified by institution to compare the impact of different academic institutions.

PlumX altmetrics use downloads over time and referrals to the digital preservation system (Collier & Deli-yannides 2016; Wong & Vital 2017; Bar-Ilan et al. 2019). Altmetrics data are the aggregated views, mentions, downloads, shares, discussions, and recommendations of research outputs across the scholarly web (Fenner 2014). More emphasis is being placed on the use and availability of research findings (Vanclay et al. 2015; Holmberg et al. 2019a, b). The impact of research can be viewed as all the different ways in which research can benefit individuals, organizations, and nations (ESRC 2016). In 2017, Plum Analytics was acquired by Elsevier (, and its altmetrics information was added to the Scopus database (Elsevier 2017; Michalek 2017).


Natural resource articles were scanned into ScholarWorks and archived in the Ralph W. Steen Library at SFASU. These articles were evaluated for use and distribution using the Forest Sciences DCN for bepress from February 13, 2013 to June 11, 2020. Data extracted and posted from bepress included number of institutional records by country compared across natural resource digital archives, impact by authors in the forest sciences, downloads of articles from ScholarWorks, referrers to search engines for ScholarWorks, downloads of articles by country, downloads of articles by month, and examples of PlumX altmetrics.


ATCOFA had five of the top eight authors in the country representing total number of downloads for the Forest Sciences DCN in bepress as of June 11, 2020 (Table 1) and ATCOFA was fourth out of 144 institutions in the United States, with 744 works in the Forest Sciences Commons (Figure 1). All-time downloads for ATCOFA in the SFASU ScholarWorks were 118,205 from 198 countries. There were 8,111 full-text articles with 9,167 authors and 1,140,202 downloads in the Forest Sciences Commons. There were 7,648 institutions downloading information, with 59% from education, 27% commercial, 8% government, and 3% organizations. The most accessed institution was SFASU, with 2,142 downloads. The highest country total was for the United States (52,692), followed by India, China, the Philippines, Canada, the United Kingdom, Indonesia, Malaysia, Vietnam, and Germany (Figure 2). The major referrers (3,435) for retrieval of information from the SFASU ScholarWorks site were Google, Google Scholar, and ScholarWorks at SFASU (Figure 3). Total downloads from February 13, 2013 to June 11, 2020, were 118,205 (Figure 4).

Table 1

Top eight scholar citations for Forest Sciences Commons, May 2020.

Steven Bullard, Arthur Temple College of Forestry and Agriculture, Stephen F. Austin State University
Jerome Vanclay, Southern Cross University
David Kulhavy, Arthur Temple College of Forestry and Agriculture, Stephen F. Austin State University
Daniel Unger, Arthur Temple College of Forestry and Agriculture, Stephen F. Austin State University
I-Kuai Hung, Arthur Temple College of Forestry and Agriculture, Stephen F. Austin State University
Yanli Zhang, Arthur Temple College of Forestry and Agriculture, Stephen F. Austin State University
Richard Schultz, Iowa State University
Jake Delwiche, US Forest Service

Figure 1 

Institutions in Forest Commons by number of articles cataloged.

Figure 2 

Downloads of articles from ScholarWorks, Stephen F. Austin State University.

Figure 3 

Referrers to search engines for articles in ScholarWorks, Stephen F. Austin State University.

Figure 4 

Downloads from February 2013 to June 2020, Stephen F. Austin State University ScholarWorks.

The most downloads for a paper were for Li et al. (2011), with 20,872 usages (18,489 downloads and 2,383 abstract views), captures by 696 readers, and 52 citations (Figure 5), determined using PlumX altmetrics. This article is listed under the Natural Products Chemistry and Pharmacognosy Commons, with SFASU third out of 50 institutions represented in the United States; there are 159 articles, 490 authors, and 104,457 downloads. Three of the top four authors are from SFASU: Shiyou Li, Wei Yuan, and Ping Wang. The research articles are published from the National Center for Pharmaceutical Crops established at ATCOFA in 2004 to improve human health by discovering novel anti-tumor and antiviral agents from native and invasive plant species (Li et al. 2010; Kulhavy et al. 2017). The center publishes the international peer-reviewed journal Pharmaceutical Crops ( to investigate cultivated species used for extraction or preparation of therapeutic substances used in pharmaceutical formulations, vaccines, and antibodies and therapeutic proteins (Li et al. 2010). The center is currently investigating endocide-induced abnormal growth forms of giant salvania (Salvinia molesta) (Li et al. 2018). Endocides (endogenous biocides) are metabolites that can poison or inhibit the parent via induced biosynthesis or external applications (Li et al. 2016).

Figure 5 

Published paper most downloaded from Stephen F. Austin State University ScholarWorks.


As much of the research is funded by the McIntire-Stennis Cooperative Research Program, which is federal funding provided to increase forestry research in the production, utilization, and protection of forestland, increased readership and downloads of natural resources articles in the SFASU ScholarWorks enhance the flow of information to users in educational, government, commercial, and organizational entities. The number of downloads indicates a measure of societal impact, as more downloads across a variety of user groups and countries expands the range of the articles (Holmberg 2019a). The purpose of the McIntire-Stennis program is to increase research on forest productivity, utilization, and protection; to train future forestr scientists; and to cooperate with other states in forestry research (Thompson & Bullard 2004; Bullard et al. 2011; Rickenbach et al. 2013; Allen 2017). The development of ScholarWorks for natural resources at SFASU increases the expanding role of libraries in archiving and dissemination of scholarly writing in an open access environment (American Association of College and Research Libraries 2000). At SFASU, ScholarWorks is supported and maintained by the Ralph W. Steen Library and librarians, which provides worldwide visibility of research in a single location with a stable URL and open access to all faculty research at SFASU. A stable platform is essential to increase the use and reliability of research material that is produced (Gracy & Kahn 2012).

PlumX Altmetrics

Work is uploaded to ScholarWorks in the Center for Digital Scholarship, including metadata for information retrieval using PlumX altmetrics. PlumX is an aggregator that offers more metrics for citation and usage (i.e. views and downloads) for over 52.6 million artifacts (Ortega 2018b; Plum Analytics 2018). PlumX provides alternative metrics (altmetrics) to view reader impact from an academic institution, individual programs, or individual faculty members (Collier & Deliyannides 2016). Account administrators create profiles in PlumX for individual researchers or groups, including images and biographical and contact information (Champieux 2015). PlumX Metrics can be used to provide information on interaction with research, including articles, conference proceedings, book chapters, and books, in an online environment (Wong & Vital, 2017). The five categories of PlumX Metrics include Usage, Captures, Mentions, Social Media, and Citations. These five metrics are color coded: usage (green), captures (magenta), mentions (yellow), social media (blue), and citations by others (orange). The size of the circle (e.g., usage, Figure 5) indicates the importance of this metric for each article in ScholarWorks.

Usage includes downloads, views, clicks on the work, and library holdings. The most important usage metric according to a Taylor & Francis Open Access Survey in 2014 (Frass et al. 2014) is citations of articles (81% of respondents), followed by article downloads (60%). Next is captures, which includes blog posts, reviews, Wikipedia, and news references; social media includes Facebook, tweets, and shares (Lindsay 2016). PlumX implementation by libraries raises the academic profile and adds to the altmetrics for faculty by adding information on authors across social media, mentions, and citation counts (Wong & Vital 2017). The University of Pittsburgh created an extensive digital library and used PlumX altmetrics to measure usage with the PlumX artifact widget (Collister et al. 2017). As digital resources and library services expand, altmetrics for tracking user information for articles is increasing, including blogs, social media, and users. Librarians serve as the conduit to the altmetrics that acquire and provide access to resources for east of tracking and to evaluate the scholarly impact of an academic institution. Altmetrics like PlumX have a cost structure tied to ScholarWorks, the institutional repository for digital preservation at SFASU. PlumX altmetrics provide data for DOIs or other digital identifiers (Roemer & Borchardt 2013, 2015; Peters et al. 2016), and use with ScholarWorks provides user information in connection with the DCN and bepress.

Altmetrics by themselves may not express societal impact but reflect both individual and institutional use of records. The actual use of the information is determined in further exploration of citation uses in scientific search engines. Altmetrics also promote new forms of scholarly communication, with the goal of assessing the societal impact of research to identify and measure how a specific research document has been used and what kind of influence it has had, not just within academia but also beyond (Holmberg et al. 2019a). Altmetrics are currently being investigated to determine if they could be used to assess societal impact of research. The rapid development of the use of altmetrics (or alternative metrics) provides guidance on how research is being used, viewed, and moved. The electronic transfer of information can be tracked to provide data on the use and distribution of research findings (Penfield et al. 2014).

For PlumX, tweets and blog mentions had the earliest views but were not persistent. Bookmarking (Mendeley readers), usage metrics (downloads and views) and bibliographic indicators (citations) increased over time and were persistent (Ortega 2018a). PlumX altmetrics identified the highest numbers of Mendeley readers of an article compared to Crossref Event Data (CED) and (Ortega 2018b, 2019). Wong & Vital (2017) highlight the effectiveness of PlumX to showcase the reader impact and academic profile of Saint Mary’s College of California. Nuzzolese et al. (2019) used PlumX altmetrics to analyze the National Scientific Qualification for scholars in Italy as a method of measuring the impact of their research. The altmetrics led to an assessment at the product level of scholarly products. (Lapinski et al. 2013). Bar-Ilan et al. (2019) and Brigham (2014) reviewed the use of altmetrics to expand the use of this tool for researchers to showcase impact metrics. PlumX Dashboards integrates the DCN as an institutional repository, with Selected Works as a profile system leading to Repository Harvesting through web crawling. Use includes articles, book chapters, books, conference papers, discussion papers, presentations, posters, reports, and theses and dissertations.

With the expanded role of digital measures for libraries, increased use of altmetrics enhances the availability of research materials across a variety of platforms. The primary use of altmetrics at SFASU is currently usage (for download and abstracts). However, expansion on the use of social media, blogs, and mentions expands the potential readership and use. The use of PlumX altmetrics at SFASU reflects the major audience with usage, followed by captures and citations, with very little use of mentions or social media. For ScholarWorks at SFASU, the primary search engine for finding articles is Google, followed by Google Scholar and ScholarWorks. As ScholarWorks expands the number of articles listed, the usage will increase, expanding the importance of download statistics as a measure of societal impact.


This research was supported by the National Institute of Food and Agriculture, U.S. Department of Agriculture, McIntire-Stennis project under TEXY146 1001874 administered by the Arthur Temple College of Forestry and Ag-riculture, Stephen F. Austin State University.

Competing Interests

The authors have no competing interests to declare.


  1. Allen, J. A. (2017). Sustaining healthy and productive forests, McIntire Stennis Strategic Plan: Investing in America’s competitive position in the global marketplace. The McIntire-Stennis Cooperative Forestry Research Program 2017 strategic plan. Retrieved August 9, 2020 from 

  2. American Association of Colleges and Research Libraries. (2000). Information literacy competency standards for higher education. American Library Association, Chicago, Illinois, USA. 

  3. Bar-Ilan, J., Halevi, G., & Milojević, S. (2019). Differences between altmetric data sources – A case study. Journal of Altmetrics, 2(1), 1. DOI: 

  4. Brigham, T. (2014). An introduction to altmetrics. Medical References Services Quarterly, 33, 438–477. DOI: 

  5. Bullard, S. H., & Straka, T. J. (1986). Continuing education needs of natural resource professionals. Resource Management and Optimization, 3, 281–290. 

  6. Bullard, S. H., Brown, P. J., Blanche, C. A., Brinker, R. W., & Thompson, D. H. (2011). A “Driving Force” in developing the nation’s forests: The McIntire-Stennis Cooperative Research Program. Journal of Forestry, 109, 141–148. 

  7. Bullard, S. H., Stephens Williams, P., Coble, T., Coble, D., Darville, R., & Rogers, L. (2014). Producing “society-ready” foresters: A research-related process to revise the bachelor of science in forestry curriculum at Stephen F. Austin State University. Journal of Forestry 112, 354–360. DOI: 

  8. Champieux, R. (2015). PlumX. Journal of the Medical Library Association, 103(1), 63–64. DOI: 

  9. Collier, L. B., & Deliyannides, T. S. (2016). Altmetrics: Documenting the story of research. Against the Grain, 28(1), Article 9. DOI: 

  10. Collister, L. B., Kirschner, J., Bradbury, M., Deliyannides, T. S., & Kear, R. (2017). Altmetrics and library publishing. IFLA WLIC, Wroclaw, Poland. 7 p. 

  11. Economic and Social Research Council (ESRC). (2016). What is Impact? Retrieved August 9, 2020 from 

  12. Elsevier. (2017). Elsevier acquires leading ‘altmetrics’ provider plum analytics. Retrieved May 23, 2019 from https://www.elsevier. com/about/press-releases/corporate/elsevier-acquires-leading-altmetrics-provider-plum-analytics. 

  13. Enis, M. (2013). Uncommonly open: The new digital commons network. The Digital Shift, Library Journal. 

  14. Erway, R. (2012). Last impact: Sustainability of disciplinary repositories. Online Computer Library Center, Inc., Dublin, Ohio. 

  15. Fenner, M. (2014). Altmetrics and their novel Measures for Scientific Impact. In S. Bartling & S. Friesike (Eds.), Opening Science (pp. 179–189). Cham: Springer International Publishing. DOI: 

  16. Frass, W., Cross, J., & Gardner, V. (2014). Taylor and Francis Open Access Survey, June 2014. Taylor and Francis/Routledge. 35 pp. 

  17. Gracy, K. F., & Kahn, M. B. (2012). Preservation in the digital age. Library Resources & Technical Services, 56(1), 25–43. DOI: 

  18. Holmberg, K., Bowman, S., Bowman, T., Didegah, F., & Kortelainen, T. (2019a). What is societal impact and where do altmetrics fit into the equation? Journal of Altmetrics, 2(1), 6. DOI: 

  19. Holmberg, K., Bowman, T., Didegah, F., & Lehtimäki, J., (2019b). The relationship between institutional factors, citation and altmetric counts of publications from Finnish universities. Journal of Altmetrics, 2(1), 5. DOI: 

  20. Kulhavy, D. L., Reynolds, R. P., Unger, D. R., Bullard, S. H., & McBroom, M. W. (2017). Digital preservation and access of natural resource documents. Journal of Education and Practice, 8, 121–128. 

  21. Lapinski, S., Piwowar, H., & Priem, J. (2013). Riding the crest of the altmetrics wave: How librarians can help prepare faculty for the next generation of research impact metrics. arXIV:1305.3328, 4 p. 

  22. Li, S., Wang, P., Su, Z., Lozano, E., LaMaster, O., Grogan, J. B., Weng, Y. Decker, T., Findeisen, J., & Garrity, M. (2018). Endocide-induced abnormal growth forms of invasive giant salvania (Salvania molesta). Scientific Reports, 8, Article Number 8006. DOI: 

  23. Li, S., Wang, P., Yuan, W., Su, Z., & Bullard, S. H. (2016). Endocidal regulation of secondary metabolites in the producing organisms. Scientific Reports, 6, Article Number 29315. DOI: 

  24. Li, S., Yuan, W., Deng, G., Wang, P., Yang, P. & Aggarwal, B. B. (2011). Chemical composition and product quality of turmeric (Curcuma longa L.). Pharmaceutical Crops, 2, 28–54. DOI: 

  25. Li, S., Yuan, W., Yang, P., Antoun, M. D., Balick, M. J., & Cragg, G. M. (2010). Pharmaceutical crops: An overview. Pharmaceutical Crops, 1, 1–17. DOI: 

  26. Lindsay, J. M. (2016). PlumX from Plum Analytics: Not just altmetrics, Journal of Electronic Resources in Medical Libraries, 13(1), 8–17, DOI: 

  27. Michalek, A. (2017). Plum Analytics joins Elsevier. Retrieved from: 

  28. Nuzzolese, A. G., Ciancarina, P., Gangemi, A., Peroni, S., Poggi, F., & Presutti, V. (2019). Do altmetrics work for assessing research quality? Scientometrics, 118, 539–562. DOI: 

  29. Ortega, J. L. (2018a). Reliability and accuracy of altmetric providers: A comparison among, PlumX and Crossref Event Data. Scientometrics, 116, 2123–2138. DOI: 

  30. Ortega, J. L. (2018b). The life cycle of altmetric impact: A longitudinal study of six metrics from PlumX. Journal of Informetrics, 13, 579–589. DOI: 

  31. Ortega, J. L. (2019). Availability and audit of links in altmetric data providers: Link checking of blogs and news in, Crossref Event Data and PlumX. Journal of Altmetrics, 2(1), 4. DOI: 

  32. Penfield, T., Baker, M. J., Scoble, R., & Wykes, M. C. (2014). Assessment, evaluations and definitions of research impact: A review. Research Evaluation, 23(1), 21–32. DOI: 

  33. Peters, I., Kraker, P., Lex, E., Gumpenberger, C., & Gorraiz, J. (2016). Research data explored: An extended analysis of citations and altmetrics. Scientometrics, 107, 723–744. DOI: 

  34. Plum Analytics. (2018). Coverage: Expanding the world of altmetrics. DOI: 

  35. Rickenbach, M., Mohamed, A., Blanche, C., & Norland, E. (2013). Final report: Review of the McIntire-Stennis Cooperative Forestry Research Program. USDA National Institute of Food and Agriculture Institute of Climate, Bioenergy, and the Environment final report to USDA Forest Research Advisory Council. 22 p. 

  36. Roemer, R. C., & Borchardt, R. (2013). Institutional altmetrics and academic libraries. Information Quality Standards, 25(2), 14–19. DOI: 

  37. Roemer, R. C., & Borchardt, R. (2015). Chapter 4. Altmetrics and the role of librarians. Library Technical Reports, 51(5), 31–38. 

  38. Thompson, D. H., & Bullard, S. H. (2004). History and evaluation of the McIntire-Stennis Cooperative Forestry Research Program. Forestry and Wildlife Research Center, Research Bulletin FO 269, Mississippi State University, Starkville, Mississippi. 

  39. Vanclay, F., Esteves, A. M., Aucamp, I., & Franks, D. M. (2015). Social impact assessment: Guidance for assessing and managing the social impacts of projects. Fargo ND: International Association for Impact Assessment (p. 170). Retrieved June 9, 2020 from 

  40. Wilson, T. C. (2017). Rethinking digital preservation: definitions, models, and requirements. Digital Library Perspectives, 33(2), 128–136. DOI: 

  41. Wong, E. Y., & Vital, S. M. (2017). PlumX: a tool to showcase academic profile and distinction. Digital Library Perspectives, 33(4), 305–313. DOI: 

comments powered by Disqus