EarthCube Metrics Analysis Report#

Overview#

The EarthCube program has produced a wide array of scholarly assets, including papers, software, data and other project outputs. These outputs have accumulated since the inception of the EarthCube program. This project is focused on developing metrics related to these outputs. Publications are not the only way to measure the impact of a technology-focused program like EarthCube. EarthCube projects are often focused on tool and/or community development as the primary outputs. Scholarly metrics that focus on journal article publications provide a good starting point for EarthCube program metrics, but should be complemented by metrics that encompass other kinds of project outputs and outcomes.

The goal of this project is to support the EarthCube Office in assessing the impacts and reach of the EarthCube program within its target scientific communities. The project will enable better understanding of the contributions of the EarthCube community since the program began awarding grant funding. Additional goals of the project include to help drive community activity and decision making through metrics frameworks built with publications, software, data and other inputs. Through consistently reporting program scholarly outcomes, activities, and accomplishments, this project will inform historical understandings of the EarthCube program’s trajectory.

The scope of this investigation is to gather and analyze publications authored by EarthCube program grantees. The EarthCube Office provided a list of 78 NSF grant award IDs, which included projects funded by the EarthCube program between the years 2013-2021. Data and software produced by EarthCube projects were also included in the analysis if they were reported by the project PIs, and had been assigned Digital Object Identifiers (DOIs).

Methodology Overview#

Process Phase Overview Diagram

The analysis process involved three major phases:

  1. Data Collection

    Data was collected from a variety of sources, but the initial publication data was extracted from the EarthCube project NSF identifiers and used to extract publications from the NSF Awards database.

  2. Analysis

    Analysis of citations and altmetrics for the publications extracted from the NSF.

  3. Reporting

    The reporting covers individual, project and program level analysis from a citation analysis lens.

Detailed methodology can be found in the Methodology chapter of this report.

Limitations of this report#

Finding publications produced from funded projects is not a science. Despite the best efforts of funders to encourage awardees to submit detailed reports of their peer-reviewed activities, publications – and other outcomes that may qualify for or be amenable to minting DOIs – are not always reported. The publications represented in this report are not the entire set of publications produced by the funded programs. The majority of publications are likely represented (perhaps as many as 80-90%), but it is acknowledged that this is not an exact and complete representation of all publications produced by EarthCube funded projects.

About the report Authors#

Keith Maull and Matt Mayernik are staff members of the National Center for Atmospheric Research. Keith is a software engineer and data scientist at the NCAR Library, while Matt Mayernik is a project scientist and assistant director at the NCAR library.

This report was funded as a subaward to the University Corporation for Atmospheric Research (UCAR) by the Geoscience EarthCube Community Office (GECCO) awarded to University of California San Diego under NSF Award #1928208.

Citing this report#

Maull, K., and M. Mayernik, 2022: EarthCube Program Metrics Analysis 2013-2022. https://doi.org/10.5065/VMFJ-QY55.