KC1 Overview for FullStacks

DCPPC-DRAFT-#: 2

DCPPC-DRAFT-Title: KC1 Phase 1 Overview for Full Stacks

DCPPC-DRAFT-Type: Informational

Name of the person who is to be Point of Contact for the DCPPC-DRAFT: Avi Ma'ayan, Stanley Ahalt

Email of the person who is to be Point of Contact for the DCPPC-DRAFT: Avi.Maayan@mssm.edu, ahalt@renci.org

Submitting Team: KC1

Requested DCPPC-DRAFT posting start date: 07/16/2018

Date Emailed for consideration: 07/14/2018

DCPPC-DRAFT-Status: Active and open for comment

URL Link to this Document: https://docs.google.com/document/d/19-0NtTutSoe6T9XkDAFcIdH7fEDMGVW69OIqCUh0D-M/edit?usp=sharing

License: This work is licensed under a CC-BY-4.0 license.

KC1: Phase 1 Overview for Full Stacks

An informational overview of the KC1 recommendations, standards, and technologies aimed at the Full Stacks for Phase 1

The FAIR principles layout a path to augment the potential for discovering and reusing existing digital resources as well as those produced by the NIH research community. An important consideration, therefore, is to assess the degree to which digital resources in the Commons are FAIR, and to provide constructive feedback to increase their FAIRness.

To achieve the goal of performing FAIR assessments, members of KC1 designed, developed, and released FAIRshake. The FAIRshake APIs are compatible with existing API standards: Swagger, OpenAPI and SmartAPI. FAIRshake enables the interplay of a number of existing, new, and future applications to assist the users in the FAIR assessment of various digital objects, as well as to improve their FAIRness.

FAIRshake is available at:

https://fairshake.cloud and the SmartAPI of FAIRshake are documented at http://smart-api.info. More general documentation for various aspects of FAIRshake can be found at: https://fairshake.cloud/documentation/. The documentation provides information about JSONSchema for Automated Assessments, and examples of how to use the FAIRshake API. In addition a tutorial Jupyter Notebook was created to demonstrate how to interface with the FAIRshake API.

Although the specific information flow between two or more applications that perform FAIR assessment depends on the type of digital object being assessed (e.g. dataset, repository, protocol, tool, or standard) and its location, FAIRshake generically enables an application to:

1. Initiate a FAIR assessment project. A Project is a collection of digital objects and their assessments, for example, manual assessments of Alliance of Genome Resources (AGR) tools. It is a way to form a collection of digital objects and their assessments by any practical unifying theme.

2. Associate a list of digital objects that need FAIR assessments with a project.

3. Associate each digital object with one or more rubric that should be used to perform the assessments. Rubrics are collections of questions/metrics to assess FAIRness.

4. Submit a FAIR assessment question/metric to a repository of questions/metrics and rubrics. Each question/metric is assigned an identifier and a version number. An initial set of Universal FAIR metrics have already been established to assess compliance to the key elements of the FAIR principles (http://fairmetrics.org). Additional metrics have been defined to reflect the expectations of the full stacks with respect to the digital objects contained within the Data Commons and other projects.

4.1. Compliance with community standards

An important aspect of FAIR is that the publication of digital objects follows community best practices. As such, FAIRness-related questions should be answered by providing URLs that demonstrate adherence to a community expectation, for example, the adherence to a metadata standard, or having a license that describes the terms of use. FAIRsharing.org provides URLs and DOIs for documents that describe recognized standards for biological and biomedical data management, as well as additional standards for identifying, citing, and representing digital objects used by the Data Commons. Hence, answers to FAIRness questions/metrics can simply point to entries in FAIRsharing, semi-automated assessments are used to do this automatically if the DOI registered by FAIRsharing is a registered url. If new standards are developed, these should be deposited in FAIRsharing.org.

5. Submit a rubric (a collection of question/metrics) to a repo of questions/metrics and rubrics. Rubrics in FAIRshake have landing pages and are required for performing a FAIR assessment of a digital object. Rubrics enable questions/metrics reuse while providing a way to accommodate unique needs of different communities.

6. Perform the FAIR assessment. Using a specific rubric, a user may perform a FAIR assessment on any digital object that is associated with that rubric. An answer must be given for each question/metric in the rubric. This assessment can also be made as part of a specific project.

6.1. Registration of digital objects

To ensure that the resulting assessments (output) are transparent and can be verified against the provided information (input), digital objects should be described in relevant registries, such as FAIRsharing (for databases, knowledge bases, repositories, standards and data policies). The implementation of the FAIRshake API allows applications to access and use this registered information in the assessment process.

7. Retrieve the FAIR assessment scores. The returned scores are made available in a form that allows the digital object producer, or any web service or registry that lists the digital object, to display the score as an insignia. The FAIR scores are also used to perform statistical analyses of collections of assessments (Projects). Alongside the scores, the number of assessments, manual or automated, are provided. This number can impact how one interprets a score, for example, scores from 2 people may have biases that from 2000 people.

The set of these APIs do not dictate which rubrics to use. This provides flexibility for the full stacks to apply any rubric that fits their implementation, different rubrics co-exist in FAIRshake. We hope that a set of well formulated rubrics and metrics will naturally emerge. We can provide expert advice about useful questions/metrics and rubrics. We also provide a set of initial questions/metrics and rubrics that we developed, for example, the questions/metrics from FAIRmetrics.org or others listed in FAIRshake under the Rubrics tab.

Automated vs. Manual Assessments. The FAIRshake APIs support both manual, semi-automated and automated assessments. The system is agnostic to whether the assessment was automated or performed manually.

APIs

We developed a set of REST-based APIs to enable users to create, modify, delete digital resources corresponding to various aspects of the FAIRness assessment process. These are well documented with examples under the FAIRshake documentation page.

1. Start a FAIR Assessment Project

FAIR Assessment Project API

Interface to search, create, modify and delete a FAIR Assessment Project

Allowed HTTP operations: GET, POST, PUT, DELETE

Only users that create projects can change or delete projects. Users can assign author privileges to other users.

Data attributes

  • Project URLs (string, newline delimited)

  • Title (text)

  • Description (text, optional)

  • Image (text, optional)

  • Tags (text, space-delimited optional)

  • Authors (URI) - e.g. ORCID or software id

  • Associated Digital Objects (List of URIs)

2. List of Digital Objects that need FAIR assessment

FAIR Digital Object Store API

Interface to search, create, modify and delete a Digital Object capable of being assessed for FAIRness

Allowed HTTP operations: GET, (optional, POST, PUT, DELETE)

Data attributes

  • Digital Object URLs (string, newline delimited)

  • Title (text, optional)

  • Description (text, optional)

  • Tags (text, optional)

  • Type (controlled vocabulary)

  • Associated Rubrics (list of software ids)

  • Associated Projects (list of software ids)

  • Authors (URI) - e.g. ORCID or software id

  • 3. Request Assessment

FAIR Request Assessment API

Interface to request a FAIRness assessment from a server capable of performing one (manual or automated)

Allowed HTTP operations: GET, POST, PUT

Data attributes

  • Digital Object URI (string)

  • Rubric URI (string)

  • Project URI (string, optional)

  • Methodology (controlled vocabulary) - manual, automated, crowdsourced, etc

  • Requestor (URI) - e.g. ORCID or software id of user who triggered the request

  • Request timestamp (timestamp)

  • Callback (URL) - the place where the results will be sent

4. Submit Assessment

FAIR Assessment API

Interface to store the results of a FAIRness assessment

Allowed HTTP operations: GET, POST, PUT

Data attributes

  • Digital Object URI (string)

  • Rubric URI (string)

  • Project URI (string, optional)

  • Methodology (controlled vocabulary) - manual, automated, crowdsourced, etc

  • Requestor (URI) - e.g. ORCID or software id of user who triggered the request

  • Assessor (URI) - e.g. software id assessment API that made the assessment

  • Answers (metric -> answer (serialized), key/value)

  • Assessment timestamp (timestamp)

5. Submit Question/Metric

Metric API

Interface to search, create, modify, and delete FAIR metrics

Allowed HTTP operations: GET, POST, PUT, DELETE

Data Attributes

  • Metric URLs (string, newline separated)Type (controlled vocabulary)

  • Title (text)

  • Description/Rationale (text)

  • Tags (text, optional)

  • Authors (URI) - ORCID

  • FAIR principle it applies to (controlled vocabulary)

  • Reference to FAIRmetrics ID on GitHub if applicable

  • License (URL)

6. Submit a Rubric

Rubric API

Interface to search, create, modify, and delete a collection of metrics (aka rubric)

Allowed HTTP operations: GET, (optional, POST, PUT, DELETE)

Data Attributes

  • Rubric URLs (string, newline delimited)

  • Type (controlled vocabulary)

  • Title (text)

  • Description (text)

  • Tags (text, optional)

  • Authors (URI) - ORCID

  • Metrics (list of Metric URIs)

  • License (URL)

7. Retrieve FAIR assessment score

FAIR Assessment Score API

Interface to aggregate FAIR Assessments

Allowed HTTP operations: GETData Attributes

  • URL (string)

  • Project URI (string)

  • Target URI (string)

  • Rubric URI (string)

  • Assessor (URI) - ORCID

  • Methodology (controlled vocabulary)