Introduction
The Biobanking and Biomolecular Resources Research Infrastructure – European Research Infrastructure Consortium (BBMRI-ERIC) Directory collects information about biobanks throughout Europe as well as the biosample collections and data those biobanks make available for research (currently, over 400 biobanks are registered, collectively making available over 1400 sample collections)1. The data model of the BBMRI-ERIC Directory is currently based on the Minimum Information About BIobank data Sharing (MIABIS) Core 2.0 metadata schema and is focused on describing two principal entities: biobanks (institutions) and collections (within biobanks containing data/samples)2.
The European Clinical Research Infrastructure Network (ECRIN) Metadata Repository – MDR – collects metadata about clinical research studies and digital objects related to them3. The MDR is based upon the ECRIN metadata schema. It contains metadata from 800,000 clinical research studies (around 80% interventional clinical trials and 20% observational studies) and 1.3 million related digital objects, such as trial registry entries, journal papers, results summaries, protocols etc. The MDR covers all primary World Health Organisation (WHO) registries, ClinicalTrials.gov, Pubmed and two repositories (Yale University Open Data Access (YODA), Biologic Specimen and Data Repository Information Coordinating Center (BioLINCC)).
There is much value to be gained by enhancing the metadata in both these systems to allow linking of entities between BBMRI-ERIC and ECRIN in those cases where samples have been generated in the context of a registered clinical study. This linking would allow users of the MDR to easily identify studies that had associated samples, listed in the BBMRI-ERIC Directory, and users of the BBMRI-ERIC Directory to find information more easily about the studies that generated samples in collections of interest.
The benefit of linking clinical studies/clinical trials to biobanks and sample collections is outlined in the following user story. A clinical trial was performed to investigate whether an interactive internet-based intervention strategy, targeting vascular and lifestyle-related risk factors, can lead to improvement of cardiovascular risk profiles and prevention of cardiovascular diseases, and whether this in turn may prevent or delay the onset of cognitive decline and dementia4. This trial entitled “Healthy aging through internet counseling in the elderly” (sic) has been registered in a trial registry (ISRCTN48151589). After identifying this trial in the ECRIN metadata repository (MDR), several digital objects are presented by the system that are directly linked to this trial: the Statistical Analysis Plan, the Patient Information sheet, website and journal articles (n=6). In the trial registry and MDR there is no mention, however, of the biosamples collected in the context of this study. If a researcher is interested in such material and thus wants to identify and access a biobank collection, he/she must find it through other means; for instance, the researcher can search the BBMRI-ERIC Directory for collections linked to this study (bbmri-eric:ID:NL_AMCBB:collection:AB16-005) – making the best of the available metadata in the hopes of being successful. Currently, there is no direct and specific link between a clinical trial in the MDR and a biobank/collection in the BBMRI-ERIC Directory, so this search can be time-consuming and even difficult due to lack of metadata interoperability across the two repositories, potentially yielding different representations of the same entities or even simply different spellings, naming conventions etc. Likewise, if a researcher has identified a collection in the BBMRI-ERIC Directory and would like to find more specific information on the underlying trial, he/she must try to identify the trial through a manual search of the MDR, or the International Clinical Trials Registry Platform (ICTRP) or some other resource (e.g., PubMed), coping with difficulties due to the lack of standardisation across the various types of registries.
The situation would be simplified tremendously and become machine-actionable if the clinical trial and the related biobank collection were directly linked. This link can be established by adding the biobank collection ID to the MDR, and conversely adding the study identifiers in the MDR to the biobank collection in the BBMRI-ERIC Directory. Then, a researcher interested in a biobank collection found in the BBMRI-ERIC Directory would be able to also access the clinical trial directly in the MDR, and thus also access various digital objects and documents related to the trial – for example, the patient information sheet and a specific journal article. Conversely, a researcher that has identified a trial in the ECRIN MDR and wants to know whether any biobank collections are available will automatically have access to this information as well as a direct link to the collection’s page in the BBMRI-ERIC Directory.
Both systems, the BBMRI-ERIC Directory and the ECRIN MDR, already include much of the information required to link clinical studies and sample collections. In this work, we present the work performed to find and implement links across existing corresponding records in the two systems, charting a path to simplify the activity of the researcher in the previously described use case. At this stage, we have limited the work to COVID-related studies and collections. In the following sections, we describe the difficulties posed by the problem of identifying corresponding records and the procedure implemented to mine the two repositories in search for them; this process consisted in a preliminary analysis, which was performed to investigate the feasibility of a more extensive linking between the two systems, followed by an extensive systematic search for all corresponding COVID-19 related studies and biosample collections. We present the outcomes of the linking process applied to the MDR and Directory and, finally, we discuss implications and future prospects for these initial results.
Methods
The linking problem was considered as hard because usually no direct references between a clinical trial/clinical study in the MDR and an associated collection in the BBMRI-Directory are available. To cope with this, the linking process between MDR studies and related collections in the BBMRI-ERIC Directory was performed in two steps:
Two-stage linking process
The aim of the first step in our process was to determine whether direct links between collections in the BBMRI-ERIC Directory and registered clinical trials would be possible, and in how many cases. For this purpose, we implemented a two-stage linking process – one stage searching the BBMRI-ERIC Directory for candidate hits to try to link with MDR records, and a second stage searching the ECRIN MDR for candidate hits to try to link with Directory collections. In the first stage, we used the full-text searching capabilities of the Directory catalogue to search for a set of purpose-designed query strings in any data/sample collections. The full-text search script targeted all text fields of the catalogue; the code is available on GitHub (https://github.com/BBMRI-ERIC/directory-scripts).
The search terms used (applied with both left and right truncation) and then number of corresponding results returned from the BBMRI-ERIC Directory are summarised in Table 1 below.
Table 1. Search patterns used to scan the BBMRI-ERIC Directory in the two-stage linking process and corresponding number of search hits.
Search patterns were left and right truncated.
| No. | Search pattern | No. of resources
found |
|---|
| 1 | *random* | 7 |
| 2 | *blind* | 30 |
| 3 | *clinical trial* | 135 |
| 4 | *clinical stud* | 0 |
| 5 | *compar* | 84 |
| 6 | *investig* | 126 |
| - | Total | 382 |
The resources identified in the BBMRI-ERIC Directory with the search pattern “clinical trial” were investigated in detail (n=135). The free text pertaining to the record was assessed to explore whether a linked clinical trial could be identified. Main and specific keywords of the resource were entered into the ECRIN MDR search system. The descriptions of the clinical trials identified by the MDR were compared manually with the descriptions in the BBMRI-ERIC Directory. If in reasonable time (no more than 10 minutes), a unique link between the collection in the BBMRI-ERIC Directory and a specific clinical trial in the MDR could be made, the URL of the clinical trial registration was mapped to the collection.
The second stage of the preliminary two-stage linking process started with a search for candidate studies in the MDR, to then determine whether a linked biosample collection could be identified in the Directory. We began by identifying those studies in the MDR that had some reference to COVID-19, coronavirus, or SARS-2 in their titles or listed topics (n=15,763). 134 of these were removed because they had start dates prior to 2020 (and so were pre-pandemic), unless they explicitly mentioned COVID-19 in the title (in which case they are presumed to have been studies adapted to include COVID-19, or which had a wrong date entered accidentally). Identifying the distinct studies in this dataset (because some studies are entered into more than one trial registry, and thus may have multiple records) yielded a final global COVID set of 15,201. From these studies, a subset of 3,893 were selected by the presence of an acronym – because it was suspected (and later confirmed) that it would be more accurate to try and match acronyms than fragments of full titles. Out of that group, 630 were identified as being registered in a European, or largely European trial registry: i.e. the EU Clinical Trials Register (EU CTR), ISRCTN (International Standard Randomised Controlled Trial Number), German Clinical Trials Register (DRKS), and Dutch Trial Register. This group was seen as particularly relevant to BBMRI-ERIC, which mainly catalogues biobanks and collections within its member states, which are to a large extent located in Europe.
Systematic search of BBMRI-ERIC Directory
Once the preliminary analysis confirmed the linkage potential between the two metadata collections, we proceeded with the second, more thorough step in our process to find and link corresponding clinical studies and biospecimen collections. We performed a systematic search of the public BBMRI-ERIC Directory (i.e. all biobanks and collections). Using the collection name, and the description, it was relatively straightforward to identify collections that, at least potentially, were linked to clinical research studies (rather than, for instance, population- or diagnosis-based sample collections). In many cases, the title of the collection included a named study or related to a comparison of different treatments or cohort groups.
Our investigation demonstrates the difficulty in unambiguously linking registered sample collections to specific registered studies. To achieve our cross-linking goal while avoiding potential mismatch errors, we defined and executed the following procedure.
1. If a description in the Directory included a direct reference to a trial registry ID (a few do), that information was used to identify the study directly.
2. If a description in the Directory included a link to a web site or a journal paper (again some do), information that was used to retrieve a trial registry id.
3. Otherwise, an attempt was made to match key words in the BBMRI-ERIC Directory collection title or description with words in MDR titles. This was done manually using direct interrogation of the MDR database. In some cases, the BBMRI-ERIC description is the same or almost the same as the ‘full’ study title in the MDR (the scientific or protocol title). In other cases, the BBMRI-ERIC title was a study acronym, which very often needed to be disambiguated through further work. Note that the default MDR title for a study is the usually shorter ‘public title’, and so may not always obviously match the BBMRI-ERIC collection title.
4. Disambiguation of acronyms was also attempted by using the detailed description in the BBMRI-ERIC Directory, when available, and matching that to the study’s scientific title or description.
5. In some cases, disambiguation was also achieved by matching the sponsor of a study to the host organisation for the repository.
Results
Two-stage linking process
In order to get a first impression about the scale of possible links we focussed on the retrieval for “clinical trial”. This term is specific for describing interventional studies and was seen as a good starting point to find links between a collection in BBMRI-ERIC Directory and the related study in ECRIN MDR. From 135 identified resources in the BBMRI-ERIC Directory with the keyword “clinical trial”, 58 collections could be manually mapped to one of 37 COVID-related clinical trials5. For the remaining resources: no collection could be identified from the link identified through the search process in the BBMRI-ERIC Directory (n=42, the majority of these cases was linked to biobanks/collections in United Kingdom), no specific trial from several possible trial matches could be linked (n=23) or a trial could not be found in the MDR (n=12).
The search for linkable studies in the MDR provided a list with potentially identifying information to be used to search the BBMRI-ERIC Directory, or at least provided information that could be further processed before checking it against the Directory. The information consisted of study records with the attributes of (internal) MDR-ID, Registry ID or IDs, acronyms and the long title used for the trials. The attributes of the list of candidates from MDR was then used to scan a download of the full BBMRI-ERIC Directory metadata. The scan was done once with exact matching and then again with truncation or embedding within certain characters like e.g. “- , ; . ' :”. The results of this process are shown in Table 2.
Table 2. Results from preliminary search process starting from MDR candidates.
For each attribute, we report the number of selected MDR records for which we found a text-based match in the Directory metadata.
Attribute
name | Number of search hits from Directory collection metadata |
|---|
| Registry-ID | 0 |
| Acronym | 2530 (This included lots of false positives, as anticipated, due to use of commonly used, rather general
terms, like ‘achieve’, ‘act’, ‘COVID’, ‘trust’ etc.; also acronyms are often not created in an unambiguous way) |
| Title | 2 (Small number expected as searching with rather long phrases including many words) |
The resources identified in the BBMRI-ERIC Directory were compared with resources in the MDR to identify possible matches between clinical studies/clinical trials and biobanks/collections. Only 7 studies were identified unambiguously5.
Systematic search of BBMRI-ERIC Directory
The results of the systematic search of Directory, after some de-duplication and removal of what appeared to be spurious ‘placeholder’ collection entries, was the identification of 202 studies, spanning 284 collections5. The linkage data (along with related information such as collection and biobank IDs, MDR IDs, and sample details) have meanwhile been imported into ad hoc prototypes of both the MDR(search by linked data object = “available samples”) and a functionally extended version of BBMRI-ERIC Directory to showcase the links.
Discussion
The FAIR guiding principles for scientific data management and stewardship state in its section on interoperability that data needs to be integrated with other data (I3 from 6). The linking across identifiers has been investigated in different projects (e.g., THOR, FREYA) and conceptual models have been developed7,8. The different approaches to how data should be interlinked is described in detail in the FAIR cookbook9. In the ECRIN-BBMRI use case, mapping is performed between databases containing different information about the same entity, here a clinical trial/clinical study. In the MDR a clinical trial/clinical study is identified by a study identifier (sometimes there are several study identifiers) and in the BBMRI-ERIC Directory usually indirectly, e.g. by mentioning in free text in the description or other metadata fields or sometimes by direct reference to a clinical trial/clinical identifier. Consequently, direct mapping was not possible in most cases and considerable manual input was needed.
The results of this exercise show that quite a substantial number of collections in the BBMRI-ERIC Directory were created in the context of specific clinical trials. Considerable time with manual work was needed to do this linking in an example dataset. Therefore, the retrospective approach taken in this study is only ever likely to be useful as a demonstration of the potential usefulness of linking the two systems. To have linkage more realistic and complete, it will be necessary to include that linkage (i.e. a clinical trial identifier) within the metadata of the biobank collections at the time when metadata are created in the process of onboarding to the BBMRI-ERIC Directory. It is estimated that there are approximately another 100 studies / trials in the MDR for which corresponding studies exist in the Directory, but which could not be unambiguously linked because of insufficient information. The full extent of the overlap of the two systems is therefore – currently – estimated at about 300 studies with more than 300 collections involved. Doing the aforementioned matching also took several days. So, given the time required, this is clearly a one-off exercise that needs to be replaced by an (semi-) automatic process as soon as possible. The size of the retrospective linking that was missed in our study is unclear. A systematic approach should start from all collections to identify possible links to trials. But the less specific the terms are (e.g., "investig"), the smaller the linkage rate will be. So, the effort increases with decreasing success rate. This would mean a lot of effort with limited success. Whether AI or ML techniques could help needs to be sorted out.
The retrospective analysis indicated that links between the BBMRI-ERIC and ECRIN collections exist, but also that they would be difficult to continuously identify and maintain without a great deal of manual work – which neither organisation could support. The question is then whether, in the future, systems could be put into place to make the exchange of information and the mapping of identifiers almost automatic. To do that, there is first a need to clarify exactly what information would be required by each system from the other. That will determine how the metadata schemas used by the two systems will need to be extended or adjusted, to cover potential links between them. Each ERIC’s data collection processes need to be extended, where possible, to include the new metadata right from the beginning. Populating the relevant metadata, in the source data used by both systems, is the final step – though perhaps the most difficult. First ideas and concepts how to use and eventually extend the metadata schemas of the ECRIN MDR and the BBMRI-ERIC Directory to support linkage between clinical studies and collections that have been created within a study as well as possibilities for a prospective implementation of such links are under discussion.
