1 Introduction
This paper introduces a novel methodology based on network analysis techniques for assessing the long-term impact of EU-funded research projects on companies, academic institutions, and research organisations. The application of network analysis tools to the innovation networks derived from collaboration to research project is not new and it has been used to study: topological properties of collaboration networks1; the relation between network structure and the EU broad policy objectives2; innovative dynamics at country-level3,4; technological diversity5,6 proposed the use of a rank-size approach for clustering complex networks derived from the collaboration in European project at organisational level. The novelty of the present application lies in its advanced data preparation using the European Science Vocabulary7 or EuroSciVoc taxonomy, the incorporation of weighted networks, the application of community detection algorithms and temporal analysis to track collaboration over time.
To demonstrate the practical application of the methodology, the paper includes a case study on hydrogen research and innovation, focusing on the development of hydrogen valleys and specifically the North Adriatic Hydrogen Valley project (NAHV).
Section 1 introduces the concept of hydrogen valleys and outlines the key research questions driving the study: Are there significant and stable communities within these networks? Which organisations assume leading roles? Do these leaders establish enduring communities, or do they primarily act as connectors between different groups? Addressing these questions requires a complex approach, employing network analysis and tackling challenges such as input-ordering bias and result variability, as detailed in Section 2 (Methodology).
Section 3 presents a case study focused on the development of hydrogen-related research and the current support to hydrogen valleys, using data from Horizon 2020 and Horizon Europe projects from 2015 to 2029. The results allow to identify organisation with leading roles, a community structure, and its evolution over time.
1.1 Hydrogen valleys
A “Hydrogen Valley” is a geographical area where several hydrogen applications are combined into an integrated ecosystem that produces, exchanges, and consumes a significant amount of hydrogen, covering the entire hydrogen value chain: production, storage, distribution, and final use8.
Currently more than 80 different hydrogen valleys are already ongoing or under development in Europe and in USA9. However, due to their relatively recent constitution10 few studied have been developed to map the hydrogen valley impact on a territory, and its development over the years. This small number of studies has been also assessed in 2023 by a Polish team11 by applying the bibliographic method and a quantitative analysis of the collected publications. The researchers identified 284 publications dealing with hydrogen valleys, with a systematic increase in the number of cited papers. Furthermore, from a deeper analysis, they found that the most frequently cited publications are studies presenting different types of technological solutions and concepts. This study indicated the existence of a significant research gap in the field of research on the creation and development of Hydrogen Valleys. Among the several, some other studies have been focused on the techno-economic assessment of green hydrogen valley12 providing multiple end-users and their LCA impact13. For example, the South Africa Hydrogen Valley report14 has examined the socioeconomic impact from the Hydrogen Valley project across multiple dimensions in terms of GDP, job market and tax revenue.
The North Adriatic Hydrogen Valley or NAHV project15 is one of the first transnational hydrogen valleys developed in Europe. It embraces the EU territories of Croatia, Slovenia and the Italian region Friuli Venezia Giulia and involves 37 partners based mainly on those countries. The project is co-financed by Horizon Europe programme and supported by the Clean Hydrogen Partnership16.
NAHV has been planned under the willingness to activate a long-lasting hydrogen economy. The heuristic structure of the NAHV is based on a systemic vision and the development of a holistic model of the valley. As showed in Figure 1, this heuristic approach considers not only innovation related to vertical “hard pillars” (which are typical of a hydrogen valley, such as production, transport, storage and distribution of hydrogen) but also to the ”soft pillars“, which are regulatory framework, training and capacity building and involvement of civil society. Equally important is the level of cooperation among the various actors within the valley. NAHV aims also to develop and test a pathway of industrial transformation linked to green transitions17 that will produce system innovation leading to a radical change of routines, and transform economic activities and industry ecosystems.
Figure 1. A schematic representation of the North Adriatic Hydrogen Valley project.
Testbeds are primarily driven by industrial partners, while collaboration with policymakers, universities, and research organisations is crucial for enabling innovation.
1.2 Collaboration-driven innovation in Hydrogen Valleys
Hydrogen valleys are innovative ecosystems focused on hydrogen technology, leveraging regional and national initiatives to develop sustainable energy solutions. These ecosystems align well with the Quintuple Helix innovation model, which involves collaboration between five key actors: academia, industry, government, civil society, and the environment. By looking closer the hydrogen valley modus operandi, it is possible to observe that it is based on two main paradigms: the open innovation model18, where the exchange of know-how among the several actors of the valley is stimulated and (b) the quintuple helix19 model that is a model in which innovation is pushed by the integration in the classical triple helix model of the needs expressed by the civil society and media and by a targeted effort to solve a specific challenges20. In the context of Hydrogen valleys, the primary challenge is typically addressing climate change by driving the decarbonization of the economy. According to this vision a trans-regional innovation ecosystems21,22 is needed to stimulate the interaction, creativity and innovation and to develop a strong and inclusive macro-regional ecosystems that support business growth, as well as the entrepreneurial environment necessary to foster more innovative regional economies23 and to increase competitiveness of small- and medium-sized enterprises (SMEs). To reach this wide and ambitious scope, it is necessary to put in action a combined strategy24 based on:
Developing Research, Development, and Innovation (R+D+I) within the Hydrogen Valley ecosystem to foster collaboration and identify synergies that can be effectively implemented.
Establishing a common development framework to enhance the business ecosystem’s capacity. This includes encouraging the creation of new businesses and attracting investment to address gaps in the hydrogen value chain, particularly by supporting new ventures25.
Innovation, in this context, is inherently driven by collaboration, particularly through partnerships between industry and academia, which combine diverse approaches and cover the entire Technology Readiness Level (TRL) scale. However, a quantitative understanding of collaborations and their evolution over time remains elusive. This paper seeks to address this issue by providing a methodology to identify and gauge collaboration, offering insights into the mechanisms that support innovation and the dynamics of innovation ecosystems.
From the perspective of individual organizations, understanding the dynamics of innovation at the regional or international level can be a good opportunity. Consider, for example, an organization involved in a new hydrogen valley project: it must not only focus on the development of its own project activities, but it may benefit from identifying partners and competitors that are engaged in other hydrogen valleys. This broader awareness may help identifying synergies, risks, and market opportunities.
Policymakers adopt yet another perspective. They have a strategic vision to align local, national, and European resources, aiming for medium- and long-term impact of the innovation ecosystems. Achieving this requires a clear understanding of how collaborations emerge between industry and research, who the key stakeholders are, and how these collaborations evolve over time.
This paper addresses the elusiveness of innovation proposing a methodology that is suitable for two approaches: project-specific (e.g., examining the NAHV project to gain deeper insights into its role within the broader landscape of EU-funded hydrogen research) and a policy-oriented (e.g., analysing how Horizon grants are shaping competitive ecosystems in hydrogen sector). In both cases, two research questions are essential:
1. Which organisations are the most influential in driving research and innovation? Identifying these key players — whether companies, academic institutions, or research organisations — is important for other organisation that want to be in contact with them in future projects. Moreover, policy makers may be interested in assessing whether the policy put in place in their territory has produced an improvement of leadership roles over the years.
2. Are EU-funded projects fostering partnerships that extend beyond the duration and scope of individual projects? If such long-term communities exist, they serve as vital indicators of the open innovation and quintuple helix models effectiveness. Stable communities suggest robust exchanges of ideas and collaboration, which are essential for sustaining innovation and achieving the long-term goals of hydrogen valleys.
The methodology presented in Section 2 aims to provide quantitative answers to the research questions above. Key requirements of this approach are that results must be independent of contingent factors (such as software implementation or ordering of the input data) and tested for validity. Additionally, when multiple algorithmic options are available, the selection must be data-driven and performance-oriented, ensuring that the chosen algorithm yields the most interpretable and reliable outcomes.
2 Methods
The proposed methodology builds on well-established network analysis techniques, such as centrality measures and community detection, while incorporating novel elements to enhance the stability and quality of results, and to produce a temporal analysis. Notably, a new method for exploring the solution space is introduced, along with a comprehensive quality check process to select the most appropriate algorithm, validate the results, and manage input-ordering bias. When multiple solutions emerge, a consensus community detection approach is employed to consolidate them into a single, coherent outcome.
The methodology foresees 6 main steps: data acquisition further discussed in Section 2.1, data preparation and enrichment (Section 2.2), network analysis (Section 2.3), centrality measures (Section 2.4 community detection (Section 2.5), and temporal analysis of communities (Section 2.6). An overview of the process is provided in Algorithm 1
Algorithm 1. Methodology for analysing research collaborations using CORDIS data
1: Data acquisition:
2: Access CORDIS and download all files related to Horizon 2020 projects
3: Access CORDIS and download all files related to Horizon Europe projects
4: Merge corresponding tables for Organisations O*, Projects P* and Topics T*
5: Enrich data with AI generated keywords and categories
6: Select relevant topics T ⊂ T*
7: P ⊂ P*(T) subset projects by selected topics
8: O ⊂ O*(P) subset organisations that are involved in selected projects
9: Calculate weights matrix W
10: Data preparation:
11: y1 ← min(project start year ∀p ∈ P)
12: y2 ← max(project end year ∀p ∈ P)
13: for y = y1 to y2 do
14: for each project p ∈ P do
15: fp,y ← (duration of p within year y)/365
16: for each organisation o involved in project p do
17: W(o, p, y) ← fp,y* total cost of p
18: end for
19: end for
20: end for
21: for y = y1 to y2 do
22: Ay ← Wy Calculate adjacency matrix for one-mode network
23: Gy ← network nodes and edges from Ay
24: Calculate centrality measures of Gy and associate values to each organisation
25: Community detection Cy ← a valid and stable partition of Gy
26: end for
27: Temporal Analysis
28: for y = y1 to (y2 – 1) do
29: for ci ∈ Cy1 do
30: for cj ∈ Cy1+1 do
31: assess correlation between ci and cj as continue, split, merge, mix, disjoint
32: end for
33: end for
34: end for
35: Generate global community labels
36: Output: community labels and centrality measures associated to each organisation o ∈ O
2.1 Data acquisition
The source of data for this study is CORDIS (Community Research and Development Information Service), the European Commission’s primary public repository and portal to disseminate information on all EU-funded research projects and their outcomes26. It provides open access to comprehensive data on projects, including objectives, participants, funding details, and results. For the purpose of this study, two datasets have been extracted: Horizon 2020 (projects starting from 2014 to 2020)27 and Horizon Europe (projects starting from 2021 to 2027)28. The two datasets have the same structure, composed of several tables, the most relevant being the organisations table denoted as O* and the projects table denoted as P*.
Each project is characterized by a unique identifier (projID), its acronym, title, start date, end date. Moreover, each project is associated with a long textual field, describing its objectives, and with structured categorical information on the call and funding scheme. A separated table associates each project with one or more topics, encoded according to the European Science Vocabulary (EuroSciVoc), the multilingual taxonomy representing scientific fields developed by the EU’s Publications Office.
There is a many-to-many relationship between organisations and projects (i.e. a project has several organisations, and an organisation can be part of several projects). In table O* each organisation is identified by a unique identifier (orgID), a name, detailed geolocation information. In addition, the table records the role of each organisation in each project (coordinator, participant, or associated partner), as well as the total costs incurred by the organisation within each project (totalCost) and the corresponding eligible contribution from the Horizon programme (netEcContribution).
Data representation can be optimized by merging the corresponding tables for Horizon 2020 and Horizon Europe, to obtain the following core data structures:
table P* listing all projects, their projID, start and end dates, reference to the Horizon calls for proposal, keywords, and a long textual descriptions of project objectives;
table O* listing all organisations, their orgID location, their role in the project, and monetary values totalCost and netEcContribution;
table T* associating each project in P to one or more keywords of the EuroSciVoc taxonomy.
2.2 Data preparation
The initial step in data preparation involves identifying the subset of projects P ⊂ P* on which we aim to focus, based on the topics from that are relevant for the objectives of the analysis. Consequently, the other tables can be filtered to ensure that only pertinent data is retained in the form of O ∈ O* (organisation name and location).The core information for our analysis the weights table denoted as W, which encodes the annual effort each organisation o ∈ O puts into each project p ∈ P in a given year. A proxy for the values in W can be derived from either netEcContribution or totalCost. The former indicates the total amount of public funding received by an organisation upon project completion and serves as a useful proxy for how much the Horizon grant promotes collaboration. The latter reflects the total cost incurred by the organisation to complete the project. For non-profit organisations involved in research projects, these values often coincide, as the Horizon grant may cover 100% of the costs. However, for private companies investing in pilot projects, such as in the NAHV project, the two values can differ significantly, sometimes by an order of magnitude. In the case study presented in Section 3, the weight is based on netEcContribution, expressed in thousands of euros.
The data is then segmented on a yearly basis. Denoting by Wy the matrix for a given year y, it is a rectangular matrix of size |projID| × |orgID|. The value of each project is assumed to be equally distributed over its duration, from its start date to its end date. Consequently, the contribution of a project to the matrix Wy is divided proportionally across the years during which the project is active.
Textual fields in table P contain valuable information, but in a format unsuitable for network analysis. The project objectives are typically described in hundreds of words, and user-defined keywords lack a standardized vocabulary. To make this information usable, the last step in data preparation involves creating Boolean attributes for each project. These attributes encode relevant information in a simplified format, such as whether the project is focused on technology or market uptake, or whether hydrogen is a primary focus or one of several applications. This was accomplished using a script that interacts with the API of a Large Language Model, which processes the lengthy textual fields and generates a dataframe, indexed with projID and one or more Boolean variables, which can be merged with P.
2.3 Network analysis
The first step in the analysis is to build a network, by identifying its nodes, edges, and weights in the form of an adjacency matrix.
A network (or graph) is a set G = {V, E} composed of nv = |V| vertices and ne = |E| edges. For the purpose of our analysis, a one-mode network would be a suitable model, in which nodes represent organisations and edges encode the information about projects. However, in our dataset W is a rectangular matrix of size |projID| × |orgID|, which generates a two-mode network, i.e., a network with two distinct sets of nodes where connections are established only between nodes from different sets (e.g., organisations connected through joint participation in projects). A one-mode network G can be obtained from a two-mode network and described by a square matrix, denoted as A = WTW, where WT is the transposed of W. In our case, A is symmetric, and G is consequently an undirected graph. Figure 2 illustrates the transformation process from the two-mode network of organisations by projects to the one-mode network consisting only of organisation-by-organisation ties.
Figure 2. Schematic view of a project as two-mode network (left) and as one-mode network (right).
Uppercase letters A, B, C, D, E represent organisations. The green square denoted with P represents a project. Edge width is proportional to the weight, i.e. the value each organisation brings to the project. The sum of weights in the two-mode network is equal to the sum of weights in the one-mode network.
Since the data is calculated on a yearly basis from 2015 to 2029, the complete model consists of set of networks 𝒢 = {G2015 . . . G2029} that represents collaborations between organisations within Horizon research project.This spans from the launch of the first projects in 2015 to the anticipated completion of the longest ongoing Horizon Europe project in 2029. The generic network is denoted as Gy where the subscript y refers to a specific year.
2.4 Centrality measures analysis
The network structure can be evaluated using centrality measures—degree, strength, and coreness—to determine the importance and influence of nodes, each offering distinct insights.
The degree of a node v, denoted as deg(v) is the simplest centrality measure, calculated as the number of edges incident to that node. In this context, the degree reflects the number of projects in which an organization is involved during a given year. A higher degree indicates that the node plays a more locally central role within the network, as it has a greater number of direct interactions with other nodes; in the context of our research, a node with a high degree represents an organisation that is, or has been, a partner in many large projects, reflecting its extensive collaborative involvement.
Strength of a node is the sum of the weights of the edges incident to that node: s(v) = ∑u∈N(v)wvu where N(v) is the set of neighbours of v, and wvu represents the weight of the edge between nodes v and u. In weighted networks, strength provides a more nuanced measure of a node’s connectivity: with reference to Figure 2, all nodes have the same degree, but for example s(C) ≫ s(A). In this context, the strength reflects the monetary value of the projects in which an organization is involved during a given year.
The k-coreness (or coreness)29 of a node is a measure of the node’s position within the network’s hierarchical structure, based on its connectivity. Specifically, a node has a k-coreness of k if it belongs to the k-core of the network. The k-core is a maximal subgraph in which every vertex has at least degree k, i.e. within this subgraph, each node is connected to at least k other nodes. In this context, coreness can be interpreted as the capacity of an organisation to partner with other organisations that, in turn, possess the same level of collaborative capacity.
Centrality measures are computed individually for each Gy ∈ 𝒢 and saved as attributes of the nodes in Gy. This allows to track changes and compare the network structure across different years, providing insights into the dynamics of the collaborations and the shifting roles of organisations over time.
2.5 Community detection
Community detection is a crucial step in the analysis, as it helps to understand the role of an organisation within the network. A community C is defined as a subnetwork of G that satisfies a condition: nodes that belong to C are more densely connected within each other than with the rest of G. A partition 𝒫 is a set of k disjoint subnetworks C1, . . . , Ck whose union is equal to G.
A community detection algorithm 𝒜(G, ρ) → 𝒫 is a function that takes as input a graph G and one or more parameters ρ, and returns a partition 𝒫. Several community detection algorithms are discussed in literature30,31. Ideally, any algorithm should produce a single, valid partition each time it is applied with the same parameters. In practice, however, for large, dense networks, several issues may arise. Some community detection algorithms may produce invalid results such as communities that are internally disconnected or fail to meet the basic criterion of being more densely connected within than with the rest of the network. This issue should be addressed with a thorough quality check on 𝒫. A less known issue is the Input-Ordering Bias: Although networks are mathematically non-ordered entities, their implementation in any software algorithm is inevitably ordered (in the form of a list of edges, or a matrix as 𝒜 mentioned above). The order in which nodes and edges are stored in the practical implementation of the network can affect the results, as illustrated in 32.
A more elusive issue is the variability of results. When 𝒜 is based on a heuristic or randomized approach to identify the optimal partition, it may yield different results each time they are executed: 𝒫i ≠ 𝒫j. All these issues can be addressed as per Algorithm 2, based on the repeated execution of 𝒜(G*, ρ), where G* is shuffled version of Gy to reduce the effect of input ordering bias) and to explore the solution space. Specifically, the solution space 𝕊 = {𝒫1, 𝒫2, . . . , 𝒫ns} is the set of all unique partitions that 𝒜 produces across t trials. Our research aims to analyse 𝕊, and to determine the minimum number of trials tc required to confidently assert the completeness of 𝕊.
After selecting the most suitable algorithm, the analysis may yield either a single, ideal solution or a dominant solution—one that is more frequently observed than others. In these cases, this solution can be identified and analysed. However, if no single solution emerges as dominant, a ”consensus community detection approach” can be applied to consolidate the variable results into a single, reliable solution. This process is illustrated in Algorithm 2 and discussed thoroughly in 32.
Algorithm 2. Consensus Community Detection
1: for t = 1 to tmax do
2: G* ← random permutation of Gy
3: 𝒫t ← A(G*, ρ)
4: if 𝒫t is identical to another partition in 𝕊 then
5: Update frequency count in 𝕊
6: else
7: Append 𝒫t to 𝕊
8: end if
9: end for
10: Identify a single solution from the solution space
11: if a single or dominant solution exists then
12: 𝒫y ← 𝒫t
13: else
14: 𝒫y ← CCD(S) ▷ Apply consensus community detection on 𝕊
15: end if
Knowledge about the solution space, along with the awareness that a single or dominant solution exists, emerges gradually as a result of an experiment of repeated trials 𝒜(G, ρ) → 𝒫i, as illustrated in Figure 3. In the example, the probability of solutions in 𝕊 are represented as point estimates (solid line for ) and intervals (ribbon between (plower and pupper). A prevalent solution (shown in red) emerges soon; as t increases additional solutions are discovered and beyond t = 50, the probability distributions are virtually unaffected by any new solutions. When the experiment is repeated, a slightly different situation may be observed, but convergence towards a dominant solution consistently occurs.
Figure 3. Confidence intervals associated with different solutions in the solution space.
Probabilities associated with each solution is 𝕊 calculated under a Bayesian framework, using a Beta Binomial model: this ensures a thorough yet efficient analysis, and optimizes computational resources.
2.6 Temporal evolution of communities
As introduced in Section 1, it is important to provide insights into the underlying dynamics of collaboration over time, which can be achieved by tracking and categorize the evolution of communities over time. The information is represented as a family of networks, denoted as Gy where each network corresponds to a specific year. Within each yearly network, communities are identified, and our primary objective is to understand how these communities evolve over time. The method involves comparing each community Ci ∈ Gy with each Cj ∈ Gy+1. Through this comparison, we determine whether Ci and Cj are disjoint or have a non-null intersection. If they intersect, the relationship between the two communities is further classified as either continuing (i.e. Ci shares most of its members with Cj ), or as part of a merge or split. To ensure accurate tracking across different years, we assign global community labels that remain consistent for identical or continuing communities.
3 Results
This section applies the methodology outlined in Section 2 to a subset of hydrogen-related projects, including the NAHV project, which is the focus of this study; the data encompasses projects from Horizon 2020 and Horizon Europe, covering the period from 2015 to 2029. The analysis aims to investigate leadership roles, community structures, and their evolution over time. Furthermore, the analysis demonstrates how AI-generated keywords can be integrated to provide additional insights into the distinction between market-oriented and technology-oriented projects.
The data has been prepared using the EuroSciVoc topic hydrogen energy, and the AI-generated categories are market vs technology, as explained in Section 2.2. The family of networks {G2015 . . . G2024} has a remarkable evolution over time. G2015 is composed of 83 organisations, and the number steadily increases to 648 by 2024. The number of collaborations, represented by the network’s edges, grew proportionally during this period, and the total value of projects, measured by the sum of weights in the network, saw an even more significant increase, rising from 547 to 6628. Looking ahead to the period from 2025 to 2029, the future of collaborations will be shaped by contracts currently in operation and influenced by upcoming calls and new projects. As shown in Figure 4, after 2024 the number of partners and collaborations decreases, while the total investment remains stable for the next two years.
Figure 4. Size of the networks Gy represented by the number of organisations (horizontal axis) and the number of collaborations (vertical axis) and total investment per year (bubble size).
Between 2016 and 2024 there is an increase in the number of organisations from 83 to 648 and a proportional rise in collaborations as represented by weights, rising from 547 to 6628. Current data for projects continuing in years 2025 to 2029 suggest further growth in the total value of projects.
3.1 Leadership roles
The methodology for AI-generated categories, discussed in 2.2, is applied to the hydrogen projects dataset. This classification approach distinguishes between projects focused on market development — encompassing policy, market uptake, business models, and hydrogen valleys — and those centred on technological development. The NAHV project clearly falls into the former category. This classification enables the creation of two distinct families of networks, denoted as and . The total value of the projects in each group is shown in Figure 5, revealing an interesting trend: while investment in market-oriented projects has been consistently present since 2016, there is a notable increase in such investments starting in 2023.
Figure 5. Comparison between technology-oriented and market-oriented project.
The value is measured by netEcContribution; groups are identified by the AI generated labels: and .
Figure 6 illustrates how centrality measures and AI-generated categories provide insights into the roles of organisations in hydrogen-related innovations, highlighting differences between those engaged in market-oriented and technology-oriented projects.
Figure 6. Coreness of organisations involved in the NAHV project, shown in comparison with the range of coreness values for each year (grey background).
The figure shows in grey the range of values found each year, and highlights organisations participating in the NAHV project. The diagrams provide an overview of the evolution of coreness in (left) and (right). The grey shaded area represents the range of coreness values observed each year. The organisations involved in the NAHV project are highlighted with coloured dots, connected by lines that illustrate their progression over time. A notable observation is that, while all NAHV partners are part of (which is expected, given NAHV’s classification as a market project), only a few are involved in .
Coreness values can fluctuate over time: an increase suggests that an organisation is gaining influence through participation in relevant projects, while a decrease may indicate that, following the completion of a project, the organisation has not yet initiated a new one within the Horizon framework. The analysis also includes the years 2024 to 2029, reflecting ongoing projects that are scheduled to conclude during this period. It is important to note that this is not a forecast, but rather a record of existing projects with future completion dates.
Moreover, centrality of all organisations can be visualized in a degree-coreness plane, where each organisation is represented by a bubble. Degree serves as a proxy for an organisation’s capacity to form partnerships with many other organisations, while coreness reflects its ability to partner with influential organisations. Although coreness is limited by degree, the size of the bubble represents the organisation’s strength, which serves as a proxy for its capacity to attract substantial funding and invest in large projects. An example is shown in Figure 10, which also includes an information about communities.
3.2 Communities
In the context of this paper, communities refer to relevant subgroups of organisations that exhibit strong collaborative ties. The focus is on analysing a single partition and its evolution over time. The analysis is conducted on the family of complete networks, Gy.
As explained in Section 2.5, the first step involves exploring the solution space in terms of completeness, the number of solutions, and their validity across various algorithms.The tested algorithms include Louvain (LV)33, Leiden (LD)34, Label Propagation (LP)35, Edge Betweenness (EB)36, Leading Eigenvector (EV)37, Walktrap (WT)38, and Infomap (IM)39. The tests were conducted in R40, using the iGraph41 and communities42 libraries.
The number of solutions || are illustrated in Figure 7: most algorithms produced multiple solutions across most years, with the exceptions of 2015 and 2029, where the networks were simpler. Interestingly, WT consistently provided a unique solution.However, EB had a different issue, generating partitions that were not similar to each other, with similarity coefficients often below 0.5. EV produced multiple valid solutions in most years but failed to find a partition in 2024.
Figure 7. Number of solutions |𝕊y| generated by different community detection algorithms.
A more in-depth observation of the solution space for 2024 (the most critical year) reveals additional details, as shown in Figure 8. LV generates a large set of solutions with || = 69, which are also highly diverse, as evidenced by the similarity plot showing a bimodal distribution with peaks around 0.6 and 0.9. Moreover, approximately one-third of the solutions are invalid due to containing one or more internally disconnected communities. This is a well-documented issue of the algorithm, as discussed in the literature. LD, specifically designed to address the problem of disconnected communities, also produced invalid solutions for this network: its partition consists of a few large communities and a high proportion of singletons, with a mixing parameter exceeding 0.5, indicating that nodes within a community were more connected to nodes outside their community than to those within it.
Figure 8. Solution space diagrams.
Left: frequency and confidence intervals for the solutions identified by each algorithm. In the case of LV and LP only the 10 most frequent solutions are shown. Middle: distribution of community size for each solution. Proper communities are represented as black circles, single-node communities are represented as blue diamonds. Right: pairwise similarity between solutions.
LP generates a different solution with each run; in this case, with t = 100, there are 96 valid and distinct solutions, while 4 are invalid due to internally disconnected communities. Better results can be achieved with IM that produces |𝕊IM| = 1 in most years, and a dominant one solution in the most complex cases such as 2024. Overall, WT is the only algorithm that consistently provided valid results across all years, with valid partitions and |𝕊WT| = 1, consequently, it will be used for community detection and temporal evolution analysis.
The selected partition 𝒫y can be represented as a network in which nodes belonging to the same community are grouped and coloured to distinguish between communities. For example, the result for P2018 is shown in Figure 9. Some communities appear disconnected from the others; these consist of organisations that, in that year, participated in only one project and thus form a separate component.
Figure 9. Network G2018, with communities grouped and highlighted in different colours.
Figure 10. Centrality measures of all organisations in G2024.
Each bubble represents an organisation in the degree-coreness plane. Degree is a proxy for the organisation’s ability to win projects and attract funding, while coreness reflects its capacity to partner with other influential organisations. The largest bubbles in the top-right of the diagram highlight organisations that excel in both areas.
Community detection can be integrated with centrality measures to provide a more comprehensive understanding of network structure.
Figure 10 illustrates the results for 𝒫2018. In this visualisation, each bubble represents an organisation, with its position determined by coreness and degree centrality. Bubble colours correspond to the community, and bubble size is proportional to the strength (the total weight of its connections in the given year). The diagram clearly demonstrates that degree is the upper limit for coreness, as no points appear above the line with a slope of 1. In practical terms, this indicates that an organisation that participates in many projects (high degree) is not necessarily the most influential in the network. Strength, which reflects the total weight of an organisation’s connections, remains independent of both degree and coreness, meaning that large bubbles can appear anywhere on the graph. This means that some organisations, despite having relatively few connections (low degree), are linked to relevant partners (high coreness) and stand out for their ability to secure significant EU research funding (high strength).
3.3 Evolution of communities
Temporal evolution was calculated as described in Section 2.6, using the WT algorithm to ensure the robustness of the results. In the diagram shown in Figure 11, the horizontal axis represents time, with each community depicted as a coloured rectangle. The height of each rectangle is proportional to the sum of the strengths of its members. For the purpose of clarity, only the six largest communities are represented. The analysis reveals a significant pattern: initially a large number of small communities is formed, and their configuration changes rapidly. This reflects the fact that in Horizon 2020 public funding was primarily allocated to technology-oriented research projects, and partnerships changed rapidly. However, with the onset of Horizon Europe, and specifically the Clean Hydrogen partnership, three large communities emerge, and are set to continued collaboration through 2029.
Figure 11. Evolution of the largest communities from 2015 to 2029.
For clarity, only the six largest communities are displayed.
The significant size of the largest communities reflects the increasing allocation of resources to Horizon Europe projects in the hydrogen sector. This growth is driven by the development of hydrogen valleys, which have fostered long-term, stable partnerships, contributing to both the stability of collaborations and the continued expansion of these communities.
4 Discussion
This paper presented a comprehensive methodology for analysing the long-term impact of EU-funded research projects, with a case study focused on the hydrogen sector. By employing network analysis, centrality measures, and community detection, we demonstrated how to identify pivotal organizations, monitor their influence over time, and delineate the evolution of collaborations fostered by Horizon 2020 and Horizon Europe projects.
Specifically, the analysis shows that Horizon grants in hydrogen sector projects have prompted the emergence of communities that have significantly evolved over the years. Initially characterized by small, rapidly changing groups, these groups gradually shifted towards the formation of larger, more stable partnerships, particularly in the domain of hydrogen valleys.
The methodology presented in this paper, particularly the use of centrality measures, community detection algorithms and temporal network analysis, offers opportunities to improve the visualizations in the Horizon Dashboard43. For instance, the ”Organization Profile” visualization, which allows users to explore an organization’s performance in EU research and innovation programs, could benefit from incorporating centrality measures and tracking community memberships of each organization over time. This could be done by analyzing the entire Horizon network or by focusing on specific subsets categorized according to the top-level EuroSciVoc fields: Natural Sciences, Engineering and Technology, Medical and Health Sciences, Agricultural Sciences, Social Sciences, and Humanities.
One limitation of this study is that it is focused exclusively on the CORDIS dataset, which restricts its scope to EU-funded initiatives, potentially neglecting initiatives funded by national or local authorities, or private funds. Incorporating data from other sources could provide a more comprehensive perspective, but such datasets are currently unavailable.
5 Conclusions and future work
The methodology presented in this paper can provide valuable insights for both policymakers and project coordinators. Policy makers, such as those supporting Hydrogen Valley projects across Friuli Venezia Giulia, Croatia, and Slovenia, can leverage this methodology to monitor the engagement of organizations in their respective territories. As the Hydrogen Valley projects launched in 2022–2023 come to an end and new Horizon calls for proposals become available, policy makers will be able to assess whether today’s leading organisations can form new partnerships and maintain their leadership positions. In addition, policy makers at European level can use the methodology to gain insights into the evolution of communities, to assess their openness to newcomers and the stability of collaboration between industrial and academic partners. From an alternative perspective, project coordinators within industrial or research organisations may use the methodology presented in this paper to identify potential partners who may be influential in facilitating future collaborations.
Future work will focus on several key areas. We will continue to monitor the centrality measures of the NAHV partners over the course of the project and beyond. In addition, we will apply the methodology to further case studies on different topics, in particular pandemic preparedness and electron microscopy, both of which are topics of interest for Area Science Park and have been published in 44. Finally, future research will explore how the rich textual information from project deliverables available in CORDIS can be effectively leveraged to enhance network models.
Ethics and consent
Ethical approval and consent were not required.
Data availability
The underlying data for this research is available through data.europa.eu, the official portal for European data, under a CC0 license:
Repository name: CORDIS - EU research projects under Horizon 2020 (2014–2020). DOI: https://doi.org/10.2906/112117098108/1227. The repository contains the following underlying data: H2020 projects file, which includes participating organisations, legal basis information, topic information, project URLs and classification with the European Science Vocabulary (EuroSciVoc).
Repository name: CORDIS - EU research projects under Horizon Europe (2021–2027). DOI: https://doi.org/10.2906/112117098108/2028 The repository contains the following underlying data: HORIZON projects file, which includes participating organisations, legal basis information, topic information, project URLs and classification with the European Science Vocabulary (EuroSciVoc).
The data created by this research is available in Zenodo under the terms of the Creative Commons Attribution 4.0 International license (CC-BY 4.0). Repository name: Horizon Projects Network DOI: https://doi.org/10.5281/zenodo.13765372 The repository contains the following data:
O.csv: Organisations’ unique identifiers and attributes.
P.csv: Projects’ unique identifiers and attributes.
W.csv: Describes the participation of each organization in a project in each specific year. The participation is weighted according to the ”total cost” of the project, shared proportionally based on the project’s duration within the year.
activity type.csv provides a description of the codes used to define the activity types associated with each organization.
2015.graphml, 2016.graphml, ..., 2029.graphml: a set of files representing network model for collaboration between organizations in any given year from 2015 to 2029. Each graph encodes the centrality measures (degree, strength, coreness) and community labels.
format-info.txt provides information on the format of .csv and .graphml files used in this dataset.
sample-networks-hydrogen-energy.pdf showcases a visual representation of the networks from 2015 to 2029.
Faculty Opinions recommendedReferences
- 1.
Barber MJ, Krueger A, Krueger T, et al.:
Network of European Union-funded collaborative research and development projects.
Phys Rev E Stat Nonlin Soft Matter Phys.
2006; 73(3 Pt 2): 036132. PubMed Abstract
| Publisher Full Text
- 2.
Breschi S, Cusmano L:
Unveiling the texture of a European Research Area: emergence of oligarchic networks under EU framework programmes. In: Knowledge flows in European industry, Routledge, 2006; 294–324. Reference Source
- 3.
Balland PA, Boschma R, Ravet J:
Network dynamics in collaborative research in the EU 2003–2017.
Eur Plan Stud.
2019; 27(9): 1811–1837. Publisher Full Text
- 4.
Maruccia Y, Solazzo G, Del Vecchio P, et al.:
Evidence from network analysis application to innovation systems and quintuple helix.
Technol Forecast Soc Change.
2020; 161: 120306. Publisher Full Text
- 5.
Muscio A, Ciffolilli A, Lopolito A:
Technological diversity in collaborative projects: insights into European research policy.
Journal of Economic Policy Reform.
2022; 25(3): 322–343. Publisher Full Text
- 6.
Cerqueti R, Iovanella A, Mattera R:
Clustering networked funded European research activities through rank-size laws.
Ann Oper Res.
2023; 1–29. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 7.
Publications Office of the European Union: Euroscivoc - European science vocabulary. Accessed: 2024-09-24. Reference Source
- 8.
Clean Hydrogen Partnership:
Hydrogen valleys: insights into the emerging hydrogen economies around the world. 2021. Reference Source
- 9.
Weichenhain U, Kaufmann M, Hölscher M, et al.:
GOING GLOBAL: an update on hydrogen valleys and their role in the new hydrogen economy. Clean Hydrogen Partnership, 2021. Reference Source
- 10.
Majka A, Klimczyk M, Kucharski K, et al.:
Hydrogen valley as a hub for technological cooperation between science, business, local government and NGOS: an overview of approaches in Europe.
Torun International Studies.
2023; 1(17): 5–15. Publisher Full Text
- 11.
Frankowska M, Cheba K:
Exploring the research landscape of hydrogen valleys: a bibliometric analysis.
Journal of Sustainable Development of Transport and Logistics.
2023; 8(2): 348–359. Publisher Full Text
- 12.
Petrollese M, Concas G, Lonis F, et al.:
Techno-economic assessment of green hydrogen valley providing multiple end-users.
Int J Hydrogen Energy.
2022; 47(57): 24121–24135. Publisher Full Text
- 13.
Concas G, Cocco D, Lecis L, et al.:
Life cycle analysis of a hydrogen valley with multiple end users.
J Phys Conf Ser.
2022; 2385: 012035. Publisher Full Text
- 14.
D. of Science, B. E. Innovation, Anglo American Platinum, and ENGIE: South Africa hydrogen valley final report. 2021; Accessed: September 12, 2024. Reference Source
- 15.
European Commission: North adriatic hydrogen valley - project website. Accessed: 2024-08-26. Publisher Full Text
- 16.
European Commission: Call HORIZON-JTI-CLEANH2-2022-2 - Horizon Europe joint technology initiative on clean Hydrogen. Accessed: 2024-08-26. Reference Source
- 17.
Komninos N:
Transformation of industry ecosystems in cities and regions: a generic pathway for smart and green transition.
Sustainability.
2022; 14(15): 9694. Publisher Full Text
- 18.
Chesbrough HW:
Open innovation: the new imperative for creating and profiting from technology. Harvard Business School Publishing Company, 2003. Reference Source
- 19.
Carayannis EG, Campbell DF:
Triple helix, quadruple helix and quintuple helix and how do knowledge, innovation and the environment relate to each other?: A proposed framework for a trans-disciplinary analysis of sustainable development and social ecology.
International Journal of Social Ecology and Sustainable Development (IJSESD).
2010; 1(1): 41–69. Publisher Full Text
- 20.
Casaramona A, Sapia A, Soraci A:
How TOI and the quadruple and quintuple helix innovation system can support the development of a new model of international cooperation.
Journal of the Knowledge Economy.
2015; 6(3): 505–521. Publisher Full Text
- 21.
Hegyi FB, Mariussen A:
Connecting urban and regional innovation ecosystems to enhance competitiveness.
Osuwa.
2021. Publisher Full Text
- 22.
Pires SM, Polido A, Teles F, et al.:
Territorial innovation models in less developed regions in Europe: the quest for a new research agenda?
Eur Plan Stud.
2019; 28(8): 1639–1666. Publisher Full Text
- 23.
Agrawal A, Cockburn I, Galasso A, et al.:
Why are some regions more innovative than others? The role of small firms in the presence of large labs.
J Urban Econ.
2014; 81: 149–165. Publisher Full Text
- 24.
Soraci A:
Innovation as a strategy of reaction to delocalization: Shipbuilding sector, a study case. Promoting Entrepreneurship by Universities, Lahti University of Applied Sciences, 2009.
- 25.
Federal Economic Development Agency for Northern Ontario:
Regional innovation ecosystems. 2024. Reference Source
- 26.
European Commission: Cordis: Eu research results.
Accessed: Sept. 2024. Reference Source
- 27.
EU Publications Office: CORDIS - EU research projects under Horizon 2020 (2014–2020). 2015. http://www.doi.org/10.2906/112117098108/12
- 28.
EU Publications Office: CORDIS - EU research projects under HORIZON EUROPE (2021–2027). 2022. http://www.doi.org/10.2906/112117098108/20
- 29.
Batagelj V, Zaversnik M:
An o(m) algorithm for cores decomposition of networks. ArXiv. cs.DS/0310049, 2003.
Publisher Full Text
- 30.
Diboune A, Slimani H, Nacer H, et al.:
A comprehensive survey on community detection methods and applications in complex information networks.
Soc Netw Anal Min.
2024; 14(1): 93. Publisher Full Text
- 31.
Khawaja FR, Zhang Z, Memon Y, et al.:
Exploring community detection methods and their diverse applications in complex networks: a comprehensive review.
Soc Netw Anal Min.
2024; 14: 115. Publisher Full Text
- 32.
Morea F, Stefano DD:
Enhancing stability and assessing uncertainty in community detection through a consensus-based approach. 2024. Publisher Full Text
- 33.
Blondel VD, Guillaume JL, Lambiotte R, et al.:
Fast unfolding of communities in large networks.
J Stat Mech.
2008; 2008(10): P10008. Publisher Full Text
- 34.
Traag VA, Waltman L, Van Eck NJ:
From louvain to leiden: guaranteeing well-connected communities.
Sci Rep.
2019; 9(1): 5233. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 35.
Raghavan UN, Albert R, Kumara S:
Near linear time algorithm to detect community structures in large-scale networks.
Phys Rev E Stat Nonlin Soft Matter Phys.
2007; 76(3 Pt 2): 036106. PubMed Abstract
| Publisher Full Text
- 36.
Newman MEJ, Girvan M:
Finding and evaluating community structure in networks.
Phys Rev E Stat Nonlin Soft Matter Phys.
2004; 69(2 Pt 2): 026113. PubMed Abstract
| Publisher Full Text
- 37.
Newman MEJ:
Finding community structure in networks using the eigenvectors of matrices.
Phys Rev E Stat Nonlin Soft Matter Phys.
2006; 74(3 Pt 2): 036104. PubMed Abstract
| Publisher Full Text
- 38.
Pons P, Latapy M:
Computing communities in large networks using random walks. In: Computer and Information Sciences-ISCIS 2005: 20th International Symposium, Istanbul, Turkey, October 26–28, 2005. Proceedings 20, Springer, 2005; 284–293.
Publisher Full Text
- 39.
Rosvall M, Bergstrom CT:
An information-theoretic framework for resolving community structure in complex networks.
Proc Natl Acad Sci U S A.
2007; 104(18): 7327–7331. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 40.
R Core Team: R: a language and environment for statistical computing. 2022. Reference Source
- 41.
Csardi G, Nepusz T, Traag V, et al.:
igraph: network analysis and visualization in R. 2023. Reference Source
- 42.
Morea F:
https://github.com/fabio-morea/communities.
Aug. 2024.
- 43.
European Commission: Horizon Dashboard. Accessed: 2024-10-16. Reference Source
- 44.
Morea F:
Horizon projects network dataset. Dataset, 2024. Publisher Full Text
- 45.
Morea F:
https://github.com/fabio-morea/horizon-intelligence.
Aug. 2024.
