Introduction
Public health experts and decision-makers face situations where evidence from experimental studies is limited, such as the introduction of new health technologies or the emergence of a novel disease. In these circumstances, the potential impact and cost-effectiveness of different policy options can be highly uncertain. Mathematical modelling provides a structured way to synthesise diverse data sources and explicitly account for uncertainty, enabling the assessment of public health and health economic impacts by simulating hypothetical situations that would be impractical or unethical to study empirically.2 For example, modelling has been used to assess the cost-effectiveness of human papillomavirus (HPV)3 and malaria4 vaccination, to evaluate control strategies for foot-and-mouth disease outbreaks in livestock,5 and to guide mitigation decisions during the COVID-19 pandemic.6–10
Collaborative modelling using ensemble approaches offers important advantages over individual models. By combining projections from multiple independently developed models, ensembles provide a more comprehensive representation of uncertainty, capturing variability in assumptions, structure, and parameterisation across contributing models.11 In infectious disease forecasting, this approach was first operationalised in the U.S. FluSight challenge, which produced weekly consensus forecasts for influenza-like illness (ILI) by combining outputs from different models.12,13 Ensemble modelling has since been applied to forecast COVID-19 case and death trends,14,15 and to assess the impact of vaccination strategies for diseases such as COVID-19, malaria, and rotavirus.16–18 Ensemble approaches support more informed and resilient public health decisions by presenting a range of plausible outcomes and reducing reliance on any single set of assumptions.
Scenario modelling hubs have become established platforms for coordinating ensemble modelling efforts in infectious disease medium and long-term projections and policy planning.19 In the United States, the COVID-19 (from 2020), Flu (2022), and RSV (2024) Scenario Modelling Hubs have brought together multiple modelling teams to generate ensemble-based scenario projections for medium- to long-term planning and policy evaluations.20–22 Similarly, the European COVID-19 Scenario Hub (2022–2023) provided insights on vaccination strategies10 and the projected burden of new SARS-CoV-2 variants.23–25 Building on these efforts, ECDC launched RespiCompass in April 2024—a new scenario modelling hub for communicable diseases in the EU/EEA. The first scenario round, published in November 2024,1,26 focused on the impact of influenza and COVID-19 vaccination. In this paper, we report on the experience of running the first RespiCompass cycle, covering the definition of goals, scenario design, recruitment of modelling teams, stakeholder engagement, analysis, and communication of results, rather than its quantitative findings. The scenario projections themselves, including ensemble estimates of averted COVID-19 hospitalisations attributable to vaccination, are presented on the RespiCompass website1 and in an ECDC report.26 We conclude with a summary of insights and an outlook on future directions.
Results and discussion
RespiCompass vision and governance
RespiCompass builds on the experience of the European COVID-19 Scenario Hub, which was created to foster international modelling collaborations during the acute phase of the COVID-19 pandemic.28 With the end of the COVID-19 public health emergency, ECDC expanded the hub’s scope beyond COVID-19 to create RespiCompass. The founding vision of RespiCompass is:
RespiCompass aims to become a leading resource for evidence-based decision-making in public health preparedness and response to respiratory infectious diseases. RespiCompass aspires to empower policymakers, healthcare professionals, and other stakeholders with robust scenario-based modelling insights to inform strategic planning and resource allocation. By fostering collaboration across European and global modelling communities, the hub aims to bridge the gap between advanced modelling expertise and practical public health needs.
RespiCompass follows an annual project cycle designed to deliver relevant and actionable insights and to ensure project improvement. The process begins with consultations with disease and public health experts to define policy-relevant questions and prioritise scenario options. Scenarios are then designed and shared with participating modelling teams, who—applying a common set of shared assumptions—independently develop their models and submit results following a standard submission format. After analysis and interpretation, findings are disseminated through different channels. In the 2024 project cycle, results were shared through a website1 and stakeholder presentations, followed by a technical report.26 Each cycle concludes with an evaluation phase to assess impact and identify areas for improvement. An illustration of the inputs, processes, and outputs for the 2024 project round is provided in
Figure 1.
Figure 1. Overview of inputs, processes, and outputs for the 2024 RespiCompass project round.
The process begins with the policy question, which informs the scenario definitions (illustrated here through the influenza and COVID-19 scenario tables) and underlying assumptions. Based on these inputs, modelling teams develop their models and generate projections. Results are then analysed, compared, and interpreted. The findings from this analysis inform the final outputs and recommendations. *European Respiratory Virus Network.
RespiCompass brings together a network of contributors (
Table 1). It is coordinated by ECDC and draws on internal and external expertise in modelling, epidemiology, virology, immunisation, and preparedness. In its first year, RespiCompass engaged with the European Respiratory Virus Network (ERVI-Net),29 which helped shape the initial scenario round and provided feedback through a survey on decision-making support needs and communication preferences. Looking ahead, RespiCompass plans to expand engagement with networks such as the EU/EEA NITAG Collaboration Network,30 to better align scenario design with national decision-making priorities. ECDC outsources technical support for the project to a contractor (EU-SMIND, a modelling consortium) who also contributes through submission of modelling results.
Table 1. RespiCompass actors and their roles.
| Function/Actor | ECDC | National experts & networks | Contributing modelling teams | Contractor |
|---|
| Defining priorities & objectives
| Defines project scope and objectives in alignment with ECDC strategy, incorporating input from national experts. | Provide national perspectives on priorities and practical considerations. | Engage in discussions on modelling scope and may propose scenarios. | Contributes to scope definition, Assesses technical feasibility and advises on implementation. |
| Facilitation & coordination
| Connects experts, aligns efforts (e.g., modelling and ECDC literature reviews). | Engage in discussions, provide feedback, and insights into national contexts. | Contributes to modelling discussions and provides feedback on technical feasibility. | Organises scenario launch, supports modeller meetings, develops digital infrastructure. |
| Knowledge & network building
| Integrates expertise across modelling, epidemiology, virology, vaccines, and behavioural science. | Act as the link between national needs and ECDC priorities, foster connections with national modellers. | Participate in project meetings, share insights on technical approaches. | Engages in discussions with ECDC to refine implementation. |
| Infrastructure & technical implementation
| Defines platform requirements, ensures usability and adaptability. | Provide feedback on interpretability and usability for public health agencies and stakeholders, may share relevant data. | Develop models and submit modelling results, provide feedback on user experience and functionality for modelling contributions. | Develops models and submits modelling results, develops and maintains websites, scripts for data reporting, and visualisation tools. |
RespiCompass benefits from collaboration with the US COVID-19, Flu, and RSV Scenario Modelling Hubs.20–22 Since 2020, these hubs have collectively conducted 25 scenario rounds, demonstrating the value of collaborative modelling and providing a wealth of best practices.11,16,19,31–36 RespiCompass and the US hubs share methodologies, including ensemble approaches, and they rely on shared infrastructure such as the hubVerse37 for validation, visualisation, and submission management. Collaboration between European and US teams has been mutually beneficial; some US teams contributed directly to ECDC collaborative modelling initiatives, while European teams participated in US hub activities, fostering a global exchange of knowledge and tools.
Defining a scenario round
The structured design of infectious disease modelling scenarios has seen important advances in recent years.2 Defining such scenarios is an inherently iterative process that requires ongoing refinement to balance feasibility constraints, available data, and stakeholder priorities. In the 2024 RespiCompass scenario round, we developed scenario projections for influenza and COVID-19 burden for EU/EEA countries from September 2024 to May 2025, based on plausible country-specific ranges for vaccination coverage and epidemiological contexts. We aimed to harmonise modelling outputs for influenza and COVID-19 to facilitate joint interpretation and communication. However, data availability differed across diseases, particularly for burden indicators. While there was sufficient data availability across countries for COVID-19 hospitalisations, there was comparatively limited coverage for influenza hospitalisations.1 Consequently, an alternative surveillance target with adequate coverage—pathogen-specific primary care consultation rates—was estimated for influenza. Early testing of scenarios revealed additional challenges, including implausible projections under certain epidemiological assumptions. These experiences highlighted the value of mock-model runs before launching a scenario round to identify potential issues and allow adaptive adjustments. The use of mock-models will require careful consideration (e.g., What is an appropriate mock-model type?) and an evaluation of costs versus benefits of mock-modelling efforts.
Precise definitions, scenario documentation, and cross-team communication were essential to avoid misunderstandings. For example, the meaning of “waning immunity” in COVID-19 models required clarification—did it refer to vaccine-induced immunity, natural immunity, or both? Similar ambiguities arose when vaccine effectiveness (VE) was given without explicitly stating the control group against which effectiveness is measured (i.e. absolute vs relative VE), or for the implementation of influenza transmission potential, which required careful discussion to specify which biological factors were included or excluded. Early modeller feedback thus helped uncover which assumptions were left implicit by our guidance. These experiences highlight that scenario modelling is not a linear process, but an ongoing dialogue among modellers, disease and surveillance experts, data specialists, and public health decision-makers. Embedding structured preliminary testing of scenario definitions, learning from previous scenario rounds, targeted stakeholder reviews, and time for iterative refinements into future cycles will strengthen RespiCompass and improve scenario design.
Additionally, in a multi-model hub like RespiCompass, model diversity creates both value and complexity. Different models bring varying assumptions, data needs, and levels of flexibility, making it difficult to anticipate in advance which scenario elements will be problematic for whom. What is trivial for one model may be infeasible or unclear for another. This reinforces the importance of early engagement and continuous dialogue to clarify definitions, surface technical barriers, and align expectations. A key lesson from 2024 is that structured mechanisms for model-sharing and discussion, targeted support, and collaborative validation processes are essential to ensure consistency, comparability, and usefulness of outputs.
Modeller recruitment and engagement
Recruiting modelling teams for RespiCompass was challenging, as participants not contracted to ECDC were neither directly compensated nor able to tap into competitive grants linked to the project. To ensure projections from at least four distinct models, ECDC contracted a modelling consortium, and its in-house team also contributed models. Additional teams were recruited through professional networks, where existing collaborations from previous projects proved valuable. Interviews with the external teams outside the consortium revealed that their primary motivation for participation was the opportunity to test and showcase their models in a practical, policy-relevant setting, and to compare their approach with those of other international groups. One of these teams was already using their model to submit forecasts to ECDC’s forecasting hub (RespiCast),27 while the other had refined their model since the end of the COVID-19 public health emergency of international concern (PHEIC) and viewed RespiCompass as an opportunity to validate their work.
Technical complexities and a submission deadline for model results scheduled in September 2024, shortly after the summer holiday period, created resource challenges for the teams. Contributing teams made substantial efforts to update different model components, including details of the vaccination and related immunity processes, but faced issues due to high computational needs. Despite these hurdles, teams provided positive feedback on the clarity of project goals, the timely availability of scenario documentation, and the active support and communication by ECDC and the EU-SMIND contractor. Regular meetings to clarify processes, assumptions, and timelines added to a positive collaborative experience. Participants suggested further improvements including the creation of a dedicated communication channel and topic-specific discussion groups (e.g., on the use and availability of human-to-human contact data for simulating viral transmission). Finally, participating teams emphasised the value of formal recognition through co-authorship in peer-reviewed publications or official ECDC reports, particularly for junior researchers, and they also recommended in-person meetings to strengthen community building and sustained engagement.
Beyond the contributing modelling teams, a handful of additional groups followed the 2024 cycle more informally. These teams attended the virtual launch and technical meetings and responded to scenario round e-mails, but ultimately did not submit. From our informal interactions, their decision appeared to be driven by limited staff or funding in 2024 and the compressed submission timeline. Even so, they provided useful feedback on the scenario design and data inputs. Importantly, several of these groups have already expressed their intention to submit in the 2025 scenario round. This points to an additional pool of modelling capacity that could support future iterations of RespiCompass.
Moving forward, we aim to continue strengthening our engagement with external modelling teams as this is crucial for the long-term sustainability of RespiCompass. The motivations and constraints of potential partners differ. For academic groups, continued participation will likely depend on a clear return on investment in the form of publications, funding opportunities, or career advancement. Modelling teams in public health agencies, on the other hand, rely on institutional dedication and often face severe resource constraints that can limit their capacity for contributions. Modelling in both academic and public health agencies often relies on external funding. Careful planning around these challenges is crucial. To start with, supporting participation will require attention to timelines and resource needs, including providing sufficient lead time for model development and submission, offering access to cluster computing resources to improve processing power, and designing specific funding channels. Organising in-person meetings could further build collaboration and a sense of community among participants. In addition, complementing ECDC reports with contributions to peer-reviewed publications may serve as an added incentive for participation, particularly among academic partners.
Analysing and interpreting results
Interpreting the results required more than analytics and computation. On one hand, it involved discussing the contexts that reveal potential biases and limitations of surveillance data with disease and surveillance experts. On the other hand, we directly engaged with the modellers to understand the differences between individual models. Discrepancies between models often reveal critical insights into how assumptions and methods can shape projections. These discussions not only enhance the understanding of current results but also serve as a learning opportunity to improve future scenario definitions, modelling approaches, and data needs.
A core element of the analysis is synthesising results through ensemble methods—an umbrella of methods that combine results from different models into single estimates. Here, methodological decisions can influence both point estimates and uncertainty ranges. Due to tight project timelines, we were unable to systematically compare different ensemble approaches in the 2024 cycle, highlighting a broader gap: the lack of clear, systematically evaluated guidance for ensemble method selection in infectious disease modelling.38,39 Such guidance would need to account for factors such as the number of contributing models, biases in geographic or temporal coverage, and hierarchical data structures. For the 2024 RespiCompass round, we applied a linear opinion pool method,40 and incorporated a trimming step to limit the impact of extreme outputs, defined as model estimates that markedly deviate from the ensemble’s central statistical distribution.40 In the next cycle, we plan to formally compare ensemble methods before selecting any specific ensemble approach.
While ensemble estimates aggregate outputs from multiple models into a single result, individual model outputs provide valuable insights into where different models agree and disagree. Presenting both the ensemble results—regardless of the methodology used—and the individual model outputs (as we did for the 2024 RespiCompass results)1 allows end-users to better understand the underlying variability, fostering trust when communicating modelling results to stakeholders. We anticipate maintaining this approach in the next iteration to ensure transparency and enhance stakeholder trust.
Communication strategies
A survey conducted with ECDC’s European Respiratory Virus Disease Network revealed a preference for a dedicated RespiCompass website for time-sensitive communications over traditional formats like technical reports. Considering this, we developed a focussed and visually engaging RespiCompass website. However, managing outputs through a web platform poses challenges. Websites imply that information is up to date, yet some results—such as winter 24/25 burden estimates—quickly become obsolete. More enduring outputs, like estimates of averted COVID-19 hospitalisations due to vaccination, may be better suited for dissemination through technical reports or peer-reviewed publications. To address these needs, we plan to adopt a multi-channel communication strategy, combining both static and website-based reports to better align the intended use and lifespan of results.
A recurring communication challenge has been clarifying the distinction between forecasting (as pursued by ECDC’s RespiCast hub)27 and scenario modelling (as pursued by RespiCompass).1 Forecasting aims to predict the most likely short-term future based on trends observed in recent data, while scenario modelling explores disease dynamics under hypothetical conditions, typically in the medium- to long-term—essentially running in-silico experiments (i.e. experiments carried out virtually by a computer, server, or cluster) (Definition Box 1). By explicitly reinforcing this distinction in all communications, we aim to help stakeholders appreciate the complementary roles of forecasting and scenario modelling.
Definition Box 1. Scenario modelling versus forecast modelling.
Forecast modelling aims to predict the future trajectory of epidemiological indicators, such as weekly COVID-19 hospitalisations. Like weather forecasts, these models provide near-term projections based on recent data. For diseases with short serial-intervals like COVID-19 and influenza, reliable forecasts typically extend only a few weeks into the future.15 These models rely heavily on timely surveillance data to project short-term outcomes. Launched in 2023, ECDC’s RespiCast hub leverages an ensemble approach to deliver short-term forecasts of respiratory virus indicators for the EU/EEA.27
Scenario modelling, in contrast, is not designed to predict the most probable future. Instead, it explores multiple “what-if” scenarios, often referred to as counterfactuals.2 Scenarios may explore future as well as past time-windows. This approach helps assess the potential impact of interventions (e.g., through modelling vaccination strategies) and supports preparedness for emerging threats (e.g., through modelling future pandemic scenarios). Scenario modelling can also reveal how different policies, or external conditions might have altered disease dynamics. These models often require mechanistic representations of disease transmission, and they can be used even when surveillance data are limited. Launched in 2024, ECDC’s RespiCompass hub leverages an ensemble approach to deliver scenario projections for respiratory virus burden in the EU/EEA.28
Summary and future directions
The first scenario round of RespiCompass confirmed the value of coordinated scenario-based modelling, leveraging an ensemble approach, for informing strategic public health decision-making.
Table 2 summarises the principal challenges encountered during the hub’s design phase and first round, together with the lessons drawn from each. A key lesson was the importance of early stakeholder engagement. Involving public health experts and decision-makers in the scenario design phase helped tailor questions to policy priorities, clarify expectations, and enable transparent discussions around data limitations, which was critical for building trust in the results.
Table 2. Summary of challenges and lessons learnt during RespiCompass design phase and first round.
| Challenge | Key lessons learned/Solutions identified |
|---|
| Scenario definition | Engage stakeholders early to ensure relevance and feasibility |
| Use mock/pilot runs to test assumptions and scenario clarity |
| Establish structured feedback loops to refine definitions iteratively |
| Recruitment and engagement of participants | Hold regular touchpoints and thematic discussions to maintain engagement |
| Provide logistical and technical support (e.g., compute resources, documentation, timely announcements, funding initiatives) |
| Recognise contributions through authorship and community-building activities |
| Timelines and capacity constraints | Align submission deadlines with academic calendars and provide adequate lead time |
| Streamline internal coordination to anticipate delays and bottlenecks |
| Model synthesis and insight generation | Introduce peer review of model submissions to enhance rigour |
| Compare ensemble approaches systematically to inform method choice |
| Interpret discrepancies between models as learning opportunities |
| Communication and dissemination | Adopt multi-channel strategies: webinars and web-based for fast updates, reports for detailed summaries, publications for more focussed insights and modeller engagement |
| Clarify distinction between forecasting (RespiCast) and scenario modelling (RespiCompass) in all communications |
| Stakeholder trust and uptake | Transparently present both ensemble and individual model results to highlight uncertainty |
| Frame results within real-world decision-making constraints to enhance usability. |
Technical coordination and feedback loops with modelling teams were equally important. Pilot runs and mock exercises surfaced feasibility and consistency issues early, enabling pre-emptive refinement of scenario definitions and assumptions. These practices mirror successful design choices from previous hubs,32 which emphasise the value of iterative development. To enhance rigour further, future RespiCompass cycles will incorporate structured assumption and transparency review of model submissions. Additionally, methodological strengthening remains a priority. While ensemble methods allow outputs from different models to be synthesised for easier communication of results, there is still no standardised approach for this. In some cases (e.g., when there is large between-model heterogeneity) presenting individual model results may be more appropriate. Improving transparency will require clear guidance on multi-model analysis, likely informed by lessons from meta-analysis, as well as systematic comparisons of ensemble strategies to identify best practices.
Lastly, communication and uptake of project results remain central to their impact. RespiCompass adopted a multi-channel dissemination strategy, including a website for rapid pre-season communication of early results, technical documentation for intervention impact estimates, and live briefings with surveillance networks and policy makers to contextualise findings. Looking ahead, RespiCompass will further integrate these lessons through refined timelines, formal assumption and transparency review, expanded stakeholder engagement, and capacity-building activities. By aligning technical innovation with operational clarity and effective communication, the hub is well-positioned to support timely, transparent, and decision-focused scenario modelling across the EU/EEA.
Software availability
No new software was created for this article.
Ethics and consent
Ethical approval was not required for this work. The stakeholder engagement described (a survey of the European Respiratory Virus Network and interviews with participating modelling teams) was conducted as part of the operational development and evaluation of a public health project, rather than as research involving human subjects. The network survey was administered through the official EUSurvey tool and opened with a written notice setting out the purpose of the survey, stating that only authorised staff could access any personal information, asking respondents not to enter personal data in free-text fields, and informing them that aggregated results would be presented to the network and/or published. Consent to participate was implied by voluntary completion of the survey following this notice. Interviews with modelling teams were undertaken by contributors in their professional capacity as project collaborators. No personal or identifying data were collected or reported.
Disclaimer
The views expressed in this article are those of the author(s). Publication in Open Research Europe does not imply endorsement of the European Commission.
