Introduction
Over the last few decades, unmanned aircraft systems (UAS) and drones have been used in various shapes and sizes, ranging from handheld micro-UASs to medium-sized tactical systems to fully grown and Remotely Piloted Aircraft (RPA)1. UAS have found applications in agriculture, health, and military purposes2,3. Aside from their practical uses, such as search and rescue, surveillance, traffic monitoring, weather monitoring, firefighting, drone-based photography, videography, even delivery services, to name a few, drones continue to make headlines for their ability to additionally engage in all manner of recreational uses4–6. Unfortunately, terrorists and criminals are proving as innovative as their industry counterparts in finding novel uses for UAS7. The increasingly common use of drones by terrorists to launch strikes abroad has raised concerns that domestic male factors may plan and execute similar attacks. Meanwhile, some criminal actors use drones to smuggle drugs across borders and prisons or to support their nefarious enterprises8. These incidents, as well as others (which include unauthorized flights over sports stadiums or in controlled airspaces near airports), have exposed both the vulnerability of sensitive facilities and critical infrastructures to hostile or recklessly operated UAS, as well as serious shortcomings in the capabilities of law enforcement and national security agencies to address these threats9.
The need to protect people, infrastructure, and assets signifies the importance of counter Unmanned Aerial Systems, including detection, tracking, and identification systems1. As the availability of UAS or drones increases, law enforcement agencies face the unique challenge of regulating access to the lower airspace. Although commercial solutions are available, the evaluation of the capabilities of these systems is challenging. This results in end users struggling to match appropriate tools to specific use cases. In order to be able to evaluate DTI systems, there is a need to develop a set of appropriate standardized scenarios to encapsulate as best as possible the elements involved in the countering of malevolent actions launched by UAS10.
The operational needs and subsequent operational capability gaps that these standardized scenarios are called upon to address stem from the limitations of the C-UAS. Specifically, these limitations refer to the technical and non-technical obstacles faced by the detection and mitigation technologies used for C-UASs11.
To address this issue, the COURAGEOUS project aimed to create a standardized test methodology for detecting, tracking, and identifying malicious drones using countermeasure systems to safeguard the lower airspace. The proposed methodology relies on standard user-defined scenarios, covering diverse use cases, such as prison and airport security, critical infrastructure protection, border security, and combating drugs and human trafficking. COURAGEOUS end-users will extract the operational and functional performance requirements for these scenarios, forming the basis for developing a comprehensive test methodology. This methodology enables fair qualitative and quantitative comparisons of different counterUAS systems. Validation trials conducted in Greece, Belgium, and Spain confirm the effectiveness of the test methodology.
In the short term, the COURAGEOUS standardized test methodology will enhance LEAs’ understanding of counterUAS system capabilities. This knowledge will not only benefit COURAGEOUS partners but also contribute to the broader EU network of LEAs and the global dissemination of project results, which will be facilitated by the International Criminal Police Organization (INTERPOL).
Methods
The methodology for developing standard scenarios in the COURAGEOUS project involves four key steps (Figure 1), each aimed at systematically gathering information and insights to inform the creation of scenarios for countering unmanned aerial system (UAS) threats. A more detailed description of each step is provided below:
1. Literature Review / Need for Standardization Scenarios:
This step involved a comprehensive literature review to define the project scope. The focus is on collecting relevant literature related to UAS threats and analyzing them to identify the need for standardization scenarios. This initial review helps establish a foundation for the knowledge and understanding of existing research and practices in the field.
2. Previous Incidents & Identification of Gaps:
Building upon the literature review, this step involves analyzing previous incidents involving UAS threats. By studying past incidents, the project team aims to identify the common factors associated with these incidents and the gaps in the existing approaches to countering them. Recognizing patterns from past incidents is crucial for developing effective strategies for mitigating future threats.
3. Current C-UAS Framework (Methods & Technologies):
In this step, the project team examined the current framework of the (C-UAS) methods and technologies. This examination helps understand the existing landscape of tools, techniques, and technologies used for countering UAS threats. By assessing the current state-of-the-art, the team can identify areas for improvement and innovation in the C-UAS capabilities.
4. End-User Feedback:
This step involves gathering feedback from end users, including security authorities and other stakeholders involved in countering UAS threats. A questionnaire was created and distributed to COURAGEOUS partners, particularly end users. The feedback obtained helps to identify specific factors that should be considered when developing C-UAS scenarios. By incorporating end-user perspectives, scenarios can be tailored to meet the practical needs and requirements of those implementing them.
The integration of these four techniques ensures a comprehensive and systematic approach to scenario development that is applicable to various environments and security authorities. By selecting one scenario for each execution, the project aims to focus on developing high-quality relevant scenarios to address UAS threats effectively. Each step serves the specific purpose of gathering information and insights, ultimately contributing to the overall effectiveness of the scenario collection process.
Figure 1. Methodology diagram.
Need for standard scenarios/literature review
The rapidly expanding C-UAS industry serves as a crucial defense against rogue UAS activity, with systems designed to locate, track, and neutralize such threats. This demand extends across the commercial and law enforcement sectors, owing to the escalating threat posed by unidentified UAS in civil and law enforcement scenarios. Despite this, there remains an absence of international standards for C-UAS design, and not all systems perform as advertised, necessitating the development of global standards and policies as technology evolves12. Predictions estimate the global C-UAS market will reach two billion dollars by 2024, as highlighted in a comprehensive report titled "Report on 537 Anti-Drone Systems Shows How Wild the Market Has Become," showcasing a wide array of anti-drone technologies13. Notably, Bard College has contributed to the discourse on counterdrone Systems14.
C-UAS systems encounter specific challenges when dealing with small unmanned aerial vehicles (sUAVs), where detection plays a crucial role in providing initial threat information15. Various approaches, such as radiofrequency (RF) detectors16, active radars17, optoelectronic (OE) detectors18,19 coupled with AI-based classification systems20, LIDAR detectors21, and acoustic detectors22, offer early warning information about incoming UAVs. A methodology for evaluating different types of detectors, accompanied by an experimental test using the active radar system ReGuard developed by Retia, was formulated23. Furthermore, Doroftei24 presented a test methodology for qualitatively and quantitatively assessing the performance of multimodal drone detection systems. However, this test method focused on one specific detection system25 and lacked generality to enable its application to a wide range of C-UAS systems.
Effective C-UAS solutions require collaboration among numerous stakeholders, as demonstrated by the European Commission’s Joint Research Centre (JRC) in its development of a handbook based on experience gained from the DRONE project26. This collaborative effort involved targeted consultations and workshops with key stakeholders, providing advice on safeguarding against malicious UAS activities and offering guidelines across various stages, from detection to neutralization. These efforts align with the Commission’s C-UAS package, announced under the Commission communication ‘A drone strategy 2.0, for a smart and sustainable unmanned aircraft ecosystem in Europe27.
Continuing its support, the JRC has produced two handbooks, one outlining a five-phased approach to C-UAS solution development, and the other containing recommendations for assessing risks associated with the malicious use of UAS28. Additionally, a paper titled "The Methodology of Counter-UAS System Comparison" aimed to develop a method for comparing UAS detection systems, ensuring compatibility with diverse situations29.
By harmonizing European legislation and integrating unmanned aircraft systems into the common airspace (U-space), addressing unauthorized UAS operations in restricted areas is imperative. The paper titled "Procedures for Prevention of Unauthorized Incursion into UAS Geographical Zones" aims to establish a methodology and verify its functionality for intervening against unauthorized drone intrusion30. However, during the literature review, limited literature was found on the methodology for developing standard scenarios for C-UAS testing.
UAS have evolved significantly in size and application, raising concerns regarding their misuse by criminals and terrorists. C-UAS aims to address these threats but faces various operational needs and capability gaps31 (Table 1). Project COURAGEOUS collected data on over eight hundred and twenty-three drone incidents globally, revealing gaps and needs in the current technical countermeasures used by authorities32. While most incidents occur in restricted areas, free-access areas remain susceptible to illegal activities owing to challenges in deploying surveillance and detection technologies, which are often hindered by legislative barriers.
Table 1. Operational Needs and Capability Gaps.
Operational Needs and
Capability Gaps. | Explanation |
|---|
Minimizing False Negatives and
Positives | C-UAS detection systems must balance sensitivity to detect all drones with minimizing false positives
and false negatives to ensure effective operation. |
Distinguishing Legitimate and
Illegitimate UAS Use | In environments with common drone use, C-UAS operators need to differentiate between legitimate
and rogue drones quickly and reliably. |
Meeting Response Window
Constraints | Counter-drone operators have a limited time to assess and respond to potentially malicious drones,
and the increasing speed of commercial drones further reduces this response window. |
Improving Interdiction
Effectiveness | Various interdiction methods have limitations, such as the potential destructiveness of physical
capture and risks associated with high-power Electromagnetic Pulse (EMP), Radio Frequency (RF)
jamming, hacking, and spoofing. |
| Avoiding Interdiction Hazards | C-UAS techniques can be dangerous, posing risks to by-standers and legitimate communication
systems, emphasizing the need for precision and safety. |
Keeping up with UAS
Technology Advances | The dynamic nature of drone technology requires C-UAS systems to adapt to a wide range of drone
shapes, sizes, and capabilities. |
Tackling the lack of Operational
Data | The absence of information on the performance of deployed C-UAS systems makes it challenging for
users to make informed decisions. |
| Facing C-UAS Costs | Counter-drone technology incurs significant expenses, including per-sonnel training, maintenance,
and operational costs. |
| Achieving Legality | Legal uncertainties and ambiguities surround the use of C-UAS systems, with existing laws often
inadequately addressing this evolving technology. |
Dealing with the lack of
Standards | The absence of international standards for C-UAS design and use raises concerns about
performance variations, safety, and potential misuse. |
| Protecting Privacy | Counter-drone detection systems, as forms of surveillance technology, pose privacy risks if misused
or if data handling practices are inadequate. |
Although custom drones are not predominant in reported incidents, they pose a significant threat because of their ability to evade certain detection components with custom setups, thereby presenting challenges for the C-UAS systems. Additionally, the capability of drones to carry various payloads, including weapons or hazardous substances, presents a significant challenge for detection systems32,33.
Addressing these operational needs and capability gaps necessitates the development of standardized scenarios to improve the effectiveness of C-UAS systems in countering malicious actions launched by UAS.
Previous incidents & identification of gaps
The escalating occurrence of drone incidents has necessitated law enforcement agencies (LEAs) to adapt their responses to combat the diverse threats posed by criminals, uninhibited hobbyists, and inexperienced or reckless drone pilots. Through COURAGEOUS research, which meticulously analyzed over 800 global drone incidents spanning from May 2015 to August 2021, a clear need for education and training of responders has emerged. The analysis underscores the prevalence of uninhibited drone pilots as the most common threat actors, often pushing the boundaries of drone capabilities to infiltrate restricted areas or to satisfy their curiosity.
These incidents are meticulously categorized into three primary areas (airports, law enforcement, first responders, and private/non-corporate), revealing discernible patterns and characteristics. Most incidents occur in urban settings, predominantly during daylight hours, and involve multi-rotor UAVs equipped with a single commercial camera payload. Notably, most incidents are of marginal severity and cause disruptions rather than catastrophic outcomes. Europe and the United States bear the brunt of these incidents, whereas the Middle East confronts the threat of terrorism by employing UAVs34.
Further examination of more than 200 drone-related incidents in Europe shed light on the gaps and challenges prevalent in current technical countermeasures and offensive UAV capabilities. These challenges include the limitations of Drone Threat Intelligence (DTI) systems, difficulties encountered in congested urban environments, lack of weather-related preferences for illegal drone activities, potential complications arising from the presence of other UAVs, and prevalence of incidents in non-restricted areas. Additionally, the emergence of custom drones and diverse payloads present formidable obstacles for DTI systems35–37.
However, despite these challenges, DTI systems exhibit strengths, particularly in rural locales with favorable lighting conditions during daytime incidents, a preference for multirotor UAVs, incidents occurring in restricted areas under surveillance, and a predominance of off-the-shelf commercial drones that are relatively easier to detect. It is imperative to address the identified challenges comprehensively to bolster the effectiveness of countermeasures against malicious drone activities.
Current C-UAS framework (methods & technologies)
The COURAGEOUS project conducted an exhaustive examination of C-UAS frameworks, focusing specifically on the methods and technologies utilized for detecting, tracking, and identifying unmanned aerial systems (UAS). The analysis encompassed 260 anti-drone systems, shedding light on prevalent technologies and their operational characteristics38–40.
Detection Technologies: Frequency monitoring has emerged as the most commonly employed technology, utilized by 35.4% of the systems under review. Following closely are Microwave radar (30.4%), visible-light cameras (25.8%), and thermal cameras (19.6%). Other technologies included infrared (1.15%), lidar (2.3%), and acoustic sensors (6.5%).
Shortcomings of Systems: Several systems lack the capability for 24/7 operation, particularly during nighttime. Certain technologies, such as infrared sensors and visible-light cameras, exhibit limitations in terms of the range and image quality. Microwave radar analysis requires approval, and is susceptible to atmospheric conditions. Moreover, acoustic- and radio-frequency-based systems depend on the background noise level in the testing environment.
Operability and Mobility: The majority of C-UAS systems (74%) are designed for permanent installation, with a substantial portion also offering mobile versions (48%). In addition, some systems could be mounted on vehicles or trailers (17%).
Combination of Technologies: Multispectral systems are regarded as the most effective for defining alarm signals. While the majority of systems rely on a single technology (53%), with frequency monitoring being the most prevalent (31.4%), a notable percentage utilizes multi-technology configurations, integrating various detection and tracking technologies.
Challenges and Considerations for Standardization: This analysis underscores the necessity of a standardized methodology that considers detection effectiveness, range, resistance to false indications, system functionality in diverse scenarios, and resilience to changing environmental conditions. Technologies can be categorized as either active (e.g., radar and lidar) or passive (e.g., cameras), with passive technologies dominating C-UAS solutions.
Environmental Impact on Technologies: Environmental factors, such as rain and strong sunlight can disrupt the operation of certain technologies. For instance, rain may impede the functioning of Visible Imaging Sensor (VIS) cameras and acoustic sensors, whereas strong sunlight can affect VIS, thermal imaging cameras, infrared (IR) sensors, and lidars. However, the frequency monitoring remains unaffected by rain or fog.
Object Identification Process: The object identification process faces numerous challenges, including weather conditions, lighting, distance, object size, and terrain-covered zones. A clear definition of the movement of a test object within the detection field is crucial for accurate testing.
In conclusion, this study underscores the complexity involved in selecting and implementing C-UAS technologies, highlighting the significance of a standardized methodology that considers various operational parameters and environmental conditions3,41.
Law enforcements feedback
A pivotal step involves the creation and circulation of a comprehensive questionnaire among the end users within the consortium. This questionnaire aimed to gather insights into the prevalent factors and components inherent in the UAS threat scenarios. By leveraging the collective expertise of stakeholders, the objective is to discern and validate the parameters crucial for shaping these scenarios, thereby ensuring their efficacy in addressing potential UAS threats33,42.
The overarching goal was to ascertain the requisite level of detail necessary to effectively categorize these scenarios. Through rigorous data collection and analysis, this project sought to gain a nuanced understanding of the diverse contexts and circumstances surrounding UAS attacks. By collating inputs from end-users across various domains, the aim is to capture a comprehensive spectrum of potential threat scenarios encompassing different environments, operational contexts, and adversary behaviors.
This meticulous process of soliciting feedback and integrating diverse perspectives underscores a project's commitment to develop robust and adaptable standard scenarios. By aligning these scenarios with the identified parameters and stakeholders' real-world experiences, the COURAGEOUS project endeavors to enhance preparedness and response capabilities against UAS threats. Ultimately, the initiative strives to provide a structured framework that enables stakeholders to anticipate, mitigate, and effectively manage the evolving landscape of UAS-related risk.
Table 2. Parameters extracted from the questionnaire.
| Parameters | Explanation |
|---|
| Target | Identifying different objectives and targets of malicious UAVs. |
| Environment | Considering the area or environment where the UAS threat scenario occurs. |
| Lighting Conditions | Recognizing lighting conditions as a critical component affecting UAS threat scenarios. |
| Weather Conditions | Understanding how weather conditions impact both malicious UAVs and deployed
countermeasures. |
| Presence of Other Aircrafts/UAVs | Considering the influence of nearby aircraft on UAS threat scenarios and implemented
countermeasures. |
| Type of UAV | Recognizing the contribution of the UAV type to the development of a UAS threat scenario. |
| Number of UAVs | Acknowledging the critical role of the number of UAVs in UAS threat scenarios. |
| Custom or Commercial | Assessing how the manufacturing method of the UAS affects the risk level of a threat scenario. |
| Dimensions of UAV | Considering the physical dimensions of the UAV, including wingspan and rotor characteristics. |
| Maximum Take-Off Mass of UAV | Recognizing the impact of the maximum take-off mass on UAS threat scenarios. |
| Flight Mode | Understanding how the pilot’s navigation method, or flight mode, contributes to the threat
scenario. |
| Payload | Identifying the payload carried by the UAV and its role in defining mission objectives. |
| Altitude | Considering the altitude at which the UAV operates as a component of the threat scenario. |
| UAV Speed | Recognizing the impact of UAV speed on risk levels, prevention, and countermeasures. |
| Radio Frequencies | Identifying the radio frequencies used for remote control and video streaming as components
of the UAS threat scenario. |
| Flight Behavior | Assessing the trajectory and behavior of the UAV during flight, which can affect established
countermeasures. |
It is emphasized that all these components must be included in scenario development, regardless of their significance, and that a scoring procedure is not suitable for this activity.
The responses collected through the questionnaire supported the creation of a UAV threat scenario element pool, thereby contributing to the definition of the standard threat scenarios.
Development of standard scenarios
Following the structured approach outlined in the methodology for developing standard scenarios in the COURAGEOUS project, Table 3 delineates the components that combine incidents and scenarios. This table serves as a comprehensive reference, and presents a detailed breakdown of the primary factors associated with incidents and their respective subcategories. For example, within the weather factor category, the subcategories encompass various weather conditions such as Sunny, Cloudy, Rainy, Foggy, Windy, Stormy, Smoke, Dusty, Snowy, and Clear.
Table 3. Scenario Factors.
| Target | Weather | Type of UAV | Maximum take-off mass of UAV | Altitude | UAV Speed | Environment | Lighting Conditions | Presence of other aircrafts/
UAVs in the nearby airspace | Number of UAVs | Flight Mode | Radio Frequencies used for
Remote Control and / or Video
Stream | Flight Behaviour | Pilot Location | Payload | Custom or Commercial | Dimensions of UAV | Terrain | EM Environment | Birds | Vegetation | UAV Signature |
|---|
Critical
Infrastructure | Sunny | Multirotor | <250g | 0-5m | 0-10km/h | Rural | Sunrise | Not
Classified | 1 | Manual | 2.4GHz | Direct
flight | Known | Optical camera | Recreational
custom-
made UAS | <30cm | Flat | Rural | Low Bird
presence | None | Strong |
Government
Building | Cloudy | Fixed Wing | <900g | 5–20m | 10–30km/h | Suburban | Sunset | Not
Controlled | 2 | GPS | 5.8GHz | Obscured | Unknown | LiDAR | Wrong-doing
custom-
made UAS | 30–50cm | Irregular | Suburban | Normal
Bird
presence | Low | Normal |
| VIP | Rainy | Flapping-
wing
Ornithopters | <4kg | 20-50m | 30-60km/h | Urban | Daylight | Own Fleet | Swarm | Waypoints | RC model
aircraft
frequencies
(depending
on national
regulations) | Drop
from High
Altitude | | Thermal
sensor | Commercial | 50–70cm | Mountainous | Urban | High
Bird
presence | Average | Low |
| Public Event | Foggy | Gliders | <25kg | 50–100m | <60 km/h | | Darkness | Friend | | Inertial
Navigation
Systems | 4G/LTE | | Explosives/
IEDs | | >1m | | Dense/
Crowded | | Meadowland | None |
| Windy | Single Rotor | <100kg | 100–120m | 60–120
km/h | | Authorized | 4G/LTE | Other | Gun | | | Wood /
Forest | |
Urban - Not
Specified | Stormy | Hybrid | >100kg | >120m | 120<160
km/h | | | Stolen | | | | | | CBRN | | | | | | | |
Border
Means of
Transportation | Smoke | | | | >160km/h | | | Alleged
Infringer | | | | | | Objects for
Commercial
Distribution | | | | | | | |
| Dusty | | | Threat | None | Sprayers | | | | | | |
| Snowy | | Escaped | Noise
Generators | | | | | | |
| Clear | | Neutralized | Jamming
Devices | | | | | | |
| | | | Different
Domestic
Payloads | | | | | | |
| | | | | | | | | | | | | | Dazzling
Laser | | | | | | | |
| | | | | | | | | | | | | | Illicit Package | | | | | | |
By systematically categorizing these factors and subcategories, the table facilitates a thorough understanding of the diverse elements that may influence incident occurrence and scenario development. Each factor and its associated subcategories provide valuable insights into the multifaceted nature of UAS-related incidents, encompassing a broad spectrum of environmental, operational, and contextual considerations.
This structured approach not only enables comprehensive documentation of incident characteristics, but also lays the groundwork for defining standardized scenarios that reflect real-world scenarios encountered by stakeholders. By providing a clear and organized framework for analyzing and categorizing incident components, Table 3 serves as a valuable tool for guiding the development of standard scenarios within the COURAGEOUS project, ultimately enhancing preparedness and response capabilities against UAS-related threats.
In the formulation of a standard scenario within the COURAGEOUS project, the pilot's intention is recognized as a critical factor, distinguishing between deliberate and accidental actions. The subcategories of the pilot's intention encompass various motivations, ranging from negligence to criminal or terrorist intent34,43.
Under negligence, two distinct subcategories are identified:
Clueless Individuals: Pilots who are unaware of or unfamiliar with regulations fly inadvertently over sensitive areas due to a lack of knowledge.
Careless Individuals: Pilots who are aware of regulations but violate them unintentionally, flying over sensitive areas without disruptive intentions.
Moving to gross negligence, two additional subcategories are outlined:
Reckless Individuals: Pilots who knowingly disregard regulations for personal or professional gain, causing disruption.
Activists/Protesters: Pilots who, whether aware of regulations or not, actively use drones to disrupt, potentially as a group, without harming human lives.
Finally, the criminal/terrorist motivation category encompasses pilots deliberately seeking to use drones for disruption, disregarding human life and property, and criminal or terrorist motivations.
Table 3 serves as a comprehensive reference, identifying and describing the factors that could constitute a scenario, including the pilot's intention. The next phase in developing standard scenarios involved combining these factors with the pilot's intent. A unique scenario can be defined by selecting specific values for the pilot's intent and each factor. This approach ensures adaptability to various users, allowing the creation of multiple scenarios based on specific selections, thereby enhancing the preparedness and response capabilities against a diverse range of UAS-related threats44,45.
Standard scenarios
For the COURAGEOUS project requirements, ten standard scenarios were devised and categorized into three main groups (Figure 2):
• Sensitivity Sites/Critical National Infrastructure
• Public space protection/events and
• Border Protection (land – maritime).
Figure 2. Standard Scenarios per category.
Each scenario provides a real-world event example, offering an overview of the operational context and its limitations. These non-technical high-level representations aim to portray events of interest from an LEA perspective, illustrating the threat and problem spaces through narrative examples. These scenarios, along with the use cases, guide technical developments that influence technological demonstrations, trial designs, and evaluations.
In the first scenario, the test was conducted in a prison area with diverse surroundings such as small trees, private houses, and public roads. A manually operated Class I mini-category multirotor UAV carries a drug payload and takes off from a concealed position, crossing a prison fence. Notably, a "friendly" UAV is also airborne, making this scenario pertinent for addressing concerns regarding careless use of UAVs near airports. In the second scenario, the setting shifts to an airport area featuring a Class I fixed-wing UAV with an optical camera payload. This UAV operates autonomously, contending with general aviation activities, as it attempts to breach airport limits. This scenario addresses security challenges inherent in high-traffic airport environments. The third scenario involves a nuclear power plant located in an isolated rural area. During the night and under cloudy weather conditions, a custom-made multirotor UAV with an explosive payload autonomously takes off with the aim of approaching a nuclear plant via controlled waypoints. The relevance of this scenario lies in the imperativeness of securing critical infrastructure to protect against external threats. In the fourth scenario, which was situated in a large urban area, three UAVs were employed for physical attacks during an afternoon event. These UAVs, equipped with video equipment to coordinate the attack, target VIP at different frequencies. This scenario underscores the significance of security measures against potential criminal activity during major urban events. The fifth scenario occurred at a suburban stadium during a concert where a custom-made multirotor UAV with an aerosol-dispersing device disrupted the event. This highlights the need for robust security protocols to counter potential threats during large public gatherings. In the sixth scenario, during an outdoor concert in a bustling urban area, a non-friendly UAV equipped with a € laser payload is detected and destroyed. This scenario emphasizes the importance of security measures against unauthorized UAVs in crowded settings. The seventh scenario occurs in the middle of a city during an authorized rally, when a commercial multirotor UAV carrying a noise generator payload disrupts the event. This exemplifies how criminal organizations may employ UAVs to induce panic and disturbance during public gatherings. Moving on to the eighth scenario, set at a historical international summit, a commercial fixed-wing UAV was utilized by a terrorist organization to create panic. The UAV autonomously approaches the VIPs and explodes, thereby releasing shrapnel. This underscores the significance of security measures for counteracting potential terrorism during crucial international events. In the ninth scenario, positioned at the land border, a custom-made multirotor UAV drops packages in a forested area, highlighting the challenges of combating trafficking activities at the border. Finally, the tenth scenario unfolds at the sea border, where a commercial multirotor UAV is used for surveillance to avoid coast guard boats during immigrant boat crossings. This scenario underscores the need for effective border control and surveillance by using UAVs.
These scenarios are realistic representations within the C-UAS domain, involving multiple actors, events, and activities and addressing various use cases. The presentation of standard scenarios followed the template used by the consortium partners, incorporating incident factors. Abstract Use case scenarios were described for specific areas without defining the interaction between the COURAGEOUS system and the user. The factors outlined in Table 2 were utilized to construct scenarios, with the consortium carefully selecting factors for adaptability and comprehensive coverage of the DTI system capabilities.
Methodology evaluation
The initial evaluation of standardized scenarios involved active participation from consortium members and law enforcement agencies (LEAs) within the consortium. The development process for these standard scenarios underwent rigorous discussion and received approval from the consortium. Ultimately, ten Standard COURAGEOUS Scenarios were formally presented and endorsed by the end users and consortium. Subsequently, further evaluation of these scenarios was conducted in a specialized workshop attended by consortium end users and relevant stakeholders.
During the workshop, participants engaged in discussions regarding standardized procedures within the methodology. Following these discussions, a questionnaire was distributed to gather feedback from participants.
The questionnaire comprised three key questions aimed at assessing participants' perspectives and satisfaction with the standardized scenario methodology. First, participants were asked to indicate their organization's level of interest in the Standardized Scenario Methodology, with response options ranging from "Slightly interesting" to "Extremely interesting." The majority of respondents (80%) rated their interest level as either "Very interesting" or "Extremely interesting."
Second, participants were asked whether the scenarios met their requirements, with response options being "Yes," "No," or "Maybe." Maybe. The overwhelming majority (80%) indicated that the scenarios met their requirements.
Finally, participants were requested to score their agreement with the extent to which standardized procedures were followed for each scenario. The scoring options ranged from strongly disagree’ to strongly agree. ’ The resulting responses, depicted in Table 4, illustrate the consensus among participants regarding adherence to standardized procedures for each scenario.
Table 4. Evaluation Question 3 – Results.
| Strongly
Disagree | Disagree | Neither Agree
nor Disagree | Agree | Strongly
Agree |
|---|
| Prison | - | - | - | 60% | 40% |
| Airport | - | - | - | 80% | 20% |
| Nuclear Plant | - | - | - | 80% | 20% |
| Government Building | - | - | - | 40% | 60% |
| Stadium | - | - | - | 40% | 60% |
| Outdoor Concert | - | - | - | 60% | 40% |
| Outdoor Political Rally | - | - | - | 40% | 60% |
| International Summit | - | - | 20% | 40% | 40% |
| Land Border | - | - | 20% | 40% | 40% |
| Maritime Border | - | - | 20% | 40% | 40% |
Results
In this article, validation trials were conducted across multiple locations, including Greece, Belgium, and Spain. These trials confirmed the effectiveness of the standardized testing methodology developed by the project. For instance, the methodology enabled the fair, qualitative, and quantitative comparison of various counter-UAS (C-UAS) systems under different operational environments (e.g., prisons, airports, critical infrastructures). The validation trials produced key data that demonstrated how well these systems performed in diverse use cases like prison area drug smuggling or airport security breaches.
Use cases
The project created 10 standard scenarios covering areas like critical infrastructure protection, public event security, and border control (both land and maritime). For example, one scenario involves a drone carrying an explosive payload attempting to breach a nuclear power plant, which simulates a real-world threat to critical infrastructure. These scenarios not only address various threat levels but also inform the broader design of C-UAS systems, making them more adaptable to different real-life applications. This section should highlight how these scenarios reflect the COURAGEOUS project’s practical implementations, even before full validation is completed.
Implementation expectations
The COURAGEOUS project provides a structured approach to scenario development, based on insights from over 800 global drone incidents and extensive feedback from law enforcement agencies (LEAs). The methodology aims to address key gaps in C-UAS capabilities, such as improving the detection of small, custom drones and reducing false positives/negatives in detection systems. These standardized scenarios offer a framework for future research and system development, ensuring C-UAS systems can be evaluated consistently across different countries and operational contexts. Peer review will further help refine and adapt the methodology, adding value to the field by promoting international collaboration and enabling the global dissemination of results through bodies like INTERPOL.
Conclusions
In conclusion, the COURAGEOUS project endeavors to establish a unified understanding among Member State authorities regarding the effectiveness of technical systems designed for detecting, tracking, and identifying noncooperative drones. This project focused on creating a uniform test methodology using a set of standard scenarios that describe situations that require protection against malicious UAS. The key outcomes of COURAGEOUS research include a collection of standard scenarios and use cases, which forms the basis for future trials in the project. The project methodology involves a comprehensive approach, starting with a literature review to define the scope and identify the need for standardization scenarios. The analysis of previous incidents, identification of gaps, examination of the current C-UAS framework, and an end-user questionnaire contributed to the development of standard scenarios. The identified parameters from the questionnaire covered a wide range of factors, including the target, environment, lighting conditions, presence of other aircraft, and more, ensuring a holistic approach to the scenario development. The evaluation process involved both consortium members and LEAs within the consortium to ensure the relevance and effectiveness of standardized scenarios. Ten standard scenarios were developed and endorsed by end users, representing diverse use cases such as critical infrastructure protection, public event security, and border protection.
The COURAGEOUS project recognizes the evolving nature of drone technologies and challenges faced by LEAs in countering malicious drone activities. By addressing operational needs and capability gaps through standardized scenarios, the project aims to enhance the effectiveness of C-UAS in responding to potential threats. The developed scenarios not only serve as a foundation for upcoming trials, but also contribute to the broader EU network of LEAs, facilitating knowledge sharing and dissemination of project results. Moving forward, a standardized test methodology and scenarios will be applied to a broader range of commercial C-UAS systems to provide valuable quantitative data for informed design decisions.
Looking ahead, the COURAGEOUS test methodology will be applied to a broader range of commercial counter-UAS systems in the medium to long term. This extended testing will provide developers with quantitative data, enabling them to make informed design decisions based on the performance metrics.
Abbreviations
The following abbreviations are used in this paper:
CBRN Chemical, Biological, Radiological, or Nuclear substance
C-UAS Counter – Unmanned Aerial System
DTI Detection-Tracking-Identification
EC European Commission
EMP Electromagnetic Pulse
EU European Union
IEDs Improvised Explosive Devices
IR Infrared
ISFP Internal Security Fund Police
LEAs Law Enforcement Agencies
OE Opto Electronic
RF Radio Frequency
sUAV Small Unmanned Aerial Vehicles
UAS Unmanned Aerial System
UAV Unmanned Aerial Vehicle
VIS Visible Imaging Sensor
Ethics and consent
The participants who responded to the questionnaire are partners (end-users) of the EU-funded project entitled COURAGEOUS. An informed consent procedure was strictly followed to ensure voluntary participation and lawful data processing. In line with research ethics, all respondents were fully informed about the study and provided their consent prior to the completion of the questionnaire. Specifically, participants were asked to carefully review the Information Sheet, which detailed, among others, the study’s purpose, participants’ rights and contact details of the lead researchers. Informed consent was obtained through tick boxes, freely checked by the participants after having reviewed this information.
