Introduction
During various phases of the COVID-19 pandemic, particularly with the emergence of the Omicron variant, manual contact tracing faced significant challenges due to the overwhelming caseloads1. Additionally, manual contact tracing is limited by recall bias, since positive cases may not remember every relevant proximity they have had to another person, nor have identifiable information on all these contacts. Digital contact tracing (DCT) solutions have been suggested as a supplement or improvement to contact tracing2, as it has been expected to notify contacts quicker and more comprehensively than manual contact tracing is able to3,4.
These expectations led more than 50 countries5 to develop and deploy DCT solutions during the COVID-19 pandemic, including at least 27 European countries6. Thus, a substantial proportion of the world’s population will have had a DCT app installed on their smartphone at some point during the COVID-19 pandemic. The DCT apps were heterogeneous in many aspects, for example with regard to system architecture and the technology used. Some apps relied solely on Bluetooth, while others integrated GPS, manual contact tracing, venue QR-code scanning or other proxies for exposure, e.g., the Chinese health code system which relied on data mining in order to assign each user a risk code (green, yellow or red)7.
Centralized DCT tools have collected data on cases and connected them to contacts in a centralized database. Due to concerns about losing public trust related to interference with privacy8, decentralized DCT solutions relying only on Bluetooth technology were chosen by many countries, and Denmark was among the first. Decentralized apps prioritize user privacy and store little or no personal data on users, but this in turn impedes public health authorities’ ability to collect data on the effectiveness of the apps and improve the technology accordingly. Details on key differences in design and technology of COVID-19 DCT tools are published elsewhere8–10.
If DCT apps are to play a role in future epidemics, it is necessary to evaluate their performance during the COVID-19 pandemic. Several studies evaluating individual DCT solutions in different countries have been published with mixed results in terms of performance11–22. The aim of this paper is to evaluate the performance of Smittestop, the official and only DCT app implemented in Denmark.
Methods
Study period
The study covers the period from November 22, 2020 to November 27, 2021. This one-year period was selected because a plateau in downloads of Smittestop was reached around November 2020, and because contact tracing and test capacity remained on par with the pandemic until the arrival of the Omicron variant in the end of November 202123. The stringency index in Denmark fluctuated dramatically during the study period, with lockdowns in the beginning and no restrictions in place towards the end24. Vaccinations were rolled out in the beginning of 202124. Throughout the study period, individuals who received an alert from the Smittestop app were advised to book a PCR test and follow general public health recommendations. However, self-quarantine based solely on a Smittestop notification was not officially recommended after November 2020. In the entire study period, there was free and easy access to SARS-CoV-2 RNA PCR-tests. Denmark has been one of the countries performing the most PCR-tests per capita25.
Materials
This study compiles data reported from several different sources as detailed below.
Smittestop: Smittestop was developed in collaboration with the Danish Ministry of Health, the Danish Patient Safety Authority and the Danish Agency for Digital Government. The app was updated according to any changing public health measures. Smittestop was decentralized, among the first apps to use the Google/Apple Exposure Notification application programming interface (API), a privacy-preserving system relying on Bluetooth within a smartphone to exchange beacons of identifiers with other Smittestop apps. It was anonymous and used no location tracking (GPS) or location tracking proxies (e.g. venue registration)8. For this reason, only basic information obtained from the app distributor are available for analysis: weekly number of downloads and of users who shared a positive test result26.
Smittestop was operational from June 18, 2020 after which began an “initialization phase” where download frequency was high (see Figure 1a in the Results). Most of the changes made in the technical settings (time- and distance criteria) of the app took place in the initialization phase.
Figure 1.
1a: Cumulated number of downloads of Smittestop since implementation in June 2020. 1b: the number of PCR-tests for SARS-CoV-2 performed in Denmark. 1c: the proportion of tested individuals who answered the test indication questionnaire, when booking their test on www.coronaprover.dk. Grey-shaded area marks the period analyzed in this study, x-axis shows calendar weeks.
The manual contact tracing: In the study period, the Danish Patient Safety Authority (DPSA) was in contact with 95% of confirmed SARS-CoV-2 infected individuals by telephone. This includes PCR-positive individuals as well as individuals who tested positive on lateral flow tests (LFT) in public test centers27. Some cases interviewed by DPSA in relation to manual contact tracing were asked if they had Smittestop installed on their smartphone.
The national surveillance of COVID-19 at Statens Serum Institut (SSI): This automated surveillance system has collected information on all COVID-19 cases from different registers each day. Data on SARS-CoV-2 tests are retrieved from the Danish Microbiology Database (MiBa) and linked to data from a number of national registers. The system is described in detail elsewhere28. Users were only able to share positive results through Smittestop, if their positive test appeared in MiBa, and publicly available LFTs were not entered into MiBa until the beginning of 2021.
Test indication questionnaire on www.coronaprover.dk: Danish citizens used the website www.coronaprover.dk to book a time for a PCR-test25, with a voluntary questionnaire available to indicate the reason for testing. One of the reasons for testing that could be selected was: “received warning from Smittestop app of potential COVID-19 exposure”. Other examples were: “I have symptoms that could be of COVID-19”, “my household member has COVID-19” and “I work in elderly care”26.
Statistics Denmark29: was used to obtain overall figures on the Danish population.
Data analyses
The indicator framework developed by ECDC and WHO was used as a guideline for this evaluation30. The majority of indicators are reported excluding “time to notification” for individuals warned of exposure through the DCT tool compared to that of manual contact tracing, as these indicators were not available.
All statistical analyses of data and graphical representations were conducted using R version 4.1.331. Analyses were primarily descriptive. Week 12, 2021 showed erroneous numbers from the app (see Figure 2 in the Results) and was excluded from data analyses. All other data was assumed trustworthy.
Figure 2.
2a: Weekly number of PCR-positive SARS-CoV-2 cases and the proportion of these cases that were also registered as infected on Smittestop. 2b: Ratio between the number of infection registrations to Smittestop per 100.000 users and the number of PCR-positive cases per 100.000 Danish inhabitants. Grey-shaded area marks the period included in this study, x-axis shows calendar weeks.
Percentage of positives and 95% confidence intervals were calculated using data from www.coronaprover.dk. The precise generation time, here defined as the interval between when an index person shares their positive test on Smittestop, and until notified contacts of this person present at testing services, was unknown. For simplicity, tested individuals were assumed to have received a notification from Smittestop in the same week as their test.
To estimate the number of cases that could have been identified through Smittestop if coverage had been higher, we calculated the ratio between the probability of testing positive when reporting Smittestop as the reason for being tested and the probability of answering the questionnaire when booking a test. The result of this was used as a correction factor to make extrapolations and related confidence intervals. In the extrapolations several assumptions were made: that Smittestop was uniformly distributed in the whole population, that questionnaire answers were fully representative of reasons to test, and therefore that questionnaire answers were identical to test indications in the group of tested individuals who did not answer the test-indication questionnaire.
Ethical statement
This study was conducted using data from the Danish COVID-19 surveillance as part of a task imposed on Statens Serum Institut according to Danish law (Sundhedsloven § 222). Therefore, no ethical approval was required.
Results
App downloads
Smittestop was downloaded 2,422,756 times by the end of the study period. According to Statistics Denmark, the Danish population included 5,867,412 individuals (in the 4th quarter of 2021). Assuming that every download represents one person, 41.3% (95%CI=41.3-41.3) of the Danish population had downloaded the app by the end of the study period.
Figure 1a shows, that most downloads were made shortly after implementation of the app and within the first few months thereafter. About 400,000 downloads (16.8% of all downloads completed by week 47, 2021) were made in the first week alone, and a stabilization in downloads was reached approximately a half year after implementation. Only a small fraction of the total downloads was made in 2021.
App distribution among positive cases
Among the cases interviewed by the DPSA, 85.3% were asked whether or not they had Smittestop. Of these, 23.1% reported that they had the app installed (corresponding to 19.7% of all individuals interviewed), Table 1.
Table 1. Distribution in the answers from interviews of manually traced SARS-CoV-2 infected individuals in the study period to the question: “Do you have Smittestop on your smartphone?”.
| Number of cases* |
|---|
| Yes | 81,913 (19.7%) |
| No | 272,674 (65.6%) |
| Undisclosed/Not asked | 61,110 (14.7%) |
| Total | 415,697 (100%) |
There were 409,486 PCR-confirmed cases of COVID-19 in the Danish population in the study period, according to numbers from the national surveillance at SSI. Furthermore, there were 94,745 positive test shares on Smittestop. Assuming that these individuals were PCR-tested, approximately 23.1% (95%CI=23.0%-23.2%) of confirmed SARS-CoV-2 infections were entered into the app.
The pattern of positive test shares on Smittestop almost follows the pattern of confirmed infections in the population (Figure 2a). Week 12, 2021 differs significantly, which is likely due to a technical error that occurred in the app on March 26, 2021. A decline over time is observed in the ratio of infections that were shared through Smittestop (Figure 2b).
Percentage of positives
During the study period, 25.8 million tests for SARS-CoV-2 were booked through www.coronaprover.dk (Table 2). Public test activity varied during the study period, as seen in Figure 1b, some weeks just below one million individuals were tested per week (about one sixth of the Danish population) while about 100.000 tests were performed in other weeks. Willingness to answer the test indication questionnaire declined during the study period (Figure 1c), from around 60% answering the questionnaire by the end of 2020, to less than 25% over the summer of 2021.
Table 2. Aggregated numbers from the test indication questionnaire on www.coronaprover.dk, the website where Danish citizens book test times for SARS-CoV-2 PCR-tests.
| All tests,
coronaprover.dk | Test
indication
questionnaire
answers | Selected indications for testing (given in the questionnaire on
coronaprover.dk) |
|---|
Symptoms
that could
be COVID-
19 | Close
contact | Notification
from
Smittestop
app | Contact
in the
household | Screening test
(because of
work/education
etc.) |
|---|
Number
of people
tested | 25,841,849 | 8,835,256 | 659,244 | 1,387,784 | 117,171 | 223,048 | 2,137,800 |
Number of
females (%) | 13,927,164
(53.9%) | 5,015,864
(56.8%) | 402,023
(61.0%) | 776,934
(56.0%) | 71,375
(60.9%) | 118,419 (53.1%) | 1,236,492 (57.8%) |
Average age
in years | 38 | 41.0 | 33.6 | 33.7 | 43.9 | 33.0 | 45.4 |
Number of
positive tests | 269,411 | 92,842 | 18,097 | 50,347 | 1,221 | 21,314 | 3,832 |
Percentage-
positive
[95%CI] | 1.0 [1.0-1.0] | 1.1 [1.0-1.1] | 2.7 [2.7-2.8] | 3.6 [3.6-
3.7] | 1.0 [1.0-1.1] | 9.5 [9.4-9.6] | 0.2 [0.2-0.2] |
Number
needed to
test to find
one person
positive for
COVID-19 | 96 | 95 | 36 | 28 | 96 | 11 | 558 |
For 8.8 million (34.2%, 95%CI=34.2-34.2) a test indication was reported in the questionnaire. 117,171 (1.3% of the 8.8 million answers) reported a Smittestop exposure notification as their test indication. The group (herein after referred to as the “Smittestop group”) includes more women (60.9%) and had a mean age of 43.9 (Table 2).
Within the Smittestop group, 1,221 were SARS-CoV-2 positive resulting in a test positive rate of 1.0% (95%CI=1.0%-1.1%). This was higher than among individuals receiving PCR-tests for screening purposes (0.2%) but lower than close contacts (3.6%). Household contacts reached a 9-fold higher percentage-positive than the Smittestop group. There was no significant difference in the percentage of positives between the Smittestop group and all individuals tested (RR=1.0, 95%CI=0.9-1.1) (Table 2).
In the Smittestop group, 14.5% (n = 16,986) also specified another reason for test. In the group that specified Smittestop as the sole indication (n=100,185), 761 were SARS-CoV-2 positive (giving a percentage-positive = 0.8%, 95%CI=0.7%-0.8%).
Percentage of positives during different periods
The predictive power of the app was slightly lower, but not significantly (RR=0.89, 95%CI=0.72-1.11, p=0.32) during a period of high SARS-CoV-2 prevalence (winter 2020-2021) compared to a period of low prevalence (summer 2021), see Table 3. These two periods differed by SARS-CoV-2 prevalence, public health measures in place, test activity, and proportion who answered the test indication questionnaire.
Table 3. Aggregated numbers from the entire study period compared to a winter period and a summer period.
Numbers shown are: individuals tested because of Smittestop, confirmed cases and percentage of positives among these, percentage of tested individuals who answer the questionnaire overall as well as the number of Smittestop infection registrations.
| Period | Confirmed cases
among individuals
tested because of
Smittestop | Tested because
of Smittestop | Percentage
positive | % who answer the
test indication
questionnaire | Number of
Smittestop
infection
registrations |
|---|
Entire study
period | 1,221 | 117,171 | 1.0 [1.0-1.1] | 34.1% | 93,260 |
Winter
2020–2021
(week 48,
2020 to week
3, 2021) | 646 | 62,439 | 1.0 [1.0-1.1] | 63% | 36,658 |
Summer 2021
(week 22,
2021 to week
34, 2021) | 88 | 9,523 | 0.9 [0.7-1.1] | 20.3% | 12,678 |
Extrapolations
Table 4 shows results of extrapolations from an assumed true coverage of 23%. A further 1,970 to 3,048 cases might have been found through Smittestop, if 30% of the population had been “active users”.
Table 4. Illustrative extrapolations of identified cases through Smittestop if coverage had been higher than 23% (the assumed true coverage).
| Coverage | Correction
factor1
[95%CI] | Cases identified
through
Smittesmiddle2
n | Adjusted number of
cases identified through
Smittesmiddle3
n [95%CI] | Hypothetical
Increased
coverage | Extrapolated
number of cases
that could be found3 |
|---|
| 23% | 0.031
[0.029-0.032] | 1,221 | 3,571
[3,377-3,776] | 30% | 6,076 [5,746-6,425] |
| 40% | 10,802 [10,215-11,421] |
| 50% | 16,877 [15,961-17,830] |
| 60% | 24,303 [22,984-25,698] |
Discussion
In this study, we presented data collected on Smittestop in Denmark during a one-year pre-Omicron period of the COVID-19 pandemic. We aimed to assess whether Smittestop identified infectious cases accurately. Despite the predictive value seeming low (percentage-positive of 1%), some of these identified cases (761 among individuals answering the questionnaire) may represent random contacts e.g. in trains or at venues. The percentage of positives among this type of contact (0.8%), was higher than in mass screenings (0.2%). In many cases, close contacts are already told directly by the infected person, so the app's added value lies in uncovering random or unknown contacts that manual tracing might miss. This kind of extra detection could have helped reduce the spread of the virus.
We found that Smittestop was downloaded 2,422,756 times in a population of 5,867,412 individuals, indicating significant initial public interest and support for the app. To begin with, the number of positive test shares on Smittestop per 100,000 users closely mirrored the number of PCR-positive cases per 100,000 inhabitants. However, over time, there was a decline in the number of positive test shares, likely due to an increase in unused apps. Worry of high battery consumption by the app may have contributed as well as stories about technical malfunctions of the app that were shared on social media and in the public press. For example, before the end of 2020, Android devices did not receive push notifications for potential exposure; instead, users had to manually open the app to view this information. Finally, some individuals may have become tired of receiving exposure notifications and/or a general pandemic fatigue may have played a part.
Extrapolating our data, with the limitations of such crude extrapolation in mind, yielded results closer to the mathematical modelling studies from the early pandemic, that suggested a favorable impact of DCT on the prevention of transmission of COVID-193,32–34. The number of cases identified by the app would have been substantially higher if coverage had been higher, especially if true coverage was low.
While high coverage is important, the long-term success of digital contact tracing tools depends equally, if not more, on the accuracy and usability of the technology, which are critical for retaining active users. The settings (distance- and time criteria) of the app and the technology on which it depends will influence how many and how accurately the app traces contacts. The published evaluation of the Australian COVIDsafe app, a centralized DCT tool where all DCT notifications are handled by manual contact tracers, underlined the demanding structure of this type of DCT, but also revealed situations where persons were classified as possible contacts because they had been in adjacent rooms13. Denmark was among the first countries to deploy a DCT app dependent on Bluetooth, thus the technology was at its infancy when Smittestop was developed and implemented, and notifications sent to contacts could have been imprecise to some degree and could not be continuously evaluated and adapted due to the high privacy requirements. Inaccurate notifications and the mentioning hereof in the media are likely to decrease adherence to usage of any DCT app. Emphasis in any future DCT systems should focus on minimizing unnecessary or inaccurate notifications.
In the event of a new pandemic with a different pathogen, Smittestop and similar DCT apps relying only on Bluetooth signals, appear, without any adjustments in the design, mainly to be able to contribute as screening tools. With the right privacy safeguards, decentralized digital contact tracing can be further adapted to address some of the main challenges mentioned in this paper. For example, increasing the functionality of Smittestop would increase incentive to download the app and adhere to use of it. This could be done by integrating it with other public-health interventions, e.g., allowing users to book a time for a PCR-test through the app, or share a positive result of self-performed LFT-samples on the app instead of having to go to a test center. Furthermore, this would enable a positive test to automatically generate an exposure notification to contacts, giving a more comprehensive notification cascade while also alleviating the administrative burden for users. A future, alternative design of DCT apps could rely not only on anonymized positive test shares and Bluetooth beacons, but include other sources of exposure information. This could be from manual contact tracing, location-specific QR-codes, or other relevant data sources such as GPS information, given that this could be done in a privacy preserving manner. Furthermore, individuals with exposure highest risk could be given quicker access to PCR-tests, or their samples be prioritized for analysis. Future use of DCT apps could provide users with the opportunity to give differentiated consent to access different data sources and to voluntarily share more of their personal data with public health authorities. Nevertheless, the success of DCT ultimately depends on its reliability. Without a well-functioning, accurate, and user-friendly tool, even the most ambitious integration strategies are unlikely to retain users and, consequently, will struggle to deliver meaningful public health impact.
Strengths and limitations
A key strength of this study was the high testing activity in the Danish population, and the ability to derive information from conventional contact tracing authorities, as well as the system for booking PCR-tests, that enabled assessment of the test indication. These proxy measures, however, hold limitations.
Assumptions on coverage and distribution of Smittestop can lead to misinterpretations. Some individuals will have had more than one phone while others e.g. young children or technologically non-proficient individuals, will have been unable to use Smittestop (85% of Danish citizens above the age of 15 have a smartphone35). Also, older smartphones were initially incapable of downloading Smittestop, although a later update remedied this. Therefore, the group of capable users was smaller than the entire population, and coverage therefore higher in this group. Furthermore, “pockets” of Smittestop users could have improved app efficacy within limited demographic groups and geographical areas, and these could be poorly representative of the Danish population36–39. In our calculations, we made assumptions about test-indication in the proportion of tested individuals who did not fill in the test-indication questionnaire, which is an oversimplification. Furthermore, some individuals may only have specified one test indication even if they had more than one reason.
Conclusion
We found that the number of cases identified by Smittestop was relatively low when put in relation to the amount of people who downloaded the app. However, these cases could have had important roles in infection transmission during the pandemic and would not have been found as effectively by mass screenings.
This also study illustrates the challenges in assessing the effectiveness of decentralized DCT apps, given the limited data that is collected from them. In future use of DCT, public health authorities should be able to determine if the overall cost of a DCT app and the trade-off of inconvenience (and privacy) for the individual user are proportional to the public health impact achieved. This could be done by integrating the app with other public health systems, and collecting more information on possible transmission events. Without further improvements in usability, accuracy and reputation, Smittestop remains screening tools to be used as a supplement to other measures during extreme public health events such as pandemics.
Acknowledgements
The study is part of the evaluation of the COVID-19 pandemic carried out at Statens Serum Institut (SSI) and was only possible due to massive efforts across many government institutions in Denmark. A special thanks goes to members of the working group on the Danish Smittestop application, including but not limited to: Nina Bergstedt, Birgitte Drewes and Marianne Voldstedlund.
Faculty Opinions recommendedReferences
- 1.
Organisation WH:
Contact tracing and quarantine in the context of the Omicron SARS-CoV-2 variant: interim guidance. 2022; Accessed July 25. Reference Source
- 2.
Mazza C, Girardi D, Gentile L, et al.:
Public health effectiveness of Digital Contact Tracing in the COVID-19 pandemic: a systematic review of available data.
Acta Biomed.
2021; 92(S6): e2021439. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 3.
Braithwaite I, Callender T, Bullock M, et al.:
Automated and partly automated contact tracing: a systematic review to inform the control of COVID-19.
Lancet Digit Health.
2020; 2(11): e607–e621. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 4.
Tupper PS, Otto CC:
Fundamental limitations of contact tracing for COVID-19. mathematics, genomics and prediction in infection and evolution - Simon Fraser University. 2021; Accessed July 25, 2022. Reference Source
- 5.
The Covid Tracing Tracker: what’s happening in coronavirus apps around the world. MIT Technology Review, Accessed May 17, 2022. Reference Source
- 6.
European Commission, Directorate-General for Communications Networks, Content and Technology, Prodan A, Birov S, et al.:
Digital contact tracing study: study on lessons learned, best practices and epidemiological impact of the common European approach on digital contact tracing to combat and exit the COVID-19 pandemic. Accessed January 24, 2023. Reference Source
- 7.
Chen W, Yao M, Dong L, et al.:
The application framework of big data technology during the COVID-19 pandemic in China.
Epidemiol Infect.
2022; 150: 1–11. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 8.
Elkhodr M, Mubin O, Iftikhar Z, et al.:
Technology, privacy, and user opinions of COVID-19 mobile apps for contact tracing: systematic search and content analysis.
J Med Internet Res.
2021; 23(2): e23467. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 9.
Lueks W, Benzler J, Bogdanov D, et al.:
Toward a common performance and effectiveness terminology for Digital Proximity Tracing applications.
Front Digit Health.
2021; 3: 677929. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 10.
Gasser U, Ienca M, Scheibner J, et al.:
Digital tools against COVID-19: taxonomy, ethical challenges, and navigation aid.
Lancet Digit Health.
2020; 2(8): e425–e434. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 11.
Wymant C, Ferretti L, Tsallis D, et al.:
The epidemiological impact of the NHS COVID-19 app.
Nature.
2021; 594(7863): 408–412. PubMed Abstract
| Publisher Full Text
- 12.
Marcel S, Althaus Christian L, Nanina A, et al.:
Early evidence of effectiveness of digital contact tracing for SARS-CoV-2 in Switzerland.
Swiss Med Wkly.
2020; 150(51): w20457. PubMed Abstract
| Publisher Full Text
- 13.
Vogt F, Haire B, Selvey L, et al.:
Effectiveness evaluation of digital contact tracing for COVID-19 in New South Wales, Australia.
Lancet Public Health.
2022; 7(3): e250–e258. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 14.
Rannikko J, Tamminen P, Hellsten R, et al.:
Effectiveness of COVID-19 digital proximity tracing app in Finland.
Clin Microbiol Infect.
2022; 28(6): 903–904. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 15.
Ballouz T, Menges D, Aschmann HE, et al.:
Individual-level evaluation of the exposure notification cascade in the Swisscovid Digital Proximity Tracing app: observational study.
JMIR Public Health Surveill.
2022; 8(5): e35653. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 16.
Ballouz T, Menges D, Aschmann HE, et al.:
Adherence and association of Digital Proximity Tracing app notifications with earlier time to quarantine: results from the Zurich SARS-CoV-2 cohort study.
Int J Public Health.
2021; 66: 1603992. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 17.
Ussai S, Pistis M, Missoni E, et al.:
“Immuni” and the National Health System: lessons learnt from the COVID-19 Digital Contact Tracing in Italy.
Int J Environ Res Public Health.
2022; 19(12): 7529. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 18.
Kendall M, Tsallis D, Wymant C, et al.:
Epidemiological impacts of the NHS COVID-19 app in England and Wales throughout its first year.
Nat Commun.
2023; 14(1): 858. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 19.
Pihlajamäki M, Wickström S, Puranen K, et al.:
Implementing and maintaining a SARS-CoV-2 Exposure Notification Application for mobile phones: the Finnish experience.
JMIR Public Health Surveill.
2023; 9(1): e46563. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 20.
Chambers T, Anglemyer A, Chen A, et al.:
Population and contact tracer uptake of New Zealand’s QR-code-based Digital Contact Tracing app for COVID-19.
Epidemiol Infect.
2024; 152: e66. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 21.
Haar WT, Bosdriesz J, Venekamp RP, et al.:
The epidemiological impact of digital and Manual Contact Tracing on the SARS-CoV-2 epidemic in the Netherlands: empirical evidence.
PLOS Digit Health.
2023; 2(12): e0000396. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 22.
Daniore P, Nittas V, Ballouz T, et al.:
Performance of the Swiss digital contact tracing app over various SARS-CoV-2 pandemic waves: repeated cross-sectional analyses.
JMIR Public Health Surveill.
2022; 8(11): e41004. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 23.
Statens Serum Institut:
Ekspertrapport Den 17. December 2021 Scenarier for Smittetal Og Nyindlæggelser Med Omikronvarianten. 2021; Accessed February 15, 2022. Reference Source
- 24.
Munch PK, Espenhain L, Hansen CH, et al.:
Case-control study of activities associated with SARS-CoV-2 infection in an adult unvaccinated population and overview of societal COVID-19 epidemic counter measures in Denmark.
PLoS One.
2022; 17(11): e0268849. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 25.
Gram MA, Steenhard N, Cohen AS, et al.:
Patterns of testing in the extensive Danish national SARS-CoV-2 test set-up. Osman M, ed.
PLoS One.
2023; 18(7): e0281972. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 26.
Statens Serum Institut:
Digital contact tracing smartphone application, smittestop, Denmark. Open Research Europe.
Zenodo.
2025. http://www.doi.org/10.5281/zenodo.16810558
- 27.
Authority DPS:
COVID-19: data og indsatser - Styrelsen for Patientsikkerhed. Accessed July 12, 2022. Reference Source
- 28.
Schønning K, Dessau RB, Jensen TG, et al.:
Electronic reporting of diagnostic laboratory test results from all healthcare sectors is a cornerstone of national preparedness and control of COVID-19 in Denmark.
APMIS.
2021; 129(7): 438–451. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 29.
Statistics Denmark. Accessed April 7, 2023. Reference Source
- 30.
ECDC/WHO:
Indicator framework to evaluate the public health effectiveness of digital proximity tracing solutions. 2021; Accessed January 4, 2022. Reference Source
- 31.
R Core Team:
R: A language and environment for statistical computing. 2022. Reference Source
- 32.
Kretzschmar ME, Rozhnova G, Bootsma MCJ, et al.:
Impact of delays on effectiveness of contact tracing strategies for COVID-19: a modelling study.
Lancet Public Health.
2020; 5(8): e452–e459. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 33.
Hinch R:
Effective configurations of a Digital Contact Tracing app: a report to NHSX. Christophe Fraser. 1:1. Accessed January 31, 2022. Reference Source
- 34.
Ferretti L, Wymant C, Kendall M, et al.:
Quantifying SARS-CoV-2 transmission suggests epidemic control with digital contact tracing.
Science.
2020; 368(6491): eabb6936. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 35.
Statistik D:
It-anvendelse i befolkningen 2022. Accessed January 7, 2022. Reference Source
- 36.
Guillon M:
Digital contact-tracing in France: uptake by COVID-19 risk factor and by exposure risk.
J Public Health (Oxf).
2022; 44(3): e366–e375. PubMed Abstract
| Publisher Full Text
- 37.
Osmanlliu E, Paquette J, Rodriguez Duarte MA, et al.:
Public perspectives on exposure notification apps: a patient and citizen co-designed study.
J Pers Med.
2022; 12(5): 729. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 38.
Dowthwaite L, Fischer J, Vallejos EP, et al.:
Public adoption of and trust in the NHS COVID-19 contact tracing app in the United Kingdom: quantitative online survey study.
J Med Internet Res.
2021; 23(9): e29085. PubMed Abstract
| Publisher Full Text
| Free Full Text
- 39.
Chen ATY, Thio KW:
Exploring the drivers and barriers to uptake for digital contact tracing.
Soc Sci Humanit Open.
2021; 4(1): 100212. PubMed Abstract
| Publisher Full Text
| Free Full Text
