Plain language summary
EU data protection rules could be difficult for researchers to apply, particularly when processing (using) massive datasets containing personal data for Artificial Intelligence (AI) developments. This article examines how data protection intersects with AI research. The key objectives of this work are to (1) look at the data protection requirements and limits for the processing of personal data in AI research, (2) reflect on possible complications regarding data quality requirements for trustworthy AI, and (3) present possible ways forward.
Introduction
In recent years, aware of the importance of not lagging behind in the global competition on Artificial Intelligence (AI), EU leaders have made investing in AI a top priority,1 with AI research and innovation initiatives playing a pivotal role in the EU policy efforts.2 The European approach for AI focuses on supporting ethical and trustworthy AI,3 making a significant portion of the available resources to foster the development and uptake of AI through funding programmes such as Horizon 2020 and Horizon Europe. The framework of these programmes make it that, for EU researchers, ‘ethics is an integral part of [their] research from beginning to end’,4 with project beneficiaries also being strongly encouraged to contribute to the trustworthy AI efforts.
By nature, AI research and development efforts, particularly those involving Machine Learning (ML) techniques, require significant amounts of training, testing and validation data.5 The quality of the data also matters for the downstream effects, since the characteristics of the data will translate into outputs of the system at deployments. The higher the quality of the data used, the better the system outputs (at least in theory). Hence, the data needed in AI research are also associated with the ‘trustworthy’ nature of EU-made AI systems, i.e. systems that respect human rights, adhere to ethical guidelines and promote human agency.6 It is therefore no surprise that the 2021 proposal for an EU AI regulation7 includes data quality criteria for certain systems.
However, the status of AI research in the General Data Protection Regulation (GDPR)8 and existing data protection guidance is not clear. AI researchers mostly rely on large-scale databases containing massive volumes of (personal) data to ‘instruct’ their systems and ensure performance, and some of the personal data therein may even fall under special categories of data, particularly when engaging in debiasing activities.9 Yet, the efforts to fulfil data quantity and quality requirements for AI research purposes may be at odds with the attempts to comply with certain data protection conditions, tensions that the current legal regime for research does not fully address, as shown in this article. Therefore, the shortcomings regarding the access to, (re)use and sharing of data by genuine researchers must be tackled.
Recent legal scholarship addresses questions about data protection in research10 and the legal issues associated with AI,11 while little attention has been paid to AI research specifically. This paper sheds light on the under-discussed issue of challenges and tensions between the processing of large-scale datasets for AI research and compliance with the GDPR. What data is needed to embed values and achieve ‘trustworthiness’ in EU-made AI systems? Is that processing allowed under EU data protection law? How to reconcile data protection requirements with legitimate data processing for genuine research (collection, storage, sharing and (re)use of massive volumes of data, including sensitive data)? Does the GDPR threaten to hinder the progress in AI research? These are ongoing challenges explored in this paper.
The analysis is structured as follows. The section ‘AI training, testing and validation data’ refers to the importance of data quantity and quality for AI development purposes, and discusses the data quality criteria in the proposed EU AI Act. Following this, ‘Legal and ethical challenges and complications of personal data use in AI research’ examines how the GDPR applies to AI research, referring to the allowances, limits and requirements for data processing in research, and explores key issues regarding the incorporation of massive amounts of (sometimes sensitive) personal data in AI developments. Considering the findings of this analysis, ‘Ways forward’ suggests some measures to tackle unaddressed challenges and to find the proper balance between privacy and innovation. The final section contains the concluding remarks.
AI training, testing and validation data
Quantity and quality of data are crucial factors for the research and development of AI. In this section, we refer to the importance of quality of data for AI research given the impact on system outcomes and downstream effects at deployments. The significance of this matter is illustrated by the inclusion of quality criteria in the proposal for an EU AI Act presented by the European Commission in April 2021, also referred to in the analysis.
The importance of data quality for AI
AI/ML training, testing and validation require large volumes of data, which also needs to meet certain quality criteria to ensure model performance. This is evidenced in current literature on the quality and representation issues with training, testing and validation data, especially those in publicly available datasets and databases.12 The problem is that AI systems built on insufficient or flawed data are prone to have significant downstream effects. In particular, ML systems that are trained on limited datasets, which only represent some part of the society, will reproduce and even exacerbate biased and partial perspectives.13 This refers to the ‘garbage in, garbage out’ principle, a phenomenon that can result in flawed outcomes and decisions based in incorrect conclusions, and eventually even lead to human rights violations if left unaddressed.14 There is also growing interest in ways to detect and mitigate data quality risks in AI research. For instance, a study on facial recognition technology showed that access to more personal data about people from different origins, races, and ethnicities may increase accuracy of the system, decrease systemic bias, and improve the ability for developers to prove this to regulators.15
Data quality in AI and ML is not only a recurrent subject in research but also forms part of AI policy discussions and documents. For instance, the European Council has acknowledged that ‘high-quality data are essential for the development of [AI]’.16 Also, the Ethics Guidelines for Trustworthy AI17 specifically refer to data governance as one of the requirements for AI systems, highlighting the impact of biases in training datasets. Similarly, the White Paper on AI by the European Commission essentially considers the availability of data as a ‘key driver’ for AI initiatives, enabling the development and improvement of products and services in support of better decision-making.18 It can therefore be said that data quantity and quality play a crucial role in the attempts to develop trustworthy and unbiased AI systems.19
Understanding data quality issues can help address potential problems, including the risks of bias and discrimination. Against this backdrop, the EU AI Act proposal establishes specific quality criteria for training, validation and testing data, further explored in the following.
Quality criteria in the EU AI Act proposal
On 21 April 2021, the European Commission presented a proposal for an AI Act (also known as the AI Regulation), laying down harmonised rules on AI.20 This heavily anticipated document represents a major regulatory development of our time since it is the first ever attempt worldwide to regulate in that area. The AI Act aims to foster a coherent and harmonised digital single market, in a way that AI systems or products powered by AI technology can circulate in the EU market according to common rules, ensuring the safeguard of European fundamental values.
The proposal establishes requirements and obligations for the development, placement on the market and use of AI systems. The initial proposal by the European Commission is framed following a risk-based approach, laying down specific obligations that apply to different categories of AI systems based on their characteristics and risks. The draft regulation classifies AI systems into four categories based on whether they pose: (i) unacceptable risks, entailing the outright or qualified prohibition of such systems; (ii) high risks, which are subject to ex-ante and ex-post requirements, as well as a supervision and enforcement regime; (iii) limited risks, subjects to specific transparency obligations; or (iv) minimal risks, which are free to circulate in the EU market.21 As becomes apparent, the vast majority of requirements concern high risk systems, including data quality requirements.
Before diving into the data quality criteria in the EU AI Act proposal, it is first necessary to assess to what extent the proposed regulation would apply to research. The initial proposal by the European Commission appears to indicate that scientific research would not fall under the scope of that law given the non-commercial nature of these activities, thus not belonging to the market context. The so-called ‘research exemption’ was however only tackled in a recital22 and not in the main text, thus arguably only addressing the corresponding provisions23 leaving the open question of whether, a contrario, research generally does fall within the scope of the proposed regulation.24
In the presidency compromise text,25 the Council in turn introduced a broad exception for ‘AI systems, including their output, specifically developed and put into service for the sole purpose of scientific research and development’.26 The ‘sole purpose’ requirement needs to be distinguished from a situation where a research activity, by e.g. a company’s research and development department, leads to or entails placing an AI system on the market or putting it into service.27 In such a case the AI Act would apply. If adopted, this research exemption would mean that AI researchers will not be subject to the requirements in the proposed legislation. It does not mean, however, that they do not have to foresee the impact or risks for fundamental rights and freedoms of the AI systems they develop as researchers still need to comply with relevant rules, including the conditions and requirements to engage in research. For instance, in the context of EU-funded initiatives, projects involving the development, deployment and/or use of AI are required, from the outset, to assess and provide minimum information on the potential ethical risks regarding AI developments and the foreseen risk mitigation measures to be taken throughout the action implementation.28
Still, one may wonder whether the Council’s choice could lead to undesirable effects given the other factors that come into play in a scientific research context, particularly the ethical obligation to make the results open and publicly accessible. Given their dissemination duties, AI researchers have to ensure that project outcomes are widely available, which could make research prone to be used by other actors and for other purposes than initially envisaged, e.g., exploited by private entities to pursue commercial interests. This could result in the research exemption leading to a heightened risk of misuse of research findings, and thus calls for a need to examine measures to avoid possible side effects. Something to consider is to strengthen the rules on potential misuse of research findings, or go for an alternative approach consisting of facilitating access to sandboxes in which the rules could be suspended for research reasons, as commentators have suggested.29
When it comes to AI datasets, the EU AI Act proposal sets out quality criteria for the data used to train, validate and test high-risk AI systems. Article 10 specifies that the data used shall be relevant, representative, free of errors and complete, and with application-area specific properties.30 To ensure representativeness of the data and to prevent risks of bias, the proposal foresees the processing of special categories of data, provided that adequate safeguards for fundamental rights are in place.
Despite establishing a commendable requirement, the draft regulation does not, however, provide any guidance as to how the data quality criteria should be accomplished considering the GDPR requirements for the processing of personal data. Also, the proposed Act does not specify what is representative data,31 neither does it mention data integrity, precision or recall, which are considered a more appropriate measures to data classification.32 At the same time, the draft regulation requires training, validation, and testing datasets to be ‘free of errors and complete’, which is considered to be technically impossible.33 While it specifies that these requirements are to be met ‘sufficiently’ and ‘in view of the intended purpose of the system’,34 it is however difficult to determine what constitutes ‘sufficient’ statistical properties35 or ‘adequate’ requirements in terms of accuracy or robustness.36 Such a prior assessment appears particularly difficult in the case of third-party or publicly available datasets. Moreover, the proposed Act does not specify how the data quality of such datasets should be checked, assessed, and mitigated.37
Legal and ethical challenges and complications of personal data use in AI research
Scientific research involving the processing of personal data of EU persons is subject to a specific regime under EU data protection law allowing for certain flexibility of the data protection requirements. Specifically, the GDPR puts forward the research regime for processing personal data, composed of specific derogations from certain obligations, as well as the requirement of appropriate safeguards. This regime applies insofar as the research takes place within an ethical framework and is carried out with the broader goal of contributing to the development of knowledge and wellbeing in society. While it does help avoid some issues when processing personal data for research purposes, important challenges remain as it does not fully accommodate the demands of AI research. This section examines how the GDPR intersects with trustworthy AI research practices and explores challenges in this context.
Compliance with data protection principles: a largely resolved issue
It would be fair to start this analysis by acknowledging the positive effects of the special GDPR regime for research. While it may initially seem challenging to comply with data protection provisions in a research context, the favourable data processing conditions for research in the GDPR seem to address relevant issues (or at least a great part of them). A clear example of this is compliance with data protection principles.
At first, it may seem hard to comply with data protection principles when processing large scale datasets for AI research. However, scientific research might be less affected by possible issues since those are largely addressed by the special data protection regime for research. Specifically, the GDPR exceptions to some data protection principles, such as purpose limitation and data minimisation, insofar as appropriate technical and organisational safeguards for the rights and freedoms of data subjects are provided,38 help address this challenge.
To illustrate how the special GDPR regime helps address some issues, one can consider the example of purpose limitation. Under the purpose limitation principle, personal data shall not be further processed for purposes incompatible with the original purposes of collection.39 In AI research, the data used are typically obtained from publicly available databases, which contain personal information likely to have been collected for different purposes.40 Given the so-called presumption of compatibility between (secondary) processing for research purposes and the original purpose of collection, further processing for scientific research purposes does not raise purpose limitation issues. It should be noted, however, that this presumption is not a general authorisation to further process data in all cases for research.41
Lack of a definition of what is research
Despite containing a special regime for research, the GDPR does not specify what is to be understood by ‘research’ or ‘scientific research’, which are concepts with no generally agreed-upon definition. Some clarification may be found in recital 159 of the GDPR, which hints that the processing of personal data for scientific research purposes should be interpreted broadly, including for example technological development and demonstration, fundamental research, applied research and privately funded research as well as studies conducted in the public’s interest in the area of public health.
In its Preliminary Opinion on data protection and scientific research, the European Data Protection Supervisor (EDPS)42 considers that academic researchers, as well as non-profit organisations, government institutions and profit-seeking commercial companies can engage in scientific research. According to the EDPS opinion, scientific research ‘applies the “scientific method” of observing phenomena, formulating and testing a hypothesis for those phenomena, and concluding as to the validity of the hypothesis’.43
After a case law analysis on this issue, a legal scholar concluded that there are three potential criteria to define what constitutes scientific research, namely: (1) the role of the legal entity; (2) the role of the person carrying out the activity; and (3) quality standards, including scientific methodology and scientific publication.44 EU-funded AI research and innovation initiatives under EU-funded programmes such as Horizon 2020 and Horizon Europe are likely to meet those criteria. When it comes to commercial research on AI some argue that it should not benefit from the GDPR research exemption except when there is a public interest and similar safeguards to those that apply in a scientific research context.45 In practice, as demonstrated by the Cambridge Analytica scandal, the relations between private companies and academic researchers can become entangled.46 This case showed how when too much connected, scientific research and commercial activity can lead to unethical results. It also showed that the line between scientific research and commercial activity is not always easy to trace.47
Challenge to comply with legal basis
AI researchers typically rely on existing datasets as training data.48 While not all the data used is personal data (and thus not covered by data protection law), often these datasets may contain personal data, which may at times fall under special categories of data, particularly during efforts for bias detection and mitigation in AI. Research shows that in order to make non-discriminatory decision models, for instance, with respect to race, the sensitive information about race needs to be used in the model building process.49
An example of this is attempting to have a more diverse dataset which accurately represents the relevant population, including sufficient data of historically under sampled groups of individuals, an activity that may require sensitive and biometric information.50 Despite the presumption of compatibility that applies in a research context (section compliance with data protection principles: a largely resolved issue), purpose specification and lawfulness are two separate and cumulative requirements. As a consequence, the initial collection of data, re-used for scientific research purposes, even with the presumption of compatibility, would still require a legal basis.
When personal data are further processed for compatible purposes, such as scientific research, ‘no legal basis separate from that which allowed the collection of the personal data is required’.51 The question arises as to what extent is it the responsibility of the re-user (in this case, the AI researcher) to investigate the legal basis for the initial data processing. When it comes to publicly available datasets, it might prove difficult if not impossible to find out whether the initial collection of data was based on a valid legal basis meeting the conditions under the GDPR. Should that be the case, the researcher would nonetheless have to find and rely on an applicable legal basis.
Further, regarding special categories of data specifically, which are subject to stricter conditions under the GDPR, those data require an extra layer of protection and their processing is in principle prohibited, unless one of the ten specific exceptions contained in the GDPR is met.52 In the context of scientific research, there are three legal bases that are most likely to apply, namely: the consent of the data subject,53 the performance of a task carried out in the public interest54 or a legitimate interest of the controller or a third party55. Yet, there are some issues with those legal grounds.
Regarding the first option (consent), it is not likely to provide a sufficient legal basis for various reasons. In case of large-scale datasets with millions of personal data points, some of which may be sensitive data, consent is almost impossible to realise. It is also problematic to contact data subjects to ask for consent in the first place. Relying on a derived consent from the consent that data subjects gave, for example, in the terms and conditions of a social media platform on which they initially uploaded their personal data, would hardly meet a threshold of ‘real’ genuine informed consent.
The second legal basis could be the performance of a task carried out in the public interest.56 There is however lack of clear guidance how to interpret the ‘public interest’ in a research context. What constitutes a ‘public interest’ is a matter of national legislation and only a few Member States (e.g., Finland) have considered scientific research as such.57 In the case of special categories of data, data processing must moreover be ‘necessary for reasons of substantial public interest’.58 Again, it is a matter for Member State law to substantiate this provision.
A third legal basis that would arguably be applicable for training AI models with personal data is legitimate interest.59 Although this may hold true for the training operation itself, depending on the degree of anonymisation, the legitimate interest must be assessed also at the very first stage of data processing, namely the collection of data to be fed into the ML pipeline. As provided by the Article 29 Working Party,60 the ‘interest of carrying out scientific research’, subject to appropriate safeguards, is one of the ‘interests [that] may be compelling and beneficial to society at large’.61 This legal basis requires a case-by-case assessment and a balancing test between the interests of researchers (and society as a whole) and data subjects’ rights. The GDPR seems to weigh decisively in favour of third parties’ interest when it states that ‘for scientific or historical research purposes or statistical purposes, the legitimate expectations of society for an increase of knowledge should be taken into consideration’.62 However, an important hurdle concerns that the basis of legitimate interests is only available to private entities, not public authorities (e.g., public universities) ‘in the performance of their tasks’. This is not to say that public authorities, or those having a hybrid status, depending on the task they perform are a priori excluded from the scope of the provision. What matters it the national laws’ interpretation of what constitutes ‘the performance of their tasks’.
In terms of establishing the second legal basis required when processing sensitive data, a way forward would be for researchers to rely on the possibility to process such data when ‘manifestly made public by the data subject’63. As the EDPS has said, ‘this provision has to be interpreted to imply that the data subject was aware that the respective data will be publicly available which means to everyone’64 including, in this case, researchers.65 In this line, the EDPB confirmed that ‘the word “manifestly” implies that there must be a high threshold for relying on this exemption’.66 A combination of the elements such as whether the data subject took a specific action to change the private settings into public ones, the nature of the social media platform, the fact whether the data subject has published the sensitive data himself/herself, or whether the data has been published by a third party, need to be considered for controllers to demonstrate that the data subject has clearly manifested the intention to make the data public.67 A case-by-case assessment is always needed. It should be noted, however, that the practice of ‘web scraping’ is generally not permitted as a method to access data and is often explicitly prohibited by platforms’ terms of service.
Difficulties to comply with data subject rights
Another challenge for re-using personal data in AI research relates to data subject rights as it may prove hard for AI researchers to comply with data subject requests. The major challenge is how to comply with the right to information.68 The GDPR provides that, where personal data have not been obtained from the data subject, as it is the case for processing data in publicly available datasets, the controller must provide the data subject with information ‘within a reasonable period after obtaining the personal data, but at the latest within one month’.69 But there is an exception to this rule, which stipulates that controllers do not have to provide information to the data subject where appropriate safeguards are in place and when the provision of information proves impossible, requires a disproportionate effort or risks seriously compromising the achievement of the research.70
This was one of the matters considered in a Belgian case regarding the further processing of personal data extracted from social media for a research study on the fight against disinformation.71 The Belgian Data Protection Authority (DPA) found that the non-profit organisation that carried out the study was dispensed from the obligation to provide prior information to the persons concerned in the context of journalistic activities, to the extent that informing them could have compromised the performance of the study and its subsequent publication. In its decision, the Belgian DPA highlighted the fact that the scientific exception requires more transparency, in particular on the methodology for collecting data and their degree of anonymisation and pseudonymisation. Moreover, to benefit from the research exemption, appropriate safeguards for the rights and the persons concerned should be adopted, as required by the special regime in the GDPR. In this case it would have included a registrar, external privacy policy on the methodology and confidentiality measures. Therefore, notwithstanding the moral legitimacy of the purposes pursued by the researchers, the DPA concluded that releasing raw and un-pseudonymised personal data containing sensitive personal data is a disproportionate way to demonstrate the research methodology.72
Another potential challenge for fulfilling individuals’ rights is the difficulty involved in identifying the individuals concerned by each dataset. Indeed, singling out data belonging to a specific individual is not always easy. The fact that the data subject can exercise his or her rights at different stages of the lifecycle of an AI system processing personal data, namely when gathering and preparing data, when training or testing the model, or during deployments, makes it even more challenging.
It can be argued that given the large scale of training data, identifying the individuals whose data belongs to, would indeed ‘prove impossible’. Should this be the case, the controller should, however, ‘take appropriate measures to protect the data subject's rights and freedoms and legitimate interests, including making the information publicly available’.73 The GDPR does not provide further guidance on how to realise this obligation in practice. One solution would be for AI researchers to be transparent about the datasets they use. However, this raises other questions, in particular on whether and how that publicly available information could help data subject. Provided data subjects would indeed receive information about their data being used for AI research, how could they exercise their other rights?
As far as the rights to restriction and objection are concerned, in certain circumstances, a large number of individuals objecting to all or part of a scientific research may have a negative effect on the representativeness and reliability of the research data, and thus on the integrity of research.74 For instance, personal data could be retained when the AI system would need re-training, quality search or evaluation processes. To this end, the GDPR75 contemplates the possibility for researchers to ignore an erasure request where it would render impossible or seriously impair the processing of personal data for scientific research purposes. However, a balance must be struck between the research objectives and the rights of data subjects.
Impact on the quality of the resulting AI system
Ethical and legal issues are integral to EU-funded research, with researchers having to comply with ethical principles and relevant legislation being part of the funding schemes.76 Accordingly, EU-funded research strives to achieve high ethical and legal commitments from inception to end, with AI projects allocating part of their budgets to addressing ethical, legal and societal issues.77 One of the main ethical issues in AI research is algorithmic bias.78 While not everything is valid in fighting bias risks, large amounts of data can be of help here. Yet, data protection rules may have an impact on the quality of the developed systems. In particular, the GDPR may interfere with, and could potentially impair, the possibilities for AI researchers to collect or be able to use the data necessary for detecting and correcting bias in AI systems, or even to develop a system that complies with AI training data criteria more broadly. If access to relevant data and features is restricted, it is much more difficult to detect bias in the model, and even harder (in some cases, impossible) to fix it without access to the relevant data and features.
Furthermore, some research may even require access to data considered more sensitive given the need for data that reflect the problem the envisaged system aims to address and the real-world conditions where it is supposed to operate. For instance, security research requires data imitating the operational environment in question.79 It is unfeasible to reproduce the high-stakes situations, such as cybercrime events, without access to relevant and representative data. Although currently available datasets capture some of the real-world conditions of the field, the ability to reproduce realistic environments and the genuine effects concerned by security situations and forensic practices might require access to criminal offense data, which are subject to stricter conditions under the GDPR.80
In this context, a number of complex legal, ethical and technical pre-requisites may make it difficult for the relevant community (including researchers, practitioners, industry, and policy makers) to have access to real, diverse and high-quality datasets, or alternatively to create representative datasets to engage in scientific research. Continuing with the same example, in security research the challenges include confidentiality constraints, data protection restrictions, and legal uncertainties regarding the legal basis to share operational data for research purposes.81 Hence, it has become very complex for some research projects, particularly in more sensitive domains, such as security, health or defence, to develop reliable AI tools due to the absence of legislation addressing these issues.82
The AI Act proposal seems to recognise these types of hindrances. The draft regulation provides that, in order to protect the right of others from the discrimination that might result from the bias in AI systems, developers should be able to process special categories of personal data, as a matter of substantial public interest, in order to ensure the bias monitoring, detection and correction in relation to high-risk AI.83 The processing of special categories of data is governed by the GDPR, which does not contain such an exemption for bias detection. The draft AI regulation seems to attempt to fill this gap by allowing the processing of special categories of personal data for bias mitigation, subject to appropriate safeguards for the fundamental rights and freedoms of natural persons.84 However, the exemption can only be used in relation to high-risk systems, leaving, for no evident reason, non-high-risk developers unable to rely on it.85 Another question is whether the AI Act proposal provides a new legal ground for processing special categories of sensitive personal data. Should that be the case, the proposal could interfere with the application of data protection rules, including the GDPR.86 Overall, there are various arguments in favour of and against adopting an exception in the AI Act that would enable the use of special categories of data to prevent AI-driven discrimination87 and there is a need to find a proper balance.
Confronted with possible adverse consequences
Due to the uncertainties associated with the use of large-scale datasets as training, testing and validation data, and lack of guidance on certain aspects of the legal regime for research, AI projects and researchers themselves face certain risks in their balancing of the different interests at stake during their activities. The risks include possible GDPR infringement and subsequent breach of contractual obligations, potential reputational damage, difficulties to publish their research results in prestigious journals and conferences, and eventually even threaten to hamper societal trust in scientific research.
First, if the activities undertaken during the AI developments were to be considered unlawful processing of personal data, that could trigger the GDPR sanction regime,88 leading to a significant liability (e.g., fines of up to €20 million or 4% of annual revenue). In the context of EU-funded research, beyond the potential penalties, a GDPR infringement will also entail a breach of the contractual obligations with the funding agency, meaning that project participants may face other sanctions contemplated in the contract.
Second, the use of datasets eventually flagged as ‘problematic’ as training data raises ethical questions about the legitimisation of such research and about possible strategies to mitigate such risks, with likely reputational consequences for the project and parties involved. This is illustrated by the controversies associated with publicly available corpora, such as the MegaFace database and the Casia WebFace.89 While initially widely used as benchmark datasets in AI research, these and other datasets were eventually flagged by researchers, activists and journalists due to the many concerns raised by the data and its use, particularly in terms of privacy and human rights.90 The controversies ultimately also affected the reputation of projects where such data were used, as well as that of the centres and researchers involved in such projects.
Third, the difficulties to find and be able to use relevant data can also affect the possibilities for researchers to produce meaningful findings to disseminate in relevant fora, particularly high-ranking journals. In this context, both publishers and academics are confronted with difficulties. Scientific journals aim for the timely publication of correct results, at the right time and to the right audience, with the review of ethical aspects (including data ethics for AI research) being one of the success factors, given the implications of the data quality.91 At the same time, researchers have to cope with the difficulties associated with the research data, including the varied terms of use, diverse legal frameworks and unclear provisions applicable to their activities.92 Therefore, the challenges regarding data for AI research could hamper the possibilities for researchers to publish their work in scientific fora to gain recognition for their efforts, earn academic advancement and maximise the impact of their endeavours.
Finally, a broader aspect to consider is the societal trust in researchers and scientists to produce knowledge and make relevant discoveries that help solve problems. Trust is essential in scientific research. Yet, concerns and controversies associated with scientific research, such as scandals associated with the use of the data available for AI research, can hinder public trust in science. Mistrust and rejection of scientific findings could in turn significantly reduce the return on research and ultimately entail a major waste of public money.
Ways forward
Navigating between the need for large amounts of data, data quality requirements and GDPR compliance is a challenge in AI research. Researchers often face questions about the quality, legality, and ethics of the openly accessible datasets that they use because those are the ones available. There are possible ways to address these issues and find a balance between the different interests at stake, particularly privacy and innovation.
More availability of high-quality datasets
The EU AI Act proposal provides that for the development of high-risk AI systems, certain relevant entities, such as digital innovation hubs, testing experimentation facilities and researchers, ‘should be able to access and use high-quality datasets within their respective fields of activities’ related to the regulation.93 In this context, it is no coincidence that the EU AI policy is accompanied by the European data strategy,94 which amongst others aims to address the challenge of lack of available data necessary to achieve EU’s digital transformation. In particular, European common data spaces established by the Commission and the facilitation of Business to Business and Business to Government data sharing in the public interest are expected ‘to provide trustful, accountable and non-discriminatory access to high-quality data for the training, validation and testing of AI systems’.95 As an example, a European health data space will facilitate access to health data and the training of AI algorithms on those datasets.96
The EU legislator seems to assume that data-related initiatives will provide a sufficient framework for researchers’ access to high-quality datasets, an aspect that the AI Act proposal does not address. In particular, a horizontal governance framework for data access and use should ‘facilitate decisions on which data can be used how and by whom for scientific research purposes in a manner compliant with the GDPR’.97
Data altruism, as contemplated in the recently adopted Data Governance Act (DGA)98 aims to make sharing of personal data on the basis of the consent of data subjects possible ‘for objectives of general interest’.99 However, the DGA does not specify what is considered ‘an objective of general interest’. It hints in the recital that the support to scientific research should also considered to be an objective of general interest.100 Yet, it is left to national laws to determine whether scientific research will fall under this definition. A wide binding data access framework for AI research would be more than welcome to enable researchers to access data in a harmonised and legally-compliant way.
Setting up data spaces that can be used in research
According to the Data Strategy, next to the horizontal data governance framework, the Commission intends to fund the establishment of EU-wide common, interoperable data spaces in strategic sectors.101 For instance, the difficulties regarding data availability in security research is one of the lessons learned from previous research projects and intended to be addressed in its the Horizon Europe framework by fostering the creation of a European common training and testing data repository for this field of research.102
Although such data spaces can be used for multiple purposes, the EDPS points out that it should be clearly defined what are the permitted purposes of its creation (i.e., research and non-research). The limitations on the cross-context use of data, including prohibitions on the use of sensitive personal data for other purposes remain applicable to personal data in a data space.103 The data gleaned in these data spaces could, however, benefit the research community.
Avoiding problematic datasets from proliferating and being used in research
The efforts to foster more availability of data should not be done in a vacuum, but as part of larger initiatives to tackle different issues, including preventing and restricting the use of problematic datasets. This is important because, despite the scandals associated with widely used datasets (e.g., MegaFace, which was retracted in 2019 due to problematic practices outlined above), AI researchers have continued using those. The original datasets, and derivatives thereof, have been cited in several publications released months after retraction of the data.104
This calls for measures to avoid troublesome datasets from continuing to be relied upon in AI research. In this context, it is important that the efforts towards improving access to relevant and sufficient data for AI research endeavours are accompanied by initiatives to prevent problematic data from continuing to proliferate and being used. To address this, a recent study105 calls for, amongst others, making ethically salient information about datasets clear and accessible, active stewardship of the data and its uses, employing ethics review procedures to promote responsible data uses, and the advance review of datasets and publications.
Developing standards or codes of conduct
Another path that could be taken in providing more clarity on how to access and use datasets in a GDPR-compliant way would be standards or codes of conduct. The GDPR106 specifically encourages the drawing up of codes of conduct intended to contribute to a proper GDPR application. According to the EDPB, codes of conduct should specify how to apply the GDPR in a ‘specific, practical and precise manner’, provide ‘unambiguous, concrete, attainable and enforceable (testable)’ standards and rules.107
In this context, codes of conduct could be developed to resolve the tensions around accessing data for AI research in the GDPR-compliant way. This approach is being explored by the European Digital Media Observatory (EDMO), which in May 2022 released its report on researchers access to platform data together with a draft Code of Conduct on how platforms can share data with independent researchers while protecting users’ rights.108 This work provides the groundwork for a Code of Conduct on Platform-to-Researcher Data Access under Article 40 of the GDPR.
Conclusions
This article has examined the interactions between AI research and the GDPR, particularly when it comes to the use of large-scale databases containing personal data to train, test and validate AI systems. This analysis identifies data protection challenges posed by the use of the data required for AI research, particularly when engaging in trustworthy AI efforts. The findings suggest that the GDPR regime for research provides for (at least partial) solutions and interpretation keys to resolve relevant complications. However, this is unfortunately not the case for all aspects of AI research. In particular, AI researchers face hurdles to meet the quantity and quality criteria for developing trustworthy AI systems, while at the same time attempting to comply with data protection requirements. This paper elucidates these issues and proposes some ways to address remaining challenges.
Beyond the GDPR, this paper discusses the EU AI Act proposal. It is not yet clear whether the proposed regulation will come into force, and if its requirements (such as the data quality criteria) will apply to AI research, as has become apparent in this study. In any case, it is important to ensure a balanced approach in the final version of the text. For instance, the research exemption that appears in the latest proposal draft at the time of writing (i.e. the presidency compromise text) may ultimately become a sort of mixed blessing. On the one hand, AI research would not be subject to stringent data quality requirements applicable to commercial, high-risk applications – which would prove difficult for researchers in any case. At the same time, researchers will not be able to benefit from the AI Act affordances such as the new possibility to process special categories of personal data for bias mitigation that actors that do fall under the scope may benefit from. As a result, the EU AI Act may ultimately not change the status quo of data (un)availability for scientific research, an outcome that the AI research community would hope for and welcome.
Hopefully, these and other issues identified in this research will be addressed soon, in a way that EU law is well equipped to capture these situations, finding the right balance between enabling to reap the benefits of AI research while mitigating privacy and other fundamental rights concerns. Research is a core part of the EU digital strategy, thus more clarity on the requirements and greater support for researchers would be welcome.
Ethics and consent
Ethical approval and consent were not required
