In a previous blog post, I drew a technology-informed feasibility boundary to assess the realistic potential of digital technologies in the specific context of procurement governance. I suggested that the potential benefits from the adoption of digital technologies within that feasibility boundary had to be assessed against new governance risks and requirements for their mitigation.
In a new draft chapter (num 8) for my book project, I now explore the main governance risks and legal obligations arising from the adoption of digital technologies, which revolve around data governance, algorithmic transparency, technological dependency, technical debt, cybersecurity threats, the risks stemming from the long-term erosion of the skills base in the public sector, and difficult trade-offs due to the uncertainty surrounding immature and still changing technologies within an also evolving regulatory framework.
The analysis is not carried out in a vacuum, but in relation to the increasingly complex framework of EU digital law, including: the Open Data Directive; the Data Governance Act; the proposed Data Act; the NIS 2 Directive on cybersecurity measures, including its interaction with the Cybersecurity Act, and the proposed Directive on the resilience of critical entities and Cyber Resilience Act; as well as some aspects of the proposed EU AI Act.
This post provides a summary of my main findings, on which I will welcome any comments: a.sanchez-graells@bristol.ac.uk. The full draft chapter is free to download: A Sanchez-Graells, ‘Identifying Emerging Risks in Digital Procurement Governance’ to be included in A Sanchez-Graells, Digital Technologies and Public Procurement. Gatekeeping and experimentation in digital public governance (OUP, forthcoming). Available at SSRN: https://ssrn.com/abstract=4254931.
current and Imminent digital governance obligations for public buyers
Public buyers already shoulder, and will very soon face further digital governance obligations, even if they do not directly engage with digital technologies. These concern both data governance and cybersecurity obligations.
Data governance obligations
The Open Data Directive imposes an obligation to facilitate access to and re-use of procurement data for commercial or non-commercial purposes, and generates the starting position that data held by public buyers needs to be made accessible. Access is however excluded in relation to data subject to third-party rights, such as data protected by intellectual property rights (IPR), or data subject to commercial confidentiality (including business, professional, or company secrets). Moreover, in order to ensure compliance with the EU procurement rules, access should also be excluded to data subject to procurement-related confidentiality (Art 21 Dir 2014/24/EU), and data which disclosure should be withheld because the release of such information would impede law enforcement or would otherwise be contrary to the public interest … or might prejudice fair competition between economic operators (Art 55 Dir 2014/24/EU). Compliance with the Open Data Directive can thus not result in a system where all procurement data becomes accessible.
The Open Data Directive also falls short of requiring that access is facilitated through open data, as public buyers are under no active obligation to digitalise their information and can simply allow access to the information they hold ‘in any pre-existing format or language’. However, this will change with the entry into force of the rules on eForms (see here). eForms will require public buyers to hold (some) procurement information in digital format. This will trigger the obligation under the Open Data Directive to make that information available for re-use ‘by electronic means, in formats that are open, machine-readable, accessible, findable and re-usable, together with their metadata’. Moreover, procurement data that is not captured by the eForms but in other ways (eg within the relevant e-procurement platform) will also be subject to this regime and, where making that information available for re-use by electronic means involves no ‘disproportionate effort, going beyond a simple operation’, it is plausible that the obligation of publication by electronic means will extend to such data too. This will potentially significantly expand the scope of open procurement data obligations, but it will be important to ensure that it does not result in excessive disclosure of third-party data or competition-sensitive data.
Some public buyers may want to go further in facilitating (controlled) access to procurement data not susceptible of publication as open data. In that case, they will have to comply with the requirements of the Data Governance Act (and the Data Act, if adopted). In this case, they will need to ensure that, despite authorising access to the data, ‘the protected nature of data is preserved’. In the case of commercially confidential information, including trade secrets or content protected by IPR, this can require ensuring that the data has been ‘modified, aggregated or treated by any other method of disclosure control’. Where ‘anonymising’ information is not possible, access can only be given with permission of the third-party, and in compliance with the applicable IPR, if any. The Data Governance Act explicitly imposes liability on the public buyer if it breaches the duty not to disclose third-party data, and it also explicitly requires that data access complies with EU competition law.
This shows that public buyers have an inescapable data governance role that generates tensions in the design of open procurement data mechanisms. It is simply not possible to create a system that makes all procurement data open. Data governance requires the careful management of a system of multi-tiered access to different types of information at different times, by different stakeholders and under different conditions (as I already proposed a few years ago, see here). While the need to balance procurement transparency and the protection of data subject to the rights of others and competition-sensitive data is not a new governance challenge, the digital management of this information creates heightened risks to the extent that the implementation of data management solutions is tendentially open access. Moreover, the assessment of the potential competition impact of data disclosure can be a moving target. The risk of distortions of competition is heightened by the possibility that the availability of data allows for the deployment of technology-supported forms of collusive behaviour (as well as corrupt behaviour).
Cybersecurity obligations
Most public buyers will face increased cybersecurity obligations once the NIS 2 Directive enters into force. The core substantive obligation will be a mandate to ‘take appropriate and proportionate technical, operational and organisational measures to manage the risks posed to the security of network and information systems which those entities use for their operations or for the provision of their services, and to prevent or minimise the impact of incidents on recipients of their services and on other services’. This will require a detailed assessment of what is proportionate to the cybersecurity exposure of a public buyer.
In that analysis, the public buyer will be able to take into account ‘the state of the art and, where applicable, relevant European and international standards, as well as the cost of implementation’, and in ‘assessing the proportionality of those measures, due account shall be taken of the degree of the entity’s exposure to risks, its size, the likelihood of occurrence of incidents and their severity, including their societal and economic impact’.
Public buyers may not have the ability to carry out such an assessment with internal capabilities, which immediately creates a risk of outsourcing of the cybersecurity risk assessment, as well as other measures to comply with the related substantive obligations. This can generate further organisational dependency on outside capability, which can itself be a cybersecurity risk. As discussed below, imminent cybersecurity obligations heighten the need to close the current gaps in digital capability.
Increased governance obligations for public buyers ‘going digital’
Public buyers that are ‘going digital’ and experimenting with or deploying digital solutions face increased digital governance obligations. Given the proportionality of the cybersecurity requirements under the NIS 2 Directive (above), public buyers that use digital technologies can expect to face more stringent substantive obligations. Moreover, the adoption of digital solutions generates new or increased risks of technological dependency, of two main types. The first type refers to vendor lock-in and interoperability, and primarily concerns the increasing need to develop advanced strategies to manage IPR, algorithmic transparency, and technical debt—which could largely be side-stepped by an ‘open source by default’ approach. The second concerns the erosion of the skills base of the public buyer as technology replaces the current workforce, which generates intellectual debt and operational dependency.
Open Source by Default?
The problem of technological lock-in is well understood, even if generally inadequately or insufficiently managed. However, the deployment of Artificial Intelligence (AI), and Machine Learning (ML) in particular, raise the additional issue of managing algorithmic transparency in the context of technological dependency. This generates specific challenges in relation with the administration of public contracts and the obligation to create competition in their (re)tendering. Without access to the algorithm’s source code, it is nigh impossible to ensure a level playing field in the tender of related services, as well as in the re-tendering of the original contract for the specific ML or AI solution. This was recognised by the CJEU in a software procurement case (see here), which implies that, under EU law, public buyers are under an obligation to ensure that they have access and dissemination rights over the source code. This goes beyond emerging standards on algorithmic transparency, such as the UK’s, or what would be required if the EU AI Act was applicable, as reflected in the draft contract clauses for AI procurement. This creates a significant governance risk that requires explicit and careful consideration by public buyers, and which points at the need of embedding algorithmic transparency requirements as a pillar of technological governance related to the digitalisation of procurement.
Moreover, the development of digital technologies also creates a new wave of lock-in risks, as digital solutions are hardly off-the-shelf and can require a high level of customisation or co-creation between the technology provider and the public buyer. This creates the need for careful consideration of the governance of IPR allocation—with some of the guidance seeking to promote leaving IPR rights with the vendor needing careful reconsideration. A nuanced approach is required, as well as coordination with other legal regimes (eg State aid) where IPR is left with the contractor. Following some recent initiatives by the European Commission, an ‘open source by default’ approach would be suitable, as there can be high value derived from using and reusing common solutions, not only in terms of interoperability and a reduction of total development costs—but also in terms of enabling the emergence of communities of practice that can contribute to the ongoing improvement of the solutions on the basis of pooled resources, which can in turn mitigate some of the problems arising from limited access to digital skills.
Finally, it should be stressed that most of these technologies are still emergent or immature, which generates additional governance risks. The adoption of such emergent technologies generates technical debt. Technical debt is not solely a financial issue, but a structural barrier to digitalisation. Technical debt risks stress the importance of the adoption of the open source by default approach mentioned above, as open source can facilitate the progressive collective repayment of technical debt in relation to widely adopted solutions.
(Absolute) technological dependency
As mentioned, a second source of technological dependency concerns the erosion of the skills base of the public buyer as technology replaces the current workforce. This is different from dependence on a given technology (as above), and concerns dependence on any technological solution to carry out functions previously undertaken by human operators. This can generate two specific risks: intellectual debt and operational dependency.
In this context, intellectual debt refers to the loss of institutional knowledge and memory resulting from eg the participation in the development and deployment of the technological solutions by agents no longer involved with the technology (eg external providers). There can be many forms of intellectual debt risk, and some can be mitigated or excluded through eg detailed technical documentation. Other forms of intellectual debt risk, however, are more difficult to mitigate. For example, situations where reliance on a technological solution (eg robotic process automation, RPA) erases institutional knowledge of the reason why a specific process is carried out, as well as how that process is carried out (eg why a specific source of information is checked for the purposes of integrity screening and how that is done). Mitigating against this requires keeping additional capability and institutional knowledge (and memory) to be able to explain in full detail what specific function the technology is carrying out, why, how that is done, and how that would be done in the absence of the technology (if it could be done at all). To put it plainly, it requires keeping the ability to ‘do it by hand’—or at the very least to be able to explain how that would be done.
Where it would be impossible or unfeasible to carry out the digitised task without using technology, digitalisation creates absolute operational dependency. Mitigating against such operational dependency requires an assessment of ‘system critical’ technological deployments without which it is not possible to carry out the relevant procurement function and, most likely, to deploy measures to ensure system resilience (including redundancy if appropriate) and system integrity (eg in relation to cybersecurity, as above). It is however important to acknowledge that there will always be limits to ensuring system resilience and integrity, which should raise questions about the desirability of generating situations of absolute operational dependency. While this may be less relevant in the context of procurement governance than in other contexts, it can still be an important consideration to factor into decision-making as technological practice can fuel a bias towards (further) technological practice that can then help support unquestioned technological expansion. In other words, it will be important to consider what are the limits of absolute technological delegation.
The crucial need to boost in-house digital skills in the public sector
The importance of digital capabilities to manage technological governance risks emerges a as running theme. The specific governance risks identified in relation to data and systems integrity, including cybersecurity risks, as well as the need to engage in sophisticated management of data and IPR, show that skills shortages are problematic in the ongoing use and maintenance of digital solutions, as their implementation does not diminish, but rather expands the scope of technology-related governance challenges.
There is an added difficulty in the fact that the likelihood of materialisation of those data, systems integrity, and cybersecurity risks grows with reduced digital capabilities, as the organisation using digital solutions may be unable to identify and mitigate them. It is not only that the technology carries risks that are either known knowns or known unknowns (as above), but also that the organisation may experience them as unknown unknowns due to its limited digital capability. Limited digital skills compound those governance risks.
There is a further risk that digitalisation and the related increase in digital capability requirements can embed an element of (unacknowledged) organisational exposure that mirrors the potential benefits of the technologies. While technology adoption can augment the organisation’s capability (eg by reducing administrative burdens through automation), this also makes the entire organisation dependent on its (disproportionately small) digital capabilities. This makes the organisation particularly vulnerable to the loss of limited capabilities. From a governance perspective, this places sustainable access to digital skills as a crucial element of the critical vulnerabilities and resilience assessment that should accompany all decisions to deploy a digital technology solution.
A plausible approach would be to seek to mitigate the risk of insufficient access to in-house skills through eg the creation of additional, standby or redundant contracted capability, but this would come with its own costs and governance challenges. Moreover, the added complication is that the digital skills gap that exposes the organisation to these risks in the first place, can also fuel a dynamic of further reliance on outside capabilities (from consultancy firms) beyond the development and adoption of those digital solutions. This has the potential to exacerbate the long-term erosion of the skills base in the public sector. Digitalisation heightens the need for the public sector to build up its expertise and skills, as the only way of slowing down or reducing the widening digital skills gap and ensuring organisational resilience and a sustainable digital transition.
Conclusion
Public buyers already face significant digital governance obligations, and those and the underlying risks can only increase (potentially, very significantly) with further progress in the path of procurement digitalisation. Ultimately, to ensure adequate digital procurement governance, it is not only necessary to take a realistic look at the potential of the technology and the required enabling factors (see here), but also to embed a comprehensive mechanism of risk assessment in the process of technological adoption, which requires enhanced public sector digital capabilities, as stressed here. Such an approach can mitigate against the policy irresistibility that surrounds these technologies (see here) and contribute to a gradual and sustainable process of procurement digitalisation. The ways in which such risk assessment should be carried out require further exploration, including consideration of whether to subject the adoption of digital technologies for procurement governance to external checks (see here). This will be the object of forthcoming analysis.