** This post is based on the seminar given at the Law Department of Pompeu Fabra University in Barcelona, on 7 November 2019. The slides for the seminar are available here. Please note that some of the issues have been rearranged. I am thankful to participants for the interesting discussion, and to Dr Lela Mélon and Prof Carlos Gómez Ligüerre for the kind invitation to participate in this activitity of their research group on patrimonial law. I am also grateful to Karolis Granickas for comments on an earlier draft. The standard disclaimer applies.**
Digital technologies, public procurement and sustainability: some exploratory thoughts
1. Introductory detour
The use of public procurement as a tool to further sustainability goals is not a new topic, but rather the object of a long-running discussion embedded in the broader setting of the use of procurement for the pursuit of horizontal or secondary goals—currently labelled smart or strategic procurement. The instrumentalisation of procurement for (quasi)regulatory purposes gives rise to a number of issues, such as: regulatory transfer; the distortion of the very market mechanisms on which procurement rules rely as a result of added regulatory layers and constraints; legitimacy and accountability issues; complex regulatory impact assessments; professionalisation issues; etc.
Discussions in this field are heavily influenced by normative and policy positions, which are not always clearly spelled out but still drive most of the existing disagreement. My own view is that the use of procurement for horizontal policies is not per se desirable. The simple fact that public expenditure can act as a lever/incentive to affect private (market) behaviour does not mean that it should be used for that purpose at every opportunity and/or in an unconstrained manner. Procurement should not be used in lieu of legislation or administrative regulation where it is a second-best regulatory tool. Embedding regulatory elements that can also achieve horizontal goals in the procurement process should only take place where it has clear synergies with the main goal of procurement: the efficient satisfaction of public sector needs and/or needs in the public interest. This generates a spectrum of potential uses of procurement of a different degree of desirability.
At one end, and at its least desirable, procurement can and is used as a trade barrier for economic protectionism. In my view, this should not happen. At the other end of the spectrum, at its most desirable, procurement can and is (sometimes) used in a manner that supports environmental sustainability and technical innovation. In my view, this should happen, and more than it currently does. In between these two ends, there are uses of procurement for the promotion of labour and social standards, as well as for the promotion of human rights. Controversial as this position is, in my view, the use of procurement for the pursuit of those goals should be subjected to strict proportionality analysis in order to make sure that the secondary goal does not prevent the main purpose of the efficient satisfaction of public sector needs and/or needs in the public interest.
From a normative perspective, thus, I think that there is a wide space of synergy between procurement and environmental sustainability—which goes beyond green procurement and extends to the use of procurement to support a more circular economy—and that this can be used more effectively than is currently the case, due to emerging innovative uses of digital technologies for procurement governance.
This is the topic in which I would like to concentrate, to formulate some exploratory thoughts. The following reflections are focused on the EU context, but hopefully they are of a broader relevance. I first zoom in on the strategic priorities of fostering sustainability through procurement (2) and the digitalisation of procurement (3), as well as critically assess the current state of development of digital technologies for procurement governance (4). I then look at the interaction between both strategic goals, in terms of the potential for sustainable digital procurement (5), which leads to specific discussion of the need for an enabling data architecture (6), the potential for AI and sustainable procurement (7), the potential for the implementation of blockchains for sustainable procurement (8) and the need to refocus the emerging guidelines on the procurement of digital technologies to stress their sustainability dimension (9). Some final thoughts conclude (10).
2. Public procurement and sustainability
As mentioned above, the use of public procurement to promote sustainability is not a new topic. However, it has been receiving increasing attention in recent policy-making and legislative efforts (see eg this recent update)—though they are yet to translate in the level of practical change required to make a relevant contribution to pressing challenges, such as the climate emergency (for a good critique, see this recent post by Lela Mélon).
Facilitating the inclusion of sustainability-related criteria in procurement was one of the drivers for the new rules in the 2014 EU Public Procurement Package, which create a fairly flexible regulatory framework. Most remaining problems are linked to the implementation of such a framework, not its regulatory design. Cost, complexity and institutional inertia are the main obstacles to a broader uptake of sustainable procurement.
The European Commission is alive to these challenges. In its procurement strategy ‘Making Procurement work in and for Europe’ [COM(2017) 572 final; for a critical assessment, see here], the Commission stressed the need to facilitate and to promote the further uptake of strategic procurement, including sustainable procurement.
However, most of its proposals are geared towards the publication of guidance (such as the Buying Green! Handbook), standardised solutions (such as the library of EU green public procurement criteria) and the sharing of good practices (such as in this library of use cases) and training materials (eg this training toolkit). While these are potentially useful interventions, the main difficulty remains in their adoption and implementation at Member State level.
While it is difficult to have a good view of the current situation (see eg the older studies available here, and the terrible methodology used for this 2015 PWC study for the Commission), it seems indisputable that there are massive differences across EU Member States in terms of sustainability-oriented innovation in procurement.
Taking as a proxy the differences that emerge from the Eco-Innovation Scoreboard, it seems clear that this very different level of adoption of sustainability-related eco-innovation is likely reflective of the different approaches followed by the contracting authorities of the different Member States.
Such disparities create difficulties for policy design and coordination, as is acknowledged by the Commission and the limitations of its procurement strategy. The main interventions are thus dependent on Member States (and their sub-units).
3. Public procurement digitalisation beyond e-Procurement
Similarly to the discussion above, the bidirectional relationship between the use of procurement as a tool to foster innovation, and the adaptation of procurement processes in light of technological innovations is not a new issue. In fact, the transition to electronic procurement (eProcurement) was also one of the main drivers for the revision of the EU rules that resulted in the 2014 Public Procurement Package, as well as the flanking regulation of eInvoicing and the new rules on eForms. eProcurement (broadly understood) is thus an area where further changes will come to fruition within the next 5 years (see timeline below).
However, even a maximum implementation of the EU-level eProcurement rules would still fall short of creating a fully digitalised procurement system. There are, indeed, several aspects where current technological solutions can enable a more advanced and comprehensive eProcurement system. For example, it is possible to automate larger parts of the procurement process and to embed compliance checks (eg in solutions such as the Prozorro system developed in Ukraine). It is also possible to use the data automatically generated by the eProcurement system (or otherwise consolidated in a procurement register) to develop advanced data analytics to support procurement decision-making, monitoring, audit and the deployment of additional screens, such as on conflicts of interest or competition checks.
Progressing the national eProcurement systems to those higher levels of functionality would already represent progress beyond the mandatory eProcurement baseline in the 2014 EU Public Procurement Package and the flanking initiatives listed above; and, crucially, enabling more advanced data analytics is one of the effects sought with the new rules on eForms, which aim to significantly increase the availability of (better) procurement data for transparency purposes.
Although it is an avenue mainly explored in other jurisdictions, and currently in the US context, it is also possible to create public marketplaces akin to Amazon/eBay/etc to generate a more user-friendly interface for different types of catalogue-based eProcurement systems (see eg this recent piece by Chris Yukins).
Beyond that, the (further) digitalisation of procurement is another strategic priority for the European Commission; not only for procurement’s sake, but also in the context of the wider strategy to create an AI-friendly regulatory environment and to use procurement as a catalyst for innovations of broader application – along lines of the entrepreneurial State (Mazzucato, 2013; see here for an adapted shorter version).
Indeed, the Commission has formulated a bold(er) vision for future procurement systems based on emerging digital technologies, in which it sees a transformative potential: “New technologies provide the possibility to rethink fundamentally the way public procurement, and relevant parts of public administrations, are organised. There is a unique chance to reshape the relevant systems and achieve a digital transformation” (COM(2017) 572 fin at 11).
Even though the Commission has not been explicit, it may be worth trying to map which of the currently emerging digital technologies could be of (more direct) application to procurement governance and practice. Based on the taxonomy included in a recent OECD report (2019a, Annex C), it is possible to identify the following types and specific technologies with potential procurement application:
AI solutions
Virtual Assistants (Chat bots or Voice bots): conversational, computer-generated characters that simulate a conversation to deliver voice- or text-based information to a user via a Web, kiosk or mobile interface. A VA incorporates natural-language processing, dialogue control, domain knowledge and a visual appearance (such as photos or animation) that changes according to the content and context of the dialogue. The primary interaction methods are text-to-text, text-to-speech, speech-to-text and speech-to-speech;
Natural language processing: technology involves the ability to turn text or audio speech into encoded, structured information, based on an appropriate ontology. The structured data may be used simply to classify a document, as in “this report describes a laparoscopic cholecystectomy,” or it may be used to identify findings, procedures, medications, allergies and participants;
Machine Learning: the goal is to devise learning algorithms that do the learning automatically without human intervention or assistance;
Deep Learning: allows computational models that are composed of multiple processing layers to learn representations of data with multiple levels of abstraction;
Robotics: deals with the design, construction, operation, and use of robots, as well as computer systems for their control, sensory feedback, and information processing;
Recommender systems: subclass of information filtering system that seeks to predict the "rating" or "preference" that a user would give to an item;
Expert systems: is a computer system that emulates the decision-making ability of a human expert;
Digital platforms
Distributed ledger technology (DLT): is a consensus of replicated, shared, and synchronized digital data geographically spread across multiple sites, countries, or institutions. There is no central administrator or centralised data storage. A peer-to-peer network is required as well as consensus algorithms to ensure replication across nodes is undertaken; Blockchain is one of the most common implementation of DLT;
Smart contracts: is a computer protocol intended to digitally facilitate, verify, or enforce the negotiation or performance of a contract;
IoT Platform: platform on which to create and manage applications, to run analytics, and to store and secure your data in order to get value from the Internet of Things (IoT);
Not all technologies are equally relevant to procurement—and some of them are interrelated in a manner that requires concurrent development—but these seem to me to be those with a higher potential to support the procurement function in the future. Their development needs not take place solely, or primarily, in the context of procurement. Therefore, their assessment should be carried out in the broader setting of the adoption of digital technologies in the public sector.
4. Digital technologies & the public sector, including procurement
The emergence of the above mentioned digital technologies is now seen as a potential solution to complex public policy problems, such as the promotion of more sustainable public procurement. Keeping track of all the potential use cases in the public sector is difficult and the hype around buzzwords such as AI, blockchain or the internet of things (IoT) generates inflated claims of potential solutions to even some of the most wicked public policy problems (eg corruption).
This is reflective of the same hype in private markets, and in particular in financial and consumer markets, where AI is supposed to revolutionise the way we live, almost beyond recognition. There also seems to be an emerging race to the top (or rather, a copy-cat effect) in policy-making circles, as more and more countries adopt AI strategies in the hope of harnessing the potential of these technologies to boost economic growth.
In my view, digital technologies are receiving excessive attention. These are immature technologies and their likely development and usefulness is difficult to grasp beyond a relatively abstract level of potentiality. As such, I think these technologies may be receiving excessive attention from policy-makers and possibly also disproportionate levels of investment (diversion).
The implementation of digital technologies in the public sector faces a number of specific difficulties—not least, around data availability and data skills, as stressed in a recent OECD report (2019b). While it is probably beyond doubt that they will have an impact on public governance and the delivery of public services, it is more likely to be incremental rather than disruptive or revolutionary. Along these lines, another recent OECD report (2019c) stresses the need to take a critical look at the potential of artificial intelligence, in particular in relation to public sector use cases.
The OECD report (2019a) mentioned above shows how, despite these general strategies and the high levels of support at the top levels of policy-making, there is limited evidence of significant developments on the ground. This is the case, in particular, regarding the implementation of digital technologies in public procurement, where the OECD documents very limited developments (see table below).
Of course, this does not mean that we will not see more and more widespread developments in the coming years, but a note of caution is necessary if we are to embrace realistic expectations about the potential for significant changes resulting from procurement digitalisation. The following sections concentrate on the speculative analysis of such potential use of digital technologies to support sustainable procurement.
5. Sustainable digital procurement
Bringing together the scope for more sustainable public procurement (2), the progressive digitalisation of procurement (3), and the emergence of digital technologies susceptible of implementation in the public sector (4); the combined strategic goal (or ideal) would be to harness the potential of digital technologies to promote (more) sustainable procurement. This is a difficult exercise, surrounded by uncertainty, so the rest of this post is all speculation.
In my view, there are different ways in which digital technologies can be used for sustainability purposes. The contribution that each digital technology (DT) can make depends on its core functionality. In simple functional terms, my understanding is that:
AI is particularly apt for the massive processing of (big) data, as well as for the implementation of data-based machine learning (ML) solutions and the automation of some tasks (through so-called robotic process automation, RPA);
Blockchain is apt for the implementation of tamper-resistant/evident decentralised data management;
The internet of things (IoT) is apt to automate the generation of some data and (could be?) apt to breach the virtual/real frontier through oracle-enabled robotics
The timeline that we could expect for the development of these solutions is also highly uncertain, although there are expectations for some technologies to mature within the next four years, whereas others may still take closer to ten years.
© Gartner, Aug 2018.
Each of the core functionalities or basic strengths of these digital technologies, as well as their rate of development, will determine a higher or lower likelihood of successful implementation in the area of procurement, which is a highly information/data-sensitive area of public policy and administration. Therefore, it seems unavoidable to first look at the need to create an enabling data architecture as a priority (and pre-condition) to the deployment of any digital technologies.
6. An enabling data architecture as a priority
The importance of the availability of good quality data in the context of digital technologies cannot be over-emphasised (see eg OECD, 2019b). This is also clear to the European Commission, as it has also included the need to improve the availability of good quality data as a strategic priority. Indeed, the Commission stressed that “Better and more accessible data on procurement should be made available as it opens a wide range of opportunities to assess better the performance of procurement policies, optimise the interaction between public procurement systems and shape future strategic decisions” (COM(2017) 572 fin at 10-11).
However, despite the launch of a set of initiatives that seek to improve the existing procurement data architecture, there are still significant difficulties in the generation of data [for discussion and further references, see A Sanchez-Graells, “Data-driven procurement governance: two well-known elephant tales” (2019) 24(4) Communications Law 157-170; idem, “Some public procurement challenges in supporting and delivering smart urban mobility: procurement data, discretion and expertise”, in M Finck, M Lamping, V Moscon & H Richter (eds), Smart Urban Mobility – Law, Regulation, and Policy, MPI Studies on Intellectual Property and Competition Law (Springer 2020) forthcoming; and idem, “EU Public Procurement Policy and the Fourth Industrial Revolution: Pushing and Pulling as One?”, Working Paper for the YEL Annual Conference 2019 ‘EU Law in the era of the Fourth Industrial Revolution’].
To be sure, there are impending advances in the availability of quality procurement data as a result of the increased uptake of the Open Contracting Data Standards (OCDS) developed by the Open Contracting Partnership (OCP); the new rules on eForms; the development of eGovernment Application Programming Interfaces (APIs); the 2019 Open Data Directive; the principles of business to government data sharing (B2G data sharing); etc. However, it seems to me that the European Commission needs to exercise clearer leadership in the development of an EU-wide procurement data architecture. There is, in particular, one measure that could be easily adopted and would make a big difference.
The 2019 Open Data Directive (Directive 2019/1024/EU, ODD) establishes a special regime for high-value datasets, which need to be available free of charge (subject to some exceptions); machine readable; provided via APIs; and provided as a bulk download, where relevant (Art 14(1) ODD). Those high-value datasets are yet to be identified by the European Commission through implementing acts aimed at specifying datasets within a list of thematic categories included in Annex I, which includes the following datasets: geospatial; Earth observation and environment; meteorological; statistics; companies and company ownership; and mobility. In my view, most relevant procurement data can clearly fit within the category of statistical information.
More importantly, the directive specifies that the ‘identification of specific high-value datasets … shall be based on the assessment of their potential to: (a) generate significant socioeconomic or environmental benefits and innovative services; (b) benefit a high number of users, in particular SMEs; (c) assist in generating revenues; and (d) be combined with other datasets’ (Art 14(2) ODD). Given the high-potential of procurement data to unlock (a), (b) and (d), as well as, potentially, generate savings analogous to (c), the inclusion of datasets of procurement information in the future list of high-value datasets for the purposes of the Open Data Directive seems like an obvious choice.
Of course, there will be issues to iron out, as not all procurement information is equally susceptible of generating those advantages and there is the unavoidable need to ensure an appropriate balance between the publication of the data and the protection of legitimate (commercial) interests, as recognised by the Directive itself (Art 2(d)(iii) ODD) [for extended discussion, see here]. However, this would be a good step in the direction of ensuring the creation of a forward-looking data architecture.
At any rate, this is not really a radical idea. At least half of the EU is already publishing some public procurement open data, and many Eastern Partnership countries publish procurement data in OCDS (eg Moldova, Ukraine, Georgia). The suggestion here would bring more order into this bottom-up development and would help Member States understand what is expected, where to get help from, etc, as well as ensure the desirable level of uniformity, interoperability and coordination in the publication of the relevant procurement data.
Beyond that, in my view, more needs to be done to also generate backward-looking databases that enable the public sector to design and implement adequate sustainability policies, eg in relation to the repair and re-use of existing assets.
Only when the adequate data architecture is in place, will it be possible to deploy advanced digital technologies. Therefore, this should be given the highest priority by policy-makers.
7. Potential AI uses for sustainable public procurement
If/when sufficient data is available, there will be scope for the deployment of several specific implementations of artificial intelligence. It is possible to imagine the following potential uses:
Sustainability-oriented (big) data analytics: this should be relatively easy to achieve and it would simply be the deployment of big data analytics to monitor the extent to which procurement expenditure is pursuing or achieving specified sustainability goals. This could support the design and implementation of sustainability-oriented procurement policies and, where appropriate, it could generate public disclosure of that information in order to foster civic engagement and to feedback into political processes.
Development of sustainability screens/indexes: this would be a slight variation of the former and could facilitate the generation of synthetic data visualisations that reduced the burden of understanding the data analytics.
Machine Learning-supported data analysis with sustainability goals: this could aim to train algorithms to establish eg the effectiveness of sustainability-oriented procurement policies and interventions, with the aim of streamlining existing policies and to update them at a pace and level of precision that would be difficult to achieve by other means.
Sustainability-oriented procurement planning: this would entail the deployment of algorithms aimed at predictive analytics that could improve procurement planning, in particular to maximise the sustainability impact of future procurements.
Moreover, where clear rules/policies are specified, there will be scope for:
Compliance automation: it is possible to structure procurement processes and authorisations in such a way that compliance with pre-specified requirements is ensured (within the eProcurement system). This facilitates ex ante interventions that could minimise the risk of and the need for ex post contractual modifications or tender cancellations.
Recommender/expert systems: it would be possible to use machine learning to assist in the design and implementation of procurement processes in a way that supported the public buyer, in an instance of cognitive computing that could accelerate the gains that would otherwise require more significant investments in professionalisation and specialisation of the workforce.
Chatbot-enabled guidance: similarly to the two applications above, the use of procurement intelligence could underpin chatbot-enabled systems that supported the public buyers.
A further open question is whether AI could ever autonomously generate new sustainability policies. I dare not engage in such exercise in futurology…
8. Limited use of blockchain/DLTs for sustainable public procurement
By contrast with the potential for big data and the AI it can enable, the potential for blockchain applications in the context of procurement seems to me much more limited (for further details, see here, here and here). To put it simply, the core advantages of distributed ledger technologies/blockchain derive from their decentralised structure.
Whereas there are several different potential configurations of DLTs (see eg Rauchs et al, 2019 and Alessie et al, 2019, from where the graph is taken), the configuration of the blockchain affects its functionalities—with the highest levels of functionality being created by open and permissionless blockchains.
However, such a structure is fundamentally uninteresting to the public sector, which is unlikely to give up control over the system. This has been repeatedly stressed and confirmed in an overview of recent implementations (OECD, 2019a:16; see also OECD, 2018).
Moreover, even beyond the issue of public sector control, it should be stressed that existing open and permissionless blockchains operate on the basis of a proof-of-work (PoW) consensus mechanism, which has a very high carbon footprint (in particular in the case of Bitcoin). This also makes such systems inapt for sustainable digital procurement implementations.
Therefore, sustainable blockchain solutions (ie private & permissioned, based on proof-of-stake (PoS) or a similar consensus mechanisms), are likely to present very limited advantages for procurement implementation over advanced systems of database management—and, possibly, even more generally (see eg this interesting critical paper by Low & Mik, 2019).
Moreover, even if there was a way to work around those constraints and design a viable technical solution, that by itself would still not fix underlying procurement policy complexity, which will necessarily impose constraints on technologies that require deterministic coding, eg
Tenders on a blockchain - the proposals to use blockchain for the implementation of the tender procedure itself are very limited, in my opinion, by the difficulty in structuring all requirements on the basis of IF/THEN statements (see here).
Smart (public) contracts - the same constraints apply to smart contracts (see here and here).
Blockchain as an information exchange platform (Mélon, 2019, on file) - the proposals to use blockchain mechanisms to exchange information on best practices and tender documentation of successful projects could serve to address some of the confidentiality issues that could arise with ‘standard’ databases. However, regardless of the technical support to the exchange of information, the complexity in identifying best practices and in ensuring their replicability remains. This is evidenced by the European Commission’s Initiative for the exchange of information on the procurement of Large Infrastructure Projects (discussed here when it was announced), which has not been used at all in its first two years (as of 6 November 2019, there were no publicly-available files in the database).
9. Sustainable procurement of digital technologies
A final issue to take into consideration is that the procurement of digital technologies needs to itself incorporate sustainability considerations. However, this does not seem to be the case in the context of the hype and over-excitement with the experimentation/deployment of those technologies.
Indeed, there are emerging guidelines on procurement of some digital technologies, such as AI (UK, 2019) (WEF, 2019) (see here for discussion). However, as could be expected, these guidelines are extremely technology-centric and their interaction with broader procurement policies is not necessarily straightforward.
I would argue that, in order for these technologies to enable a more sustainable procurement, sustainability considerations need to be embedded not only in their application, but may well require eg an earlier analysis of whether the life-cycle of existing solutions warrants replacement, or the long-term impacts of the implementation of digital technologies (eg in terms of life-cycle carbon footprint).
Pursuing technological development for its own sake can have significant environmental impacts that must be assessed.
10. Concluding thoughts
This (very long…) blog post has structured some of my thoughts on the interaction of sustainability and digitalisation in the context of public procurement. By way of conclusion, I would just try to translate this into priorities for policy-making (and research). Overall, I believe that the main area of effort for policy-makers should now be in creating an enabling data architecture. Its regulation can thus focus research in the short term. In the medium-term, and as use cases become clearer in the policy-making sphere, research should be moving towards the design of digital technology-enabled solutions (for sustainable public procurement, but not only) and their regulation, governance and social impacts. The long-term is too difficult for me to foresee, as there is too much uncertainty. I can only guess that we will cross that bridge when/if we get there…