In his Opinion of 10 March 2016 in TNS Dimarso, C-6/15, EU:C:2016:160, Advocate General Mengozzi has addressed the general question whether EU public procurement rules 'read in the light of the principles of equal treatment and transparency, [require] that a contracting authority should always, or, if not, in certain circumstances, make known in advance, in the contract notice or the contract documents, the method of evaluation or weighting rules used to assess tenderers’ bids'. The case is to be decided by the Court of Justice of the European Union (CJEU) on the basis of the phasing-out rules in Art 53(2) of Directive 2004/18, but the interpretation will be relevant for the future application of Art 67(5) of Directive 2014/24.
In my view, the case is interesting, not primarily because of the discussion on whether evaluation methods need to be disclosed together with award criteria and their weightings, but more importantly because it brings to light the simple fact that some evaluation methods are unable to meet the requirements of the EU rules--to the effect that the award phase needs to enable the contracting authority to actually determine which is the most economically advantageous tender with a sufficient degree of precision and certainty. Thus, I critically assess AG Mengozzi's excellent opinion from this perspective.
The Dimarso case
In this case, a Belgian contracting authority issued a call for tenders for the provision of services and indicated that the award criteria would be as follows:
1 Quality of the tender (50/100)
Quality of the preparation, organisation and execution of the work on the ground, and of the encryption and initial data processing. The services proposed must be described in as much detail as possible. It must be clear from the tender that the tenderer is capable of taking on the whole contract (minimum 7 000 samples / maximum 10 000 samples) within the prescribed 12-month delivery deadline.
2 Price (50/100)
Cost of delivering the contract in relation to the basic sample (7 000 samples) and cost per additional batch of 500 addresses supplied (amounts inclusive of VAT).
There was no further indication of how these criteria would be applied. When it came to evaluation of the tenders received, the evaluation team 'evaluated and compared with each other on the basis of the criteria set out above. First, the tenders were examined and evaluated on the basis of the “quality” criterion. For this, each tender was unanimously assigned a given score (high — satisfactory — low). Then, the price criterion was applied. On the basis of those scores, a final ranking was established' (Opinion in C-6/15, para 5, emphasis added).
Dimarso submitted a bid that scored high on quality grounds and was the highest on price. The contract was awarded to a competing tenderer which offer also scored high on quality and was lower in price. Dimarso challenges the way the evaluation team applied the award criteria on the following grounds:
the evaluation committee appears to have evaluated the tenders on the basis of the ‘high — satisfactory — low’ scale, not referred to in the contract documents, in relation to the tender quality criterion, whereas, according to Dimarso, it is clear from the contract documents that a score of 0 to 50 points should have been allocated to each tender. As regards the price criterion, the evaluation committee also failed to carry out an adequate examination, comparison and final assessment of the tenders taking into account the award criteria as set out in the contract documents, including the “50/100” weighting given to each of the award criteria in the call for tenders (Opinion in C-6/15, para 8, emphasis added).
This question raises then two issues: (1) whether the evaluation committee could rely on 'soft metrics' in order to apply the quality award criterion; and (2) whether such 'soft metrics' could be combined with straightforward price comparisons. I find these two questions of great practical relevance, so it is worth looking closely at AG Mengozzi's reasoning on these issues.
Assessment under Art 53(2) Dir 2004/18
It is worth reminding that Art 53(2) Dir 2004/18 established that
[when the award is made to the tender most economically advantageous from the point of view of the contracting authority], the contracting authority shall specify in the contract notice or in the contract documents ... the relative weighting which it gives to each of the criteria chosen to determine the most economically advantageous tender.
Those weightings can be expressed by providing for a range with an appropriate maximum spread.
Where, in the opinion of the contracting authority, weighting is not possible for demonstrable reasons, the contracting authority shall indicate in the contract notice or contract documents or, in the case of a competitive dialogue, in the descriptive document, the criteria in descending order of importance.
At this point, it is worth stressing that the only difference between Art 53(2) Dir 2004/18 and Art 67(5) Dir 2014/24 is that, in relation to the third paragraph, the seemingly permissive drafting of Art 53(2)III Dir 2004/18 ('Where, in the opinion of the contracting authority, weighting is not possible for demonstrable reasons') is tightened up in Art 67(5)III Dir 2014/24 ('Where weighting is not possible for objective reasons'). Given the strict interpretation that AG Mengozzi proposes for Art 53(2) Dir 2004/18 (which is to be shared), his Opinion will be equally relevant for the future interpretation of Art 67(5) Dir 2014/24 [along the same lines, see A Sanchez-Graells, Public Procurement and the EU Competition Rules, 2nd edn (Oxford, Hart, 2015) 384-385].
Going back to the Dimarso case, AG Mengozzi starts by summing up the content of this provision by stressing that
the obligation to indicate not only the award criteria but also ... the relative weighting given to each of those criteria, except where there are good reasons why weighting is not possible, at the time of publication of the contract notice or contract documents ... serves to fulfil the requirement of compliance with the principle of equal treatment and the associated obligation of transparency (Opinion in C-6/15, para 20).
And that
contracting authorities have an obligation to indicate the weightings of the award criteria in the contract notice or the contract documents. It is only in the event that this proves impossible, for demonstrable reasons, that those entities may opt to prioritise those criteria, which prioritisation must in any event be adequately disclosed in the contract notice or the contract documents (Opinion in C-6/15, para 23, emphasis added).
The AG clarifies (paras 24-28) that the dispute in the case at hand is not whether having indicated that Quality (50/100) and Price (50/100) meant that both award criteria had equal weight or how they had to be combined amongst themselves to reach a final ranking of tenders, but that it is rather
in essence, [whether] the method of evaluation used (‘low — medium — high’) was so vague that it prompted the contracting authority to downgrade the assessment of the ‘quality’ criterion relative to that of the ‘price’ criterion, since the second criterion alone was actually capable of eliminating three of the four tenders submitted. In reality, therefore, Dimarso contends, the price criterion benefited from a higher relative weighting than the 50% previously announced in the contract documents. In other words, Dimarso submits that, if the method of evaluation had been made known to tenderers in advance, at the stage when the contract documents were published, it would inevitably have had an effect on the preparation of the tenders (Opinion in C-6/15, para 29, emphasis added).
AG Mengozzi then approaches this argument in stages. His reasoning heavily rests on two aspects. First, that it is clear that Art 53(2) Dir 2004/18 does not explicitly impose an obligation to disclose the evaluation method in addition to disclosure of award criteria and their weightings (para 32). Second, and notwithstanding that literal interpretation of Art 53(2) Dir 2004/18, that the CJEU has been clear in the imposition of restrictions on the way the evaluation team carries out its tasks (paras 37 ff). In my reading, the bone of his argument is as follows.
In relation to the setting of sub-weightings (or weighting factors for award sub-criteria), the CJEU has indicated that this is not a breach of EU procurement rules provided three conditions are met: ie '[1] that it does not alter the criteria for the award of the contract set out in the contract documents or the contract notice, [2] that it does not contain elements which, if they had been known at the time the tenders were prepared, could have affected that preparation, and [3] that it was not adopted taking into account matters likely to give rise to discrimination against one of the tenderers' [with reference to judgments in ATI EAC e Viaggi di Maio and Others (C‑331/04, EU:C:2005:718, paragraph 32); Lianakis and Others (C‑532/06, EU:C:2008:40, paragraph 43); and Evropaïki Dynamiki v EMSA (C‑252/10 P, EU:C:2011:512, paragraph 33); Opinion in C-6/15, para 40, emphasis added].
The same restrictions should be applicable to the adoption of an evaluation method because it can create the same effects as the adoption of sub-weightings--or, in his words, 'it is not inconceivable that a method of evaluation may have an effect not so much on the award criteria themselves as on the weighting of those criteria and, as such, may contain elements which would have been capable of influencing the preparation of tenders if that method had been made known to tenderers in advance... In that event, the ex post determination of such a method for evaluating tenders by a contracting authority would be unlawful and should, therefore, have been disclosed in advance in the contract notice or the contract documents' (Opinion in C-6/15, para 46). Therefore, 'the lawfulness of a method for evaluating tenders which is determined by a contracting authority ex post depends on whether the three conditions established by the Court’s case-law ... are met' (Opinion in C-6/15, para 47).
© iStockphoto.com/RichVintage
Opportunity for further clarification
Having disposed of the core of the case, AG Mengozzi goes on to suggest that the CJEU take this opportunity to clarify its case law and to stress that the adoption of evaluation methods need to be subjected to tighter requirements. His arguments are based on the use of 'soft quality metrics'--and, more specifically, on the distrust in their ability to actually enable the contracting authority to identify the most economically advantageous tender (MEAT)--as it emerges from his explanation of his main concern:
let us imagine that, of the tenders submitted, one was far superior, in terms of quality, to the other three, including those that were rated ‘high’. In other words, one of those tenders could have been ranked ‘excellent’ in the assessment of the ‘quality’ criterion. The price proposed by that tenderer would then have reflected the excellence of the quality of the services proposed by it and would therefore in all probability have been higher than the prices offered by the other tenderers. However, since ‘excellent’ did not feature on the range of scores (low — satisfactory — high) chosen by the evaluation committee, that tender of excellent quality could not but be rated ‘high’, at the very most, in relation to the ‘quality’ criterion. Since the price proposed by the tenderer of that bid was higher than those proposed by the others, possibly even by some tens or hundreds of euros, that bid had to be rejected... in that situation, ... the contracting authority might have been deprived of the tender representing the best value for money, contrary to the spirit in which the selection of tenderers on the basis of the most economically advantageous tender takes place (Opinion in C-6/15, paras 56-57, emphasis added).
AG Mengozzi considers that this is an unsatisfactory state of affairs and, in my reading, proposes that the existing case law of the CJEU is clarified so that contracting authorities do not create a situation where tenderers submit offers which positive attributes are not captured by the evaluation method. His proposal thus focuses on the need to disclose the evaluation method to be used from the start of the procurement process. In his view, 'the likelihood is ... that, if the method for evaluating tenders in the light of the ‘quality’ criterion, as established by the contracting authority, had been known in advance by the potential tenderers, it would have been capable of affecting the preparation of their tenders' (Opinion in C-6/15, para 60). Therefore,
the contracting authority (to which it will fall to ensure that the tendering procedure benefits from maximum legal certainty and to protect itself against actions for the annulment of that procedure) must determine the method or methods to be used to evaluate tenders in the light of the award criteria as early as possible. It would be reasonable to suggest, then, that, if that is the case, there does not appear to be any overriding reason such as to justify a refusal by the contracting authority to make known to potential tenderers the methods of evaluation in question, which it will in any event already have had to determine before the call for tenders (Opinion in C-6/15, para 63, emphasis added).
AG Mengozzi then goes on to discuss whether the condition should be to only require upfront disclosure of evaluation methods which have the potential to create a substantial impact effect on the preparation of the tenders, which he dismisses (paras 70 ff), on the basis that the system would be properly balanced 'by the obligation incumbent on the unsuccessful tenderer, which bears the burden of proof, to demonstrate, by reference to specific examples in its legal action, the differences (substantive as well as purely formal) which its tender would have exhibited if the elements of the method of evaluation in question or the method itself, which the contracting authority neglected to communicate, had been adequately disclosed before the tenders were prepared' (Opinion in C-6/15, para 73).
personal critique
I share AG Mengozzi's views and concern, but I think that his proposal simply to disclose evaluation methods upfront would only carry us half way in sorting out the unresolved issue of the use of of 'soft quality metrics' in the evaluation of tenders. Regardless of upfront disclosure, which needs to take place, a method for the evaluation of quality aspects of procurement tenders that classifies tenders in pre-determined, tight 'quality levels' is bound to offer sub-optimal results. In the extreme,
a binary approach—ie, an approach based on meeting or not meeting a criterion, or an ‘all-or-nothing’ (or zero/one) approach—seems less desirable than a gradual approach or the adoption of sliding-scale-based evaluation rules ... whenever possible, it seems preferable that contracting authorities evaluate the degree to which tenders comply with each of the specified award criteria on a sliding scale (such as granting them points from 0 to 10, or 1 to 5, or any other scale). In this regard, the weighting of criteria will become less harsh and the appraisal of the tenders will arguably reflect with greater accuracy their relative strengths and weaknesses according to the overall set of award criteria' [A Sanchez-Graells, Public Procurement and the EU Competition Rules, 2nd edn (Oxford, Hart, 2015) 391; Along the same lines, but opting for a monetary equivalent approach, see MA Bergman and S Lundberg, ‘Tender Evaluation and Supplier Selection Methods in Public Procurement’ (2013) 19(2) Journal of Purchasing and Supply Management 73].
More importantly, the assessment of quality elements needs to take place in a manner that does result in a loss of information of the relative quality of the offers. It has been the settled case law of the CJEU that
although [the EU rules do] not set out an exhaustive list of the criteria which may be chosen by the contracting authorities, and therefore leaves it open to the authorities awarding contracts to select the criteria on which they propose to base their award of the contract, their choice is nevertheless limited to criteria aimed at identifying the tender which is economically the most advantageous (Case C-532/06 Lianakis [2008] ECR I-251 29 (emphasis added); Case 31/87 Beentjes [1988] ECR 4635 19; Case C-19/00 SIAC Construction [2001] ECR I-7725 35–36; Case C-513/99 Concordia Bus Finland [2002] ECR I-7213 54 and 59; and Case C-315/01 GAT [2003] ECR I-6351 63–64. See also Case C-448/01 EVN and
Wienstrom [2003] ECR I-14527 37).
In my view, it is particularly relevant 'to stress the need for award criteria (i) to be linked to the subject matter of the contract (ie, to be ‘relevant’), and (ii) to allow the contracting authority actually to determine which tender is economically the most advantageous (ie, to be ‘enabling’)' [A Sanchez-Graells, Public Procurement and the EU Competition Rules, 2nd edn (Oxford, Hart, 2015) 380]. The same reasoning must apply (functionally) to the selection of evaluation methods (for the reasons explained by AG Mengozzi, ie that they create the same effects).
Quite frankly, in view of the clear example AG Mengozzi has given us (para 56), I would have no doubt that the use of 'soft quality metrics' is not enabling because it does not allow the contracting authority to identify, with an adequate level of precision and certainty, the most economically advantageous offer. Ultimately, thus, they should be banned as a matter of EU law--and, more generally, of good procurement practice. I do not expect the CJEU to go as far as to agree with this, but I think it would be the only consistent solution, and one that would do away with the problem, rather than trying to fix it simply with the remedy of more transparency--which seems to be the token fix-all solution in procurement law.