Carlos Arrebola and Ana Julia Mauricio (PhD students at the University of Cambridge), together with Héctor Jiménez Portilla (of the Overseas Development Institute (ODI)) have published an interesting and thought-provoking paper (*) where they try to measure the influence of the Advocate General (AG) on the Court of Justice of the European Union (CJEU) [for a short summary of their paper, see here]. This is an area where EU law scholars have been struggling to find an objective way to measure/prove/dimiss any claim of AG influence over the CJEU--as Arrebola et al clearly stress in their excellent literature review.
In a nutshell, Arrebola et al claim that their 'findings suggest that the CJEU is approximately 67 percent more likely to annul an act (or part of it) if the AG advises the Court to annul than if it advises the Court to dismiss the case or declare it inadmissible. In their view, these results raise several questions as regards judicial independence and the relevance of the figure of the Advocate General, providing a grounded basis for future discussions and judicial reform.'
Their claim is as intuitively appealing as it is bold (and controversial, in terms of the implications they derive) and, in my view, it deserves a tough scrutiny of the way they reached this conclusion. The following are some of the doubts that I have had while reading the paper, which I am limiting to the three main doubts I am struggling with. Overall, these doubts leave me with the impression that, unfortunately, the paper does not actually deliver on its main goal of contributing 'to a more comprehensive understanding of the role of the Advocate General in the makeup of the Court of Justice of the European Union'.
Their model in a nutshell
Let me frame my doubts in an stylised summary of their econometric model. In short, they have looked at 'data from 20 years of actions for annulment
procedures before the Court of Justice. Every case from January 1994 to January
2014 has been included, with the exception of appeals from the General Court and
those cases that do not have an AG opinion. We collected a total of 285 observations.
For these cases, we have examined the behaviour of the Court and the Advocate
General as regards to their decision to annul or not to annul the legal act in question' (p. 15).
They have coded these cases to examine the relationship between two main variables: the recommendation of the AG and the final decision of the CJEU. There are other variables they take into account, but those do not affect my analysis, so I am sticking to the two main variables for simplicity of argument. They explain why they have chosen annulment cases in the following terms: 'we have created two dichotomous (also called dummy or binary)
variables: ECJannulment and AGannulment. ECJannulment is the one that we have
considered as the dependent variable. It takes the value of 1 if the Court decided to
annul or partially annul an act, and 0 if it dismissed the case or deemed it inadmissible.
AGannulment is the variable that we have considered independent. It takes the value
of 1 if the Advocate General issued an opinion recommending the Court to annul or
partially annul an act, and 0 if it recommended dismissing the case or declaring it
inadmissible' (p. 15).
With this information, they have run a 'probit model [which] is a
regression that explains the predicted probability of the dependent variable adopting
the value 1. In our case, it outputs the predicted probability of the Court annulling an
act, subject to the value given to the other variables included. Therefore, the probit
model provides a simple way to interpret the results in terms of predicted probability
from 0 to 1. Instead, if we had chosen a linear regression model, the result would not
be enclosed between 0 and 1, making the interpretation impossible, as it could yield
some predicted probabilities to be negative or above the unit' (p. 25). This is what allows them to reach their main finding that 'when the Advocate General recommends annulment, the
Court is 67 per cent more likely to annul' (p. 30).
My main doubts
Firstly, I am not sure that the model the authors use is the best suited to the analysis of such a complex issue as the influence of the AG on the CJEU. One of the reasons (probably the main reason) why the authors decide to use a probit model is that they consider that it is not possible to establish a group of annulment cases that can work as a control (ie what they call the impossibility of conducting a randomised controlled trial). They consider that this would be the best way to avoid selection bias, but that in their study 'it is not possible to create a randomised controlled trial to define the causal effect of the AG opinion on the Court of Justice. This would require having the ability to design empirical experiments using the Court of Justice as a laboratory, which is unfeasible in practice' (p. 13, with more details in fn 54).
I disagree with their view about the impossibility to use a randomised controlled trial. There is a group of annulment procedures where no AG Opinion was submitted, and this could be used as a control group. It is important to note that, according to the Statute of the CJEU, '[w]here it considers that the case raises no new point of law, the Court may decide, after hearing the Advocate General, that the case shall be determined without a submission from the Advocate General' [Art 20(5)].
This is organised according to the Rules of Procedure of the CJEU, according to which 'The preliminary report shall contain proposals as to whether particular
measures of organisation of procedure, measures of inquiry or, if
appropriate, requests to the referring court or tribunal for clarification
should be undertaken, and as to the formation to which the case should be
assigned. It shall also contain the Judge-Rapporteur’s proposals, if any, as to
whether to dispense with a hearing and as to whether to dispense with an
Opinion of the Advocate General pursuant to the fifth paragraph of Article
20 of the Statute. The Court shall decide, after hearing the Advocate General, what action
to take on the proposals of the Judge-Rapporteur' [Art 59(2) and (3)].
Therefore, the annulment cases where there is no AG Opinion are an important instrument for potential control tests. These cases only come to be decided without an AG Opinion because both the CJEU (rectius, the Judge-Rapporteur) and the AG agreed that the case raised no new point of law. Thus, there is no indication that the AG can influence the CJEU on any other point than the existence or not of new issues to be considered. Admittedly, there could already be scope for some indication of the AG (and the CJEU's) position on the substance of the case in this first judgement. However, I would think that running controls on the basis of these cases could be useful.
In these cases, the CJEU (at least formally), decided whether to admit or dismiss, annul (totally or partially) the case without submission of the AG. If there was a significant divergence of the probability of annulment between these two groups of cases, the argument that the author's raise in the paper would be strengthened. On the contrary, if the CJEU showed the same likelihood of annulling/dismissing regardless of the existence or not of an AG Opinion, the claim would be significantly weakened. I do not imagine this to the ultimate test for the arguments raised in the paper, but I would see it as an important one.
Secondly, I am skeptical of the way in which the authors simplify the setting for annulment procedures. They construct them as binary: that is, the only options available to the AG and the CJEU are to either declare the case inadmissible/dismiss it (0) or annul the provision in question totally/partially (1). I understand the need to simplify decisions to annul grouping together full and partial annulments (which they explain in p. 17). I remain unconvinced by their arguments regarding declarations of inadmissibility and dismissals. They simply indicate that 'inadmissibility and dismissal
are sometimes used as interchangeable terms, although technically the substance of
the case is not analysed in cases of inadmissibility, whilst it is in cases that are
dismissed' (p. 17). However, they do not consider this a major issue and proceed with the grouping of both types of results as a single outcome of the case.
The difficulty I have with this strategy is that the rules on admissibility/inadmissibility are procedural in nature and they set up a first filter for cases to come to a full analysis. It can also be argued that they are much simpler than the rules applicable to the potential annulment of the challenged provisions, which depend on much more complex assessments of both procedural and substantive EU law. Thus, grouping decisions on (procedural) inadmissibility with those on dismissal of the annulment claim after a full analysis seems to create a significant conceptual problem. At this point, it may be worth stressing that the authors had mentioned that
we have decided to estimate regressions including other variables that could potentially be biasing the results if we only looked at what the Advocate General said and whether the Court followed the Advocate General’s position. In particular, one of the bias factors is the clarity of the law in a given case. For example, the Court and the Advocate General could reach the same result in a case not because the Court decided to follow the AG opinion, but because the law was clear on what the outcome should be, and there was no room for different interpretations. Therefore, not accounting for the clarity of the case could overestimate our measure of the influence of the Advocate General (pp. 14-15, emphasis added).
My problem is that the authors seems to have forgotten to include this very bias-check in the way they have constructed their variables. By grouping (relatively simpler) procedural checks with (relatively more complex) full assessments, they have created a variable that is very hard to reconcile with reality outside of their model.
Thirdly, even within the context of their model, I am not sure what to make of their results. Their findings indicate that, when the AG recommends the annulment of an act, the CJEU is almost
67 per cent more likely to annul the act than if the AG had not
proposed its annulment (ie, had she advocated for either inadmissibility or dismissal). I have trouble interpreting this number due to the conceptual issue mentioned above (ie, conflation of inadmission and dismissal), which makes the recommendation of the AG (as coded) ambiguous. This makes me wary of the claim that 'even if the number of 67 per cent of increased probability is called into question, it is difficult to deny that there is some level of influence' of the AG on the CJEU (p. 34, emphasis added), and that 'our analysis shows that there is some component in the making of a decision that is simply attributed to what the Advocate General recommended' (p. 35, emphasis added).
From the numbers in the paper, I have been unable to work out the effect that an AG recommendation to inadmit/dismiss has on the CJEU's willingness to do so. Intuitively, I would expect that, if by itself the Opinion of the AG is such a relevant factor as the paper claims, then the CJEU should also be more inclined to inadmit/dismiss when the AG submitted such a recommendation. However, in that case, I would not necessarily find the causal explanation between the AG recommendation and the CJEU's decision persuasive. An alternative interpretation not linked to the influence of the AG over the CJEU would need to be dispelled: ie the zeal with which the CJEU keeps control of its docket. The intuition would be that the CJEU may be engaged in an interpretation of inadmissibility rules that prevents a floodgate of claims, which could well override whatever position the AG decides to take. In my personal opinion, and based on anecdotal impression, this is what has been happening regarding annulment procedures promoted by unprivileged applicants (with all the issues that the Plaumann, UPA, Inuit, saga have created; see here).
In the end, the difficulty I have is that their results do not necessarily make a lot of intuitive sense because they cannot (or at least not immediately) be interpreted regarding inverse AG recommendations (ie recommendations to inadmit/dismiss) and their effect on the CJEU. Somehow, there seems to be an implicit assumption that 'influence' of the AG is stronger if it prompts the CJEU to annul than if it prompts the CJEU to inadmit/dismiss. If all of this is incorrect, then my only residual criticism is that the paper could have been made more accessible for non-statisticians.
From the numbers in the paper, I have been unable to work out the effect that an AG recommendation to inadmit/dismiss has on the CJEU's willingness to do so. Intuitively, I would expect that, if by itself the Opinion of the AG is such a relevant factor as the paper claims, then the CJEU should also be more inclined to inadmit/dismiss when the AG submitted such a recommendation. However, in that case, I would not necessarily find the causal explanation between the AG recommendation and the CJEU's decision persuasive. An alternative interpretation not linked to the influence of the AG over the CJEU would need to be dispelled: ie the zeal with which the CJEU keeps control of its docket. The intuition would be that the CJEU may be engaged in an interpretation of inadmissibility rules that prevents a floodgate of claims, which could well override whatever position the AG decides to take. In my personal opinion, and based on anecdotal impression, this is what has been happening regarding annulment procedures promoted by unprivileged applicants (with all the issues that the Plaumann, UPA, Inuit, saga have created; see here).
In the end, the difficulty I have is that their results do not necessarily make a lot of intuitive sense because they cannot (or at least not immediately) be interpreted regarding inverse AG recommendations (ie recommendations to inadmit/dismiss) and their effect on the CJEU. Somehow, there seems to be an implicit assumption that 'influence' of the AG is stronger if it prompts the CJEU to annul than if it prompts the CJEU to inadmit/dismiss. If all of this is incorrect, then my only residual criticism is that the paper could have been made more accessible for non-statisticians.
Conclusion
Overall, I remain unconvinced that the results of Arrebola et al significantly contribute 'to a more comprehensive understanding of the role of the Advocate General in the makeup of the Court of Justice of the European Union'. Thus, I am not prepared to engage with the implications in terms of judicial independence and potential (further) reform of the CJEU that they draw (pp. 34-38). Given the disagreement with their methodology and the diversity of views as to how to interpret their results, I have contacted Carlos Arrebola and offered him to reply to my criticisms in a guest post. He has kindly accepted. Keep an eye out for it in the coming days.
(*) The full reference for the paper is: C Arrebola, AJ Mauricio and H Jiménez Portilla, 'An Econometric Analysis of the Influence of the Advocate General on the Court of Justice of the European Union' (January 12, 2016). Cambridge Journal of Comparative and International Law, Vol. 5, No. 1, Forthcoming; University of Cambridge Faculty of Law Research Paper No. 3/2016. Available at SSRN: http://ssrn.com/abstract=2714259.