An artificial intelligence method developed by University College London computer scientists and associates has predicted the judicial decisions of the European Court of Human Rights (ECtHR) with 79% accuracy, according to a paper published Monday, Oct. 24 in PeerJ Computer Science.

The method is the first to predict the outcomes of a major international court by automatically analyzing case text using a machine-learning algorithm.*

“We don’t see AI replacing judges or lawyers,” said Nikolaos Aletras, who led the study at UCL Computer Science, “but we think they’d find it useful for rapidly identifying patterns in cases that lead to certain outcomes. It could also be a valuable tool for highlighting which cases are most likely to be violations of the European Convention on Human Rights.”

Judgments correlated with facts rather than legal arguments

In developing the method, the team found that judgments by the ECtHR are highly correlated to non-legal (real-world) facts, rather than direct legal arguments, suggesting that judges of the Court are, in the jargon of legal theory, “realists” rather than “formalists.”

This supports findings from previous studies of the decision-making processes of other high level courts, including the U.S. Supreme Court.

The team of computer and legal scientists extracted case information published by the ECtHR in its publically accessible database (applications made to the court were not available), explained UCL co-author Vasileios Lampos, PhD.

They identified English-language data sets for 584 cases relating to Articles 3, 6 and 8** of the Convention and applied an AI algorithm to find patterns in the text. To prevent bias and mislearning, they selected an equal number of violation and non-violation cases.

Predictions based of analysis of text

The most reliable factors for predicting the court’s decision were found to be the language used as well as the topics and circumstances mentioned in the case text (the “circumstances” section of the text includes information about the case factual background). By combining the information extracted from the abstract “topics” that the cases cover and “circumstances” across data for all three articles, an accuracy of 79% was achieved.

“Previous studies have predicted outcomes based on the nature of the crime, or the policy position of each judge, so this is the first time judgments have been predicted using analysis of text prepared by the court. We expect this sort of tool would improve efficiencies of high level, in demand courts, but to become a reality, we need to test it against more articles and the case data submitted to the court,” added Lampos.

Researchers at the University of Sheffield and the University of Pennsylvania where also involved in the study.

* “We define the problem of the ECtHR case prediction as a binary classification task. We utilise textual features, i.e., N-grams and topics, to train Support Vector Machine (SVM) classifiers. We apply a linear kernel function that facilitates the interpretation of models in a straightforward manner.” — Authors of PeerJ Computer Science paper.

** Article 3 prohibits torture and inhuman and degrading treatment (250 cases); Article 6 protects the right to a fair trial (80 cases); and Article 8 provides a right to respect for one’s “private and family life, his home and his correspondence” (254 cases).

---

Abstract of Predicting Judicial Decisions of the European Court of Human Rights: A Natural Language Processing Perspective

Recent advances in Natural Language Processing and Machine Learning provide us with the tools to build predictive models that can be used to unveil patterns driving judicial decisions. This can be useful, for both lawyers and judges, as an assisting tool to rapidly identify cases and extract patterns which lead to certain decisions. This paper presents the first systematic study on predicting the outcome of cases tried by the European Court of Human Rights based solely on textual content. We formulate a binary classification task where the input of our classifiers is the textual content extracted from a case and the target output is the actual judgment as to whether there has been a violation of an article of the convention of human rights. Textual information is represented using contiguous word sequences, i.e. N-grams, and topics. Our models can predict the court’s decisions with a strong accuracy (79% on average). Our empirical analysis indicates that the formal facts of a case are the most important predictive factor. This is consistent with the theory of legal realism suggesting that judicial decision-making is significantly affected by the stimulus of the facts. We also observe that the topical content of a case is another important feature in this classification task and explore this relationship further by conducting a qualitative analysis.