It’s the stuff of science fiction: an artificial intelligence judges the merits of a court case and reaches a verdict. We are now one step closer to that being a reality. Researchers have created an artificial intelligence system that has accurately predicted the outcomes of many cases heard at the European Court of Human Rights (ECHR). Out of 584 cases, the system had a success rate of 79% where it correctly decided whether there had been a Human Rights’ violation or not.
The project was carried out by researchers at University College London and the universities of Sheffield and Pennsylvania. The method consisted of automatically analyzing case text using a machine learning algorithm. For the learning phase, equal amounts of cases with violations and without violations were used. In developing the method, the team found that judgements by the ECHR are highly correlated to non-legal facts rather than directly legal arguments, suggesting that judges of the Court are, in the jargon of legal theory, ‘realists’ rather than ‘formalists’.
The most reliable factors for predicting the court’s decision were found to be the language used, as well as the topics and ‘circumstances’ mentioned in the case text. The ‘circumstances’ section of the text includes information about the factual background to the case.
The team identified English language data sets for 584 cases related to three articles of the Convention on Human Rights:
- Article 3: cases involving torture or degrading treatment, good for 250 cases
- Article 6: rights to a fair trial, good for 80 cases
- Article 8: respect for private life, good for 254 cases
By combining the information extracted from a) the abstract ‘topics’ that the cases cover and b) the ‘circumstances’ across data for all three articles, an accuracy of 79% was achieved.
The 21% of cases where the prediction was not correct involved situations where there were similar cases but with different outcomes. This could be an indication that in those cases, the finer subtleties of the law were missed by the artificial intelligence.
One of the key researchers, Dr Nikolaos Aletras, who led the study at UCL, said the following about using AI to predict cases: ‘We don’t see AI replacing judges or lawyers, but we think they’d find it useful for rapidly identifying patterns in cases that lead to certain outcomes. It could also be a valuable tool for highlighting which cases are most likely to be violations of the European Convention on Human Rights.’
Given the huge numbers of cases that the ECHR’s legal staff has to consider every year, the program could indeed prove very useful: in 2015, e.g., the court had to process 40,650 applications for a hearing, while in 2014 it processed 56,200 cases.
It is easy to see how such programs could benefit law firms as well. Having intelligent software at your office to predict the outcome of a case would offer a clear and objective view of the strengths and weaknesses of your case and argumentation. This could then help devise a different argumentation if necessary. Alternatively, it could help avoiding the costs of litigation if similar cases were not considered ‘winnable’.
Does this mean lawyers or judges should start fearing for their jobs? No, it doesn’t. The program still misses the finer subtleties of the law. As Dr Aletras pointed out, it is meant as a tool to assist, not to replace. In the long run, though, it could result in lawyers and judges dealing with more complex or specialized cases, while the processing of simpler cases gets more automated.
Sources:
- ucl.ac.uk/news/news-articles/1016/241016-AI-predicts-outcomes-human-rights-trials
- bbc.com/news/technology-37727387
- artificiallawyer.com/2016/10/24/academics-build-ai-to-predict-human-rights-case-outcomes/