Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2017 Apr 12;12(4):e0174698.
doi: 10.1371/journal.pone.0174698. eCollection 2017.

A general approach for predicting the behavior of the Supreme Court of the United States

Affiliations

A general approach for predicting the behavior of the Supreme Court of the United States

Daniel Martin Katz et al. PLoS One. .

Abstract

Building on developments in machine learning and prior work in the science of judicial prediction, we construct a model designed to predict the behavior of the Supreme Court of the United States in a generalized, out-of-sample context. To do so, we develop a time-evolving random forest classifier that leverages unique feature engineering to predict more than 240,000 justice votes and 28,000 cases outcomes over nearly two centuries (1816-2015). Using only data available prior to decision, our model outperforms null (baseline) models at both the justice and case level under both parametric and non-parametric tests. Over nearly two centuries, we achieve 70.2% accuracy at the case outcome level and 71.9% at the justice vote level. More recently, over the past century, we outperform an in-sample optimized null model by nearly 5%. Our performance is consistent with, and improves on the general level of prediction demonstrated by prior work; however, our model is distinctive because it can be applied out-of-sample to the entire past and future of the Court, not a single term. Our results represent an important advance for the science of quantitative legal prediction and portend a range of other potential applications.

PubMed Disclaimer

Conflict of interest statement

Competing Interests: All Authors are Members of a LexPredict, LLC which provides consulting services to various legal industry stakeholders. We received no financial contributions from LexPredict or anyone else for this paper. This does not alter our adherence to PLOS ONE policies on sharing data and materials.

Figures

Fig 1
Fig 1. Case and justice accuracy 1816-2015 (by term).
Time series of the accuracy of our prediction model at both the case level (left pane) and justice level (right pane).
Fig 2
Fig 2. Reversal rate by decade.
For most of the Court’s history, Reversal was much less frequent than it is now. Only in recent history has Reversal become the more common outcome.
Fig 3
Fig 3. Case and justice accuracy compared against null models.
The first row corresponds to M = 10, the second row corresponds to M = ∞, and the third row corresponds to always guess Reverse. The left column corresponds to case accuracy, and the right column corresponds to justice accuracy. When our model outperforms the baseline, the plot is shaded green; when it fails to exceed the baseline performance, the plot is shaded red.
Fig 4
Fig 4. Cumulative number of terms won versus M = 10 null model.
Fig 5
Fig 5. Justice-term accuracy heatmap compared against M = 10 null model (1915 -2015).
Green cells indicate that our model outperformed the baseline for a given Justice in a given term. Pink cells indicate that our model only matched or underperformed the baseline. The deeper the color green or pink, the better or worse, respectively, our model performed relative to the M = 10 baseline.

References

    1. Martin AD, Quinn KM, Ruger TW, Kim PT. Competing approaches to predicting supreme court decision making. Perspectives on Politics, 2004; 2(4):761–767. 10.1017/S1537592704040502 - DOI
    1. Guimerà R., Sales-Pardo M. Justice blocks and predictability of us supreme court votes. PloS One, 2011; 6(11):e27188 10.1371/journal.pone.0027188 - DOI - PMC - PubMed
    1. Ruger TW, Kim PT, Martin AD, Quinn KM. The supreme court forecasting project: Legal and political science approaches to predicting supreme court decisionmaking. Columbia Law Review, 2004; 104(4):1150–1209. 10.2307/4099370 - DOI
    1. Breiman L. Random forests. Machine Learning, 2001; 45(1):5–32. 10.1023/A:1010933404324 - DOI
    1. Katz DM. Quantitative legal prediction—or—how i learned to stop worrying and start preparing for the data driven future of the legal services industry. Emory Law Journal, 2013; 62(4): 909–966.

LinkOut - more resources