The Supreme Court Project attempts to build on and extend empirical work explaining the United States Supreme Court’s decision making and its review of the 13 United States Courts of Appeals. We have started our inquiry by looking at the question of how to measure the Supreme Court’s reversals of the United States Courts of Appeals (Phase I) and expect to continue our analyses by building empirical models explaining the variation in reversal rates across the circuits and predicting Supreme Court Justice voting and Court decisions (Phase II).
Initially, the Project examines whether there is a better way to track how well the Courts of Appeals do before the Supreme Court, conventionally measured through reversal and affirmance rates. Traditionally, the reversal rate for a circuit court is an outcome-based, score-card test: For any given term of the Supreme Court, a reversal rate can be established by dividing the number of appeals where the Court reversed the circuit by the total number of appeals decided by the Supreme Court for that circuit. While straightforward, this traditional method suffers from two fundamental weaknesses: (1) it measures outcomes rather than legal reasoning; and (2) it fails to account for decisions implicitly reversed or affirmed by the Supreme Court, even though not directly before the Supreme Court on appeal.
Rule 10(a) of the Supreme Court Rules explains that a reason for granting certiorari is that a Court of Appeals “has entered a decision in conflict with the decisions of another United States court of appeals.” Our analysis confirms this as, for the six terms of the Roberts Court (2005-2011), nearly half of the Supreme Court’s annual merit decisions address substantive legal issues for which there is a circuit split. For each of these cases, the Supreme Court ruled on a legal issue addressed not just in the case on appeal, but also one or more “shadow decisions” (i.e., court of appeals decisions that have ruled on a legal issue that comprises the circuit split). Any measure of reversals and affirmances that does not account for these shadow decisions is therefore incomplete and potentially misleading.
We, therefore, have established a “full” measure of the circuit courts’ reversal and affirmance rates, taking into account both decisions on direct appeal and these shadow decisions. This measure, we believe, provides a more robust and more accurate view of the relationship between the United States Courts of Appeals and the United States Supreme Court.
We discuss our preliminary results in an article, “Towards a Better Measure and Understanding of U.S. Supreme Court Review of Courts of Appeals Decisions,” published in BNA Weekly. Generally, we conclude:
- the Supreme Court reverses the Courts of Appeals less often than is commonly thought;
- the full measure of reversals identifies different Courts of Appeals as least reversed than those identified under the traditional method;
- in resolving circuit splits, the Supreme Court affirms the majority approach of the circuit approximately 50% of the time;
- our analysis permits us to construct a concordance table showing the degree to which the various Courts of Appeals agree with each other, much like the concordance tables showing agreement among justices’ voting.
Overall, we believe this analysis, which takes into account both cases directly on appeal and shadow decisions, provides a more robust and more accurate view of the Supreme Court’s review of the Courts of Appeals.
For those interested in more detailed Tables and Charts regarding this analysis, they are available here.
While our Phase I work identified variations in circuit court reversal rates, it did not attempt to explain the reasons for variations. In Phase II we will apply econometric techniques to test various hypotheses that explain these variations.
One hypothesis, for example, is that the political party (or appointing president) of a justice and the members of a circuit court panel may, in part, explain the justice’s decision to affirm or reverse; a justice and a judge appointed by the same president or a president from the same party may, all else equal, lead to a lower probability of reversal. Other hypotheses include: whether a justice is less likely to reverse (1) a court of appeals over which s/he is the circuit justice; (2) a judge who attended the same law school as the justice, or (3) a judge who is the same gender as the justice.
Clearly, a few caveats are in order. First, a statistical correlation is only just that and needs to be supplemented by an underlying rationale; a statistical finding that left-handed justices reverse left-handed court of appeals judges more often than right-handed ones, for example, would likely be spurious. Nevertheless, data driven inquiries such as these (and others) are important in revealing patterns that actually occur. We welcome interested readers to email us with suggestions of plausible theories that could be tested with available data.
Second, the tests of these hypotheses should be part of a broader statistical model of Supreme Court Justice decision making. In that way, we anticipate that this work will build on the pioneering work of, among others, the Washington University Supreme Court Forecasting Project, which applied statistical techniques to predict Supreme Court Justice voting behavior and court decisions. We hope to extend that work by, among other things, updating their results for the Roberts Court and employing additional explanatory variables (including those relating to the Courts of Appeals).
- “A Sixth Sense: Sixth Circuit has Surpassed the Ninth as the Most Reversed Appeals Court,” ABA Journal, December 2012 (written by Mark Walsh) View Article »
- “The ‘Full’ Method of Measuring the Court’s Review of Decisions by the Courts of Appeals,” SCOTUSblog, October 23, 2012 View Article »
- “Towards a Better Measure and Understanding of US Supreme Court Review of Courts of Appeals Decisions,” BNA’s The United States Law Week, September 27, 2011 View Report »
- “The Third Circuit’s Reversal Rate: A Success Story,” The Legal Intelligencer, November 10, 2011 View Article »
- “First Circuit Reversal Rate Not What It Seemed,” Massachusetts Lawyers Weekly, January 30, 2012 View Article »
- John S. Summers is a shareholder at Hangley Aronchick Segal Pudlin & Schiller. He received a BA from Wesleyan University in 1980 and a JD from the University of Pennsylvania in 1984.
- Michael J. Newman is a former Hangley Aronchick Segal Pudlin & Schiller associate. He received a BA from the University of Pennsylvania in 2002 and a JD from Columbia Law School in 2006. He is a member of the Supreme Court of the United States Bar.
- Michael Cliff is Vice President at Analysis Group. He has a BS from Virginia Tech and a PhD in finance from the University of North Carolina at Chapel Hill.
- David Klein is an Analyst at Analysis Group. He has a BA in Economics and Political Economy from Washington University in St. Louis.
- Sharon Weiss is an assistant at Hangley Aronchick Segal Pudlin & Schiller.
- Danielle Acker Susanj earned her law degree from the University of Pennsylvania Law School in 2013. She attended Wheaton College and earned a BA in history.
- Jonathan Conigliari was also a member of the University of Pennsylvania Law School Class of 2013. He received a BA in French and an MA in French studies from New York University.
- Sarah Gignoux-Wolfsohn is working toward her PhD at Northeastern University. She earned her BA in biology and French, with honors in biology, from Wesleyan University.
- Gabrielle J. Niu earned her MA in East Asian languages and cultures from the University of Pennsylvania in 2012. She is a graduate of Bowdoin College, where she majored in Asian studies and minored in chemistry.
- Ben Jackal is a member of Temple University School of Law Class of 2014. He attended Cornell University and earned a BA in history.
- Colleen Daniels graduated in 2015 from the University of the Arts in Philadelphia, where she was a Crafts major with a concentration in woodworking. She also had a minor in creative writing.
- Ellen Boyer graduated from Temple University in 2013 with a Bachelors in English and Political Science.
- David Huppert is a 2015 graduate of George Washington University, where he earned a BS in Economics and a minor in English.