Incorporating Systematic Sources of Knowledge into the Social Sciences and the Law
Short Definition
Many models define wisdom as the ability to draw concise, accurate conclusions from large bodies of information. This project begins to develop rigorous systematic reviews to help the social sciences extract such conclusions from growing masses of empirical claims, and examines how to introduce these reviews into legal settings. This project consists of two goals. First, it seeks to fundamentally improve how empirical disciplines that rely on observational data determine what they know.
Summary Points
- Many models define wisdom as the ability to draw concise, accurate conclusions from large bodies of information. This project begins to develop rigorous systematic reviews to help the social sciences extract such conclusions from growing masses of empirical claims, and examines how to introduce these reviews into legal settings.
- This project consists of two goals. First, it seeks to fundamentally improve how empirical disciplines that rely on observational data determine what they know.
- This part of the project will produce two papers and a conference, which are analyzed in the article.
- The second goal is to develop ways to incorporate such reviews into an adversarial legal system. This part of the project will produce two papers over the period of the grant. , which are analyzed in the article.
- This project begins a larger effort to reform how empirical disciplines and the law determine knowledge.
Text from Wisdom Institute
Many models define wisdom as the ability to draw concise, accurate conclusions from large bodies of information. This project begins to develop rigorous systematic reviews to help the social sciences extract such conclusions from growing masses of empirical claims, and examines how to introduce these reviews into legal settings. Preliminary conclusions indicate that the methodological plurality of social science research makes such reviews challenging to develop, but that history and doctrine suggest they can be incorporated into legal disputes. Future work will continue this project and examine how to reduce the complexity of systematic reviews without weakening their reliability.
Incorporating Systematic Sources of Knowledge into the Social Sciences and the Law
This project consists of two goals. First, it seeks to fundamentally improve how empirical disciplines that rely on observational data determine what they know. It focuses on developing evidence-based systematic reviews for observational work. Such reviews provide substantially more transparent and objective assessments of knowledge than their alternatives, but they are unused in most empirical fields. This part of the project will produce two papers and a conference. The first paper, aimed at legal empiricists, will point out the shortcomings of current methods of assessment and examine the theoretical benefits and practical concerns of systematic reviews. The second will generate such a review, thus demonstrating its feasibility, using empirical work on incarceration’s effect on crime as an illustrative example. This part of the project will culminate with a roundtable conference bringing together researchers from several disciplines who work on evidence synthesis. The conference will establish general protocols for constructing evidence-based systematic reviews of observational studies, and it will lay the groundwork for a more formal institution to advance their use and development in empirical fields more generally. The second goal is to develop ways to incorporate such reviews into an adversarial legal system. Well-designed systematic reviews emphasize collaboration among independent analysts; the legal system, competition between partisan experts. This part of the project will produce two papers over the period of the grant. The first, a companion paper to the theoretical piece mentioned above, will be aimed at the legal audience, pointing out the epistemic superiority of systematic reviews while discussing legal issues they raise. The second paper will address concerns raised by the fact that legal and scientific fact-finding will never be identical. In particular, it will examine how to reconcile independent systematic reviews with adversarialism. Though lauded for its ability to ferret out knowledge, adversarialism also advances normative goals other than fact-finding, and it is essential to determine ways to incorporate systematic reviews while honoring these values. This article will also focus on pragmatic issues of implementation, such as funding and the selection of reviewers. This project begins a larger effort to reform how empirical disciplines and the law determine knowledge. The current, inferior methods are firmly entrenched in both law and the sciences. Displacing and reforming them will require the substantial efforts of numerous actors in both areas, but the payoff will be a profoundly deeper understanding of what we in fact know. The first paper under this grant is complete, and will soon be submitted to a peer-reviewed law journal by the end of summer 2009. This paper lays out the basic problems with how the social sciences currently produce knowledge, explains the key benefits of a systematic approach, and points out the key challenges of adopting such methods in the social sciences. The groundwork for two more papers is underway. The second paper begins to consider how to incorporate systematic reviews into the US legal system, and the third starts to develop prototype quality guidelines for producing systematic reviews of observational studies. An early draft of the legal paper was presented at a faculty workshop at Fordham in May.
https://wisdomcenter.uchicago.edu/about/project-1-defining-wisdom