Report Reflections, Part 1: General Thoughts

As previously noted, I was honored to be part of the ten-person team commissioned by the chancellors of Vanderbilt University and Washington University in St. Louis to inquire into the state of scholarly standards in the humanities and humanistic social sciences.

In this post, I’ll share some reflections on the commission, summarize the report, provide my takeaways from my work on the commission as well as some limitations with the work and the report. In a second post, I’ll focus on the types of evidence I used for the sociology portion of our analysis, which will also illustrate some of the themes I discuss here. Before I do, let me say: please read the report. No summary or meta-commentary will do it justice.

The Commission

Vandy and WashU commissioned us to inquire into the state of the humanities and humanistic social sciences, especially looking at the extent to which internal politicization was affecting the state of scholarly standards. We examined anthropology, history, literary studies, musicology, philosophy, and sociology; I was in charge of conducting the research on sociology and I’ll separately publish a report on the basis of my research for the commission next month.

State of Scholarship Report Summary

The report takes a middle ground on the question we were asked: every field has significant problems, but no field is entirely lost. As such, things are neither as bad as some critics say, nor as good as some defenders say. My gloss: There are problems, and we need to acknowledge them—and hopefully address them.

The report takes great pains to avoid being misread (although, as expected, it is being misread by academia’s defenders and critics alike). It emphasizes that the humanities are valuable and should remain part of universities. It also emphasizes that if administrators are concerned, they should do their own careful self-studies (involving faculty) rather than rely on a report based on national trends and basically a bird’s-eye view: problems at the field level are not necessarily problems in your local department. The report acknowledges that politicization from outside of academia (typically from conservative politicians) also happens and is a big problem for academic standards and freedom, but our focus is on politicization from within. Finally, the report spends a lot of time explaining what isn’t the problem: liberal professors, scholars who are activists on their own time or who personally have social justice motivations and agendas, an expanded canon to cover previously underrepresented topics and people, and critical scholarship generally. It’s fine—good, even—that all* of that exists and a necessary consequence of an open approach to scholarship.

*The report notes the liberal skew of the professoriate is itself not the problem; the problem is replacing scholarly standards with political/ideological ones. Hypothetically, even with this skew, if we all stuck to scholarly standards for admissions, hiring, tenure/promotion, publishing, awards, etc., we would not have to write this report. My own sense is that neither would such skew exist as strongly as it does if we stuck to scholarly standards, instead of incentivizing subpar but politically satisfying work. This is why I emphasize, when asked, that university officials and researchers in evaluative positions (hiring, T&P, awards committees) need to take their thumbs off the scale and stop rewarding work more for its politics than for its scholarly quality. (Jesse Smith writes wonderfully about why I’m wrong. He very well may be right. We may both be right. It may be there are different formulas for reaching a better future. I hope we get a chance to evaluate these speculations in practice in real-world settings.)

Instead, the report highlights several problematic themes we saw across the disciplines—we summarize them as ways in which scholarly standards of knowledge creation and dissemination are being subordinated to political goals and values. The themes are:

1. scholarship being redefined to serve political ends or disciplines adopting political missions rather than knowledge-based missions;

2. fields that treat some answers as settled, or some findings as forbidden, or some topics as unacceptable to analyze such that we start to get censorship or ostracization for some otherwise fine work;

3. the postmodernist or relativist rejection of objectivity and a knowable reality (more than acknowledging challenges to objectivity or the knowable reality and trying to confront our limitations, but rejecting such efforts as undesirable or impossible);

4. bad (incoherent, unintelligible) writing in the style of some twentieth-century French philosophers who (and whose style) became influential across the humanities and social sciences over the last several decades.

While we were charged with looking at the humanities and humanistic social sciences, the issues we discuss are not unique to these fields and are even showing up in STEM fields and STEM-adjacent fields like psychology. (Speaking from my own perspective, this warning is consistent with my reading of both high-profile episodes in various STEM fields and a growing number of studies on the same issues, especially in psychology. These issues come about both from the top-down (e.g., institutionalized norms at the level of publishing or professional associations) and bottom-up (e.g., grass-roots efforts to change the field); they meet in the middle, the average researcher (where most academic research is produced), but that layer is less visible or analyzed, and the size of the portion who are essentially unaffected is unknown.) Additionally, we suspect that these issues are bigger in interdisciplinary fields in the humanities and social sciences, which we did not explore systematically. (Areas for future research!)

How did we come to our conclusions? We don’t talk about that in the report very much—it’s a very high-level report written for high-level administrators rather than for our colleagues. As we note, we wrote internal reports for each discipline. I’ll be making a revised version of my report public in July. Since my colleagues will want to know what evidence we reviewed before then, in another post, I’ll describe what evidence I looked at (and then my more specific findings will come in my official report). But first…

My Takeaways from the Commission

Just to be clear, I am no longer summarizing the Report, but my gloss on it. What I take away or emphasize is not necessarily what other commissioners would emphasize.

1. Whatever else we do in this report, it represents an interdisciplinary group of scholars standing up for scholarly rigor and not letting political goals corrupt the research enterprise or the standards by which research is evaluated. This alone is a powerful statement. I’m sure it will be criticized by those who disagree that the pursuit of knowledge should not take a backseat to the pursuit of political goals (I wrote this last week; it’s already happening and it’s providing further evidence about the problems we are facing—the personal attacks simply for writing a very tepid, careful report that are quite something). But personally, I’m thrilled to be part of such an impressive, cross-disciplinary group of scholars making a strong endorsement of basic, fundamental standards of scholarship.

2. As we note in the report, although all of the disciplines examined were affected to some degree, the disciplines varied in how much internal politicization has affected their scholarly standards. One big caveat: our goal was not to systematically compare the disciplines against each other to see which was worse off. Moreover, we used evidence that makes sense for each field, some of which overlaps across each field and some doesn’t. That said, it was pretty clear that Anthropology and Sociology were at one extreme (heavy politicization), while (analytic) Philosophy was at the other (little politicization), with literary studies/English and History were in the middle. I was pleasantly surprised, since my stereotype was that the humanities were far worse off than the social sciences in this respect, but on many measures, that did not appear to be the case. Indeed, sociology and anthropology were worse than I expected. So the findings were actually quite surprising to me.

3. There’s also variation internal to each discipline. As we say in the report, there is good research in every field. But fields are incredibly heterogeneous. From my perspective, two findings on this front were particularly interesting:

a. First, there’s significant variation over time—some disciplines had their fight with these issues years ago and seem to be doing okay now, while others are still ramping up with no clear sign of slowing. Again, this was really surprising to me: I thought the humanities were just entirely cooked. They aren’t. The humanities are actually doing pretty well these days. A lot of the worst examples from the “humanities” are coming from interdisciplinary scholars and fields in the humanities, but not the core disciplines, nor are they taken seriously by serious scholars in the disciplines. Essentially, the reputational damage to the humanities is mostly coming from outside the core disciplines that get most of the blame. Sociology and anthropology, by contrast, appear far worse off. In sociology (and I expect a number of interdisciplinary fields), we are inheriting a lot of work from decades ago that has only recently gone mainstream in our fields. What we don’t realize is that this stuff that’s getting imported isn’t respected in what we think of as their home fields and/or we don’t see how adaptations in our fields are missing important nuances that entirely change their utility in ours.

b. Second, even within a given field, the answer to the question of how much politicization affects scholarly standards massively depends on which metric you look at. This is a really important point (and one I’ll say more about later). The most superficial markers tend to show the most politicization—conference themes, presidential addresses, high-profile scandals. Actually evaluating the quality of research produced in every field (as I’ve written here before) is difficult and there are not a lot of existing studies on this question, at least in sociology. (This isn’t surprising: it’s difficult to do, it is not incentivized, and it is badly discouraged.) The easiest analyses to run tend to be a bit superficial: variation in what topics are explored and what words are used in the work. These can show topical and political skew (which are important), but not necessarily quality issues—there may just be a lot of good work that uses particular words associated with politicization. In reality, the topical and word choices are correlated with the sorts of things we were investigating, but we can’t actually say more than that (although new LLM-based analyses keep coming out—even though I’m generally open to the approach, I was really skeptical of some of the specific efforts, but after grilling folks doing the analyses, I was convinced). Additionally, we were also limited in our investigation since some things we wanted to know just aren’t well documented (e.g., tenure and promotion cases, rates of self-censorship, etc.). Finally, some fields have survey research on relevant attitudes and behaviors, which is great because it’s one of the few measures that gives us systematic data on what we really want to know, but their availability varies by field, and you still have to worry about response rates and representativeness. So, evidence on the central questions is mixed in its availability and has different limitations, but as I’ve said before, you need to triangulate, and if you do, you do see some differences in your results and some patterns across them. The hardest question is figuring out the extent or depth of these trends in part because of the variation.

So it’s important to understand that fields are heterogeneous in multiple ways. So when I say fields can be arrayed on a spectrum from how much politicization has affected their quality, it’s based on a number of different metrics overall, rather than any one marker.

I tend to think this intra-field heterogeneity is one reason why we can have such radically different views and experiences of our fields, even among those of us who think our field needs to be reformed. The other reason for such radically different views is: a lot of folks simply aren’t bothered by politicized research and don’t agree that certain scholarly standards are important. That’s a bigger problem.

Those were my biggest general takeaways. Again, in a future post, I’ll talk about sociology specifically. I’m also working on a series of responses to questions I’ve received that I found intriguing and want to talk about. So stay tuned!