Sokal on steroids: Why hoax papers submitted successfully to academic journals proves nothing (ok, little)

A trio of authors has, during a 12 months period, submitted - by their own account - 20 manuscripts to academic journals they broadly identify as being in disciplines or fields of study associated with 'academic grievance studies'. From what I gather they are not too terribly fond of social constructionist colleagues in their own academic disciplines, and presumably other disciplines. Let me say at the outset that I am sympathetic to some of their concerns. They rightly lament that what they broadly label as 'academic grievance studies' has led to thought crimes prosecutions of sorts (just think about the Hypatia controversy involving an article by Rebecca Tuvel - mob justice might be one way to describe what happened to Tuvel). Ironically, Hypatia is again among the offending journals, which is interesting in so far as it isn't a journal entirely dedicated to publishing social constructivist content, another bugbear of the initiators of the Sokal-on-steroids hoax.

In any case, they apparently spend their while producing 20 hoax articles that they planned to submit to top journals in different disciplines or areas of study that they find disagreeable. I am genuinely curious whether this research project was submitted to their institution's ethical review committee, because it uncontroversially involved involuntary human research participants (identifiable journal editors, reviewers).

Here's the result of their efforts: Of these 20 papers 7 were accepted, 6 were rejected outright, and the rest were somewhat in-play, including a number that received a revise and resubmit verdict. The hoax initiators claim that two papers received a verdict of 'revise and resubmit' that they took to mean 'usually results in publication.' For what it's worth, at the journals I co-edit this verdict means renewed external review with the possibility of an outright rejection. So, no, not 'usually results in publication.' I wonder whether this interpretation is self-serving in so far as they needed to, of course, bolster their case as good as they could.

At the end of the day, they had 20 submissions, 7 acceptances.

I agree with the team of hoaxters that this constitutes egg on the faces of the editors of those journals, and more so, on their reviewers' faces. It's embarrassing.

I read a few comments on this project along the lines that in STEM subjects journals also face large numbers of retractions each year, and so it's not surprising that terrible content also passes peer review in humanities' areas. While that is true, it shouldn't distract from the fact that well-established, influential journals were coaxed here into accepting garbage. Of course, that matters!

My problems really lie elsewhere. One is that a study where n=20 doesn't demonstrate that a particular discipline has methodological problems. It simply means that - when all is said and done -  7 crappy papers were accepted by non-specialist journal editors based on their reviewers' recommendations. Big whoop! Frankly, this shows us that on this occasion reviewers failed. On 7(!) occasions. We do not know whether the same would have transpired if hundreds of such papers had been submitted to the same journals. It's one such paper per journal. Talking anecdotal, this is as anecdotal as it gets.

Let me be honest here, as an editor of journals that rarely if ever publish social constructionist papers, I could also be accepting papers that succeeded in fooling our peer reviewers. I don't have the subject expertise to be confident in evaluating all the manuscripts that are submitted to my journals. I rely on - these days - mostly reluctant reviewers who hopefully provide me with good (sometimes excellent, detailed) comments and recommendations. I rely on competent reviewers being diligent. I rely on colleagues I ask to review a manuscript to come back to me if they do not consider themselves competent. I rely on unpaid reviewers spending a considerable amount of time doing their job, when they could spend that time writing their own papers, or on a grant application, or a job application, or they could simply spend time with their loved ones. Not all of them, all the time, deliver a reliable review. Big whoop. If a faulty paper gets published (and it hasn't happened to us yet), I'm confident, over time responses to such a paper would eventually show where the paper we published went wrong, and why. A case in point is this Editorial I wrote. One of the articles flagged there was making false empirical claims, it has since been retracted. The article in question passed external peer review, but it is clear that the reviewers took the empirical claims made by the authors to be true, referenced as they were. It turns out that that was a mistake. 

All of that happens without hoaxters wasting my time and that of our reviewers.

The hoaxter trio takes these 7 accepted manuscripts as evidence for the methodological failings of what they refer to as 'social constructivism'or 'radical constructivism'. They apparently 'corrupt' scholarship.

The thing is, that might well be true, and social constructivism scholarship is truly a naked emperor. Unfortunately, pointing to 7 anecdotal papers as evidence that that is the case, is plain ludicrous. As far as I can see, there wasn't even a control group (say, 20 manuscripts submitted to analytical journals, 20 manuscripts submitted to STEM subjects).

To my mind, if you wish to criticize social constructivism (and much critical that I wholeheartedly agree with has been published over the last few decades) don't avail yourself of childish activities like these kinds of hoaxes. Show by means of analysis and argument that, and why, the social constructivism emperor is naked.

The process of producing and publishing peer reviewed academic content relies on an assumption of good intent and genuineness among authors. Obviously, this is not justified in all cases, that's where critical responses and retractions come in handy. The last thing needed is a cottage industry of 'gotcha authors' like our hoaxter trio. They could and should have spend their time producing one sound academic paper taking on social constructivism, placing it in a top-notch journal, and subsequently enjoying the fireworks of rebuttal and response. That's how progress in the academy is facilitated.

I have sat on this for a good week, because I wasn't sure whether I should bother writing a response. On the one hand I share many of their concerns, on the other hand, this was such a time wasting pointless exercise, it boggles the mind smart people would have resorted to that sort of thing.

New York, New York

I am currently attending the American Philosophical Association's Eastern Division's meeting in New York. I am not exactly what you'd call an avid conference goer, so - believe it or not - this is my first philosophy conference. We got an unwieldy conference program (a book of some 200pp). The variety of what's on offer is truly impressive. To my surprise, more than a fair number of contributors to our recent volume 50 Voices of Disbelief are presenting here, including Philip Kitcher and John Schellenberg. I went to a panel yesterday, chaired by my good friend Rosamond Rhodes, on health care reform in the US and heard Norm Daniels as well as the Hastings Center's Daniel Callahan speak. Impressive people. Daniels, in response to a question from an audience member, replied that the health care reform package should not be permitted to fail over the troublesome exclusion of abortion related medical services. I wondered about this very same question. Clearly what happened there is appalling, but within the political constraints of the votes available to progressive agendas in the US Senate, what's currently on the table probably is the best that can be done. To argue that it's better not to achieve small progress in order to avoid the political sell-out on the abortion front, seems short-sighted. Daniels called that one right, I think. Callahan wrote a thoughtful, critical commentary recently on the Hastings Center's website that is quite critical of the liberal embrace of stem cell research. Check it out if you feel like. I also attended a meeting with fellow journal editors, primarily aimed at an exchange of views. I was pleased to learn that Hypatia, the feminist philosophy journal moved over to Wiley-Blackwell, the publisher of two journals that I jointly edit. Last but not least I had a congenial lunch with Jeff Dean, the commissioning editor for Wiley-Blackwell's philosophy books. All that done in a day. It's worth attending conferences after all, if yesterday's outcome is anything to go by. Well, I better get my own acts together for my talk tomorrow morning. I also need to get going to be able to catch up this morning with Mary C Rawlinson, the Editor of The International Journal on Feminist Approaches to Bioethics. Not that you likely care, but it's the coldest day in New York City this winter, and stormy weather at that. Bummer.