Author: James Gilmore, Associate Professor, Clemson University (jngilmo@clemson.edu)
Maybe this will be the start of a new series? One of the logics I see people who are into AI talk about it is the “tool ideology” – it’s “just a tool,” and it’s “about how humans use the tool.” Sometimes you see comparisons between hammers and other technologies — you can use a hammer as part of carpentry, or you can use it as a weapon. In technology studies, this comparison often forms classic classroom debates about human agency vs. technology design: are some technologies inherently dangerous, or do they require specific human uses to become dangerous? The answers to these questions require how we conceptualize the relationships between humans and devices, whether we understand devices to have agency themselves, and how design and implementation have their own politics.
I thought about that line — “it’s just a tool, like a hammer” — this week as I was reading a number of things about AI use in higher education. I see plenty of proselytizing AI use in my own campus, with rampant discussions about the “best ways” to include AI in teaching, research, and university operations. I’m skeptical on a good day about all of this stuff, but particularly egregious uses of AI in academic work have me thinking about the ways people can appear to be hitting themselves in the head with their own hammers. If the tool is so useful, why are you so bad at using it?
So for this edition of “Hitting Yourself in the Head with a Hammer,” I’d like to talk about peer review. In academic publishing, our work is independently reviewed by multiple experts in the field to ascertain the quality of the work, its contribution, its uses of theory and method, etc. Most manuscripts usually go through at least one round of peer review. As part of the process of responding to critiques we receive from reviewers, we are asked to compose a letter explaining the revisions we made and how they respond to reviewer concerns.
To disclose a Very Nerdy Thing about me: I love peer review letters. I love writing them for my own work, and genuinely believe the thoroughness and care with which I treat responding to reviewers is one reason I have a pretty high conversion rate from submission to publication with my manuscripts. I also love reading other people’s letters, because I see them as a window into how people think about their writing and research. They have to explain and justify their decisions, and these letters can oscillate between perfunctory explanations to more poetic elaborations. I always find them enjoyable.
I recently completed a peer review for a manuscript I had given a recommendation of “revise and resubmit.” Myself and the other reviewers found the manuscript a bit unorganized and needing more elaboration with its methods as well as to push its analysis and discussion deeper. I’m going to withhold mentioning the journal or what the article is about to protect the anonymity of the peer review process, and because it’s not important for this story.
I read the letter the author submitted explaining their changes, and found it long-winded but seemingly detailed. They appeared to respond to everything we had asked them to do, and wrote a number of paragraphs explaining the logic of their changes. Okay, great. I go to read the revised manuscript, and within a few pages something feels “off.” The things the author said they had done don’t seem to be translating to the page. They talk about clarifying concepts, providing examples, explaining theories or models, and while that stuff is technically in the revised manuscript, it’s very garbled. Over the course of maybe three pages they introduce a half dozen different concepts and theories, providing very little elaboration along the way.
By the time I finish reading their method, analysis, and discussion, I’m feeling very confused. The first version of the paper needed a lot of work, but it wasn’t bad. It had promise. This one seems more superficial, more convoluted, trying hard to say something without saying anything at all.
So I go back to the reviewer letter to see how they talk about revising the discussion of their Method, because this in particular was exceptionally weak in the revised version. And then it hits me. The whole reviewer letter has been written (or, at least, heavily edited) by an AI. For every discussion point of how they revised the manuscript, the “author” has long-winded sentences that are four or five lines long, talking in a circular verbosity that sounds superficially ‘smart’ but actually isn’t saying anything.
It’s exactly how services like ChatGPT currently produce academic writing. The tokenization process smushes together a lot of concepts at once to try and seem elaborate and complex, but the writing is actually just walking around the same very basic point. Crucially, most AI-generated academic writing I’ve read in the last year doesn’t support its claims. It just says stuff. As I read back through the reviewer letter, and then compared what the letter says the revision did to what the new version of the article says, I can’t say for sure the author used AI to revise their manuscript. They may have just fed the manuscript to their AI service along with the first version and the peer reviewer comments and asked it to write a review letter summarizing the changes.
But it demonstrated, pretty clearly, someone who thought they were using a tool to streamline the peer review process and, in the process, ended up hitting themselves in the head with their own hammer. The reviewer letter made almost no sense, and in at least two places it claimed things were in the revised manuscript that simply weren’t there.
Writing reviewer letters can feel laborious, but they are – to me, anyway – such crucial parts of the review process. They are rare opportunities for authors to speak directly to editors and their peers about their work, albeit in a more formal, anonymized form of correspondence. Why anyone would want to cede that opportunity to a machine is beyond me.
What are some examples you’ve seen recently of people hitting themselves in the head with their own hammer?
