Everyone is praising this article, but I find it a bit disingenuous. He criticises Taylorism ("scientific management") but that's exactly what researchers are subject to, and the glam publication economy is at the heart of it.
Science magazine's business model, and the source of its editor's $700k salary, rely on this exploitation of academic labour. There wouldn't be an AI slop paper problem if it weren't for journals and ranking researchers by publication metrics, because there'd be nothing to gain from generating slop.
El Duvelle
Unknown parent • • •or not..
bsky.app/profile/holdenthorp.b…
(although, why would you use chatGpt as a search engine.. 🤦
but also apparently a google search now might also use ChatGPT?? anyway that's why I use noAI duck duck go, hopefully that's still AI-free..)
Edit: never mind, he did use ChatGPT to write the piece, specifically, to "look up the references" that he wanted and then add those links to the documents...
bsky.app/profile/holdenthorp.b…
Virginia Conde
in reply to El Duvelle • • •@elduvelle this is definitely a problem in and of itself; particularly if the references were only added *after the fact* (the writing) and not used as an a priori to support claims or assertions.
Part of the problem is that *even if the references do exist* ("the links go were they're supposed to") there's still a big risk of misattribution and misreferencing when following this approach.
I've tested this myself with my own article to see if GPT4o and GPT5o would refer to my study through a relevant prompt; in one case, it didn't refer to the study at all (even though it has 250+ citations to date, and it's from 2019), in the other, it referred to the study but got the methods/approach wrong (I later found out that those methods belonged to a different publication, not mine).
Long-story short, if all the references were known *a priori* by the author and it was a case of mere retrieval, I could see how it'd be less of a problem (still weird to use GPT for thi
... Show more...@elduvelle this is definitely a problem in and of itself; particularly if the references were only added *after the fact* (the writing) and not used as an a priori to support claims or assertions.
Part of the problem is that *even if the references do exist* ("the links go were they're supposed to") there's still a big risk of misattribution and misreferencing when following this approach.
I've tested this myself with my own article to see if GPT4o and GPT5o would refer to my study through a relevant prompt; in one case, it didn't refer to the study at all (even though it has 250+ citations to date, and it's from 2019), in the other, it referred to the study but got the methods/approach wrong (I later found out that those methods belonged to a different publication, not mine).
Long-story short, if all the references were known *a priori* by the author and it was a case of mere retrieval, I could see how it'd be less of a problem (still weird to use GPT for this given we already have accurate search engines for this purpose).
If the author was using GPT to *search* for references they were not already familiar with, and doing this a posteriori once they had already written the piece (so, an a posteriori add-on of study references to support claims already made), then it is a big, big issue.
El Duvelle
in reply to Virginia Conde • • •Dan Goodman
in reply to El Duvelle • • •El Duvelle
in reply to Dan Goodman • • •Albert Cardona
in reply to El Duvelle • • •@elduvelle @virginiaconde
Using ChatGPT for references is among the worst possible uses of ChatGPT; the are plenty of examples of when that went wrong. To on top of it need a copy editor to check whether the references are legit is really something ... first, the editor in chief of Science claims not to know before writing the piece whether the few references cited in the piece exist, as in, he looked for references to back up what he was writing rather than knowing of those references and use them to inform his statements, plus, once queried, didn't know whether they are real or not. Second, everybody who cares uses Google Scholar, OpenAlex, ORCID, and others, specifically built for this purpose, not a random sentence generation machine.
El Duvelle reshared this.
El Duvelle
Unknown parent • • •@albertcardona @virginiaconde
Schroedingers_Cat
in reply to Albert Cardona • • •Virginia Conde
in reply to Albert Cardona • • •@albertcardona yes, precisely; there is an issue with the order of events, which at the very least should be iterative -- I write a draft of something / an idea / a guess, wonder if someone has looked into this, search for studies, read/skim articles, inform my idea/guess, reshape my writing.
The alternative is to write something I believe / guess / reason, but because I'm writing an *academic article*, I need to make it look like one, so I plaster it with references that seemingly support my assertions (but that I haven't read).
A lot of this discussion seems to circle around whether the references are fake or whether they do exist. But a reference "existing" is not enough if the author hasn't engaged with the text in any capacity, or if it hasn't informed their thinking and reasoning in any way. At this point it ceases to be an academic approach and turns into mere aesthetics.
It is very easy for a GPT model to provide a reference as an output to a prom
... Show more...@albertcardona yes, precisely; there is an issue with the order of events, which at the very least should be iterative -- I write a draft of something / an idea / a guess, wonder if someone has looked into this, search for studies, read/skim articles, inform my idea/guess, reshape my writing.
The alternative is to write something I believe / guess / reason, but because I'm writing an *academic article*, I need to make it look like one, so I plaster it with references that seemingly support my assertions (but that I haven't read).
A lot of this discussion seems to circle around whether the references are fake or whether they do exist. But a reference "existing" is not enough if the author hasn't engaged with the text in any capacity, or if it hasn't informed their thinking and reasoning in any way. At this point it ceases to be an academic approach and turns into mere aesthetics.
It is very easy for a GPT model to provide a reference as an output to a prompt, many of which do exist, while still misattributing information that is not supported by said reference. The only way to ascertain whether a reference has been correctly attributed is to know what the article contains; but if you already know the reference and have engaged with its contents, you wouldn't really need for a GPT model to do that work for you. If you need to check all references for accuracy and relevance anyway, then how does using a GPT model (instead of, say, Scopus, or Web of science) save you time?
Redish Lab
in reply to Virginia Conde • • •@virginiaconde
💯 Even if the GPT model provides a valid reference, if you don't engage with it, you are not doing scholarship.
When I teach students, I tell them that they should never cite anything they haven't actually read, or even better read and re-read.
It's very disappointing (although not surprising) to find that the editors of these GlamourMagz are doing bad scholarship. (Most of the click-bait news-journalism parts of these magazines has been crap for decades and has actively damaged the scientific enterprise.)
El Duvelle
in reply to Redish Lab • • •@adredish @virginiaconde
The worst thing is some scientific labs now encourage their lab members to use #genAI for research.. a "gen AI guidelines" from a famous neuro lab advises their members to "make sure to at least read the abstract of papers that you cite through genAI" 🤦
(when they should know that what's in the abstract is sometimes (usually?) far from representing the contents)
there was also a statement about how ”it's often not allowed by the journal to use genAI for peer-review for privacy reasons, unless it's a preprint..” when we should just not be using genAI at all for peer-review!
hopefully that's not most labs but it's still scary
Schroedingers_Cat
in reply to El Duvelle • • •