The “Nature” website, which specializes in scientific research, indicates that some researchers resort to plagiarizing scientific texts or using what is known as a “ghost writer”, which is a person who writes research for payment, while applications of generative artificial intelligence, such as “ChatGPT”, have made it possible to prepare entire research papers easily.
A team of academics conducted a study to evaluate the effectiveness of plagiarism detection applications.
A study published last year showed that the “GPT Zero” application was successful in detecting all papers prepared by artificial intelligence, but it recorded a “false positive results” rate of around 16 percent when it classified papers prepared by real researchers as produced by artificial intelligence, prompting researchers to consider that reliance on these human applications is still limited.
In another experiment, Reddit users subjected the American Declaration of Independence issued in 1776 to a variety of plagiarism detection applications, with some of these applications concluding that between 95 percent and 100 percent of the text of the Declaration was written by generative artificial intelligence.
Researcher Mike Perkins, from the British University in Vietnam, believes that the results of these applications, even if they appear good in controlled tests, should not be a basis for making fateful decisions against students and researchers, noting that the frequency of false positive results makes them unsuitable for judging any researcher or student.
In 2023, a team from Stanford University tested the effectiveness of plagiarism detection applications on 91 scientific articles published in a Chinese scientific forum, all of which were written before 2020, that is, before the emergence of “Chat GPT”.
However, the apps classified more than half of these articles as generated by artificial intelligence, with the rate of false positives reaching 61.3 percent.
Last year, the British Grievances Commission warned universities in England and Wales against using these applications, after receiving a number of student complaints. Among those cases, an autistic student filed a complaint after he received a zero grade on one of his research papers after he was accused of using artificial intelligence, claiming that the plagiarism detection program “is biased against his writing style.”
According to “Nature”, the majority of British universities use the “Terrentine” application, which was launched in April 2023, to monitor academic plagiarism, but universities such as Dundee and Greenwick decided to stop using it due to what they described as a lack of transparency in the working mechanism.
In the United States, Vanderbilt University in Tennessee stopped using the application after months of experiments, and held meetings with officials of Terentine and other companies specializing in artificial intelligence and research bodies to discuss the effectiveness of these technologies.
Terentine says it has adjusted the app’s settings to reduce the rate of false positives to less than 1 percent, so that any paragraph that resembles AI text, without adequate security, will be classified as human-made.
The company added that since the launch of the application, it has analyzed more than 200 million research articles, and it was found that more than 22 million articles, representing about 11 percent of the total, included paragraphs prepared using artificial intelligence applications at a speed of up to 20 percent of the text size.
N. Cicetli, head of the production department at Terentine, said that the results of the application are not intended to provide conclusive evidence of academic misconduct, but rather to help teachers scrutinize research articles and open a discussion with students about the possibility that they can use artificial intelligence tools in the preparation of their research.
