ChatGPT applications in Academic Research: A Review of Benefits, Concerns, and Recommendations

Abstract

Background ChatGPT has emerged as a valuable tool for enhancing scientific writing. It is the first openly available Large Language Model (LLM) with unrestricted access to its capabilities. ChatGPT has the potential to alleviate researchers’ workload and enhance various aspects of research, from planning to execution and presentation. However, due to the rapid growth of publications and diverse opinions surrounding ChatGPT, a comprehensive review is necessary to understand its benefits, risks, and safe utilization in scientific research. This review aims to provide a comprehensive overview of the topic by extensively examining existing literature on the utilization of ChatGPT in academic research. The goal is to gain insights into the potential benefits and risks of using ChatGPT in scientific research, exploring secure and efficient methods for its application while identifying potential pitfalls to minimize negative consequences.

Method The search was conducted in PubMed/MEDLINE, SCOPUS, and Google Scholar, yielding a total of 1279 articles and concluded on April 23^rd, 2023. After full screening of titles/abstracts and removing duplicates and irrelevant articles, a total of 181 articles were included for analysis. Information collected included publication details, purposes, benefits, risks, and recommendation regarding ChatGPT’s use in scientific research.

Results The majority of existing literature consists of editorials expressing thoughts and concerns, followed by original research articles analyzing ChatGPT’s performance in scientific research. The most significant advantage of using ChatGPT in scientific writing is its ability to expedite the writing process, enabling researchers to draft their work more efficiently. It also proves beneficial in improving writing style and proofreading by offering suggestions for sentence structure, grammar, and overall clarity. Additional benefits identified include support in data analysis, the formulation of protocols for clinical trials, and the design of scientific studies. Concerns mainly revolve around the accuracy and superficiality of the generated content, leading to what is referred to as “hallucinations.” Researchers have also expressed concerns about the tool providing citations to nonexistent sources. Other concerns discussed include authorship and plagiarism issues, accountability, copyright considerations, potential loss of diverse writing styles, privacy and security, transparency, credibility, validity, presence of bias, and the potential impact on scientific progress, such as a decrease in groundbreaking discoveries.

Conclusion ChatGPT has the potential to revolutionize scientific writing as a valuable tool for researchers. However, it cannot replace human expertise and critical thinking. Researchers must exercise caution, ensuring the generated content complements their own knowledge. Ethical standards should be upheld, involving knowledgeable human researchers to avoid biases and inaccuracies. Collaboration among stakeholders and training on AI technology are essential for identifying best practices in LLMs use and maintaining scientific integrity.

Introduction

ChatGPT has taken the world in a storm. Within the first few days of its release to the public, the number of users reached a few million, growing to over one hundred million users within the first two months according to the latest figures [1]. While generative language models like ChatGPT were under development around the world, the significance and scale of impact of ChatGPT released by OpenAI by far exceeds any other released model. This is partly because ChatGPT is the first Large Language Model (LLM) to go fully public with direct access to its full capabilities [2]. The chatbot can perform many language generation tasks to a remarkable degree of English fluency. ChatGPT can summarize long documents from different domains, provide answers to questions from a variety of domains directly, draft stories or essays on diverse topics which include real-life figures, provide commentary and feedback on its own responses and regenerate new responses based on previously held conversations [3]. The chatbot seemed to be able to handle any sort of question or request with high confidence to the point that it became challenging for the public to decide on the authenticity and truthfulness of the generated content [4]. Indeed, many authors have commented on the tendency of chatGPT to produce false information in subtle ways, a phenomenon referred to as hallucination [5].

The use of ChatGPT in scientific research has raised initial concerns since language plays a crucial role in scientific communication, encompassing informal discussions among scientists, conference presentations, and most importantly, formal peer-reviewed publications. With the advent of generative language modelslike ChatGPT, there has been a global surge in interest in the use of these models in scientific publication.

Utilizing ChatGPT and similar Language Models (LLMs) holds potential for easing researchers’ workload, aiding in research planning, execution, and presentation. This could allow researchers to dedicate more time to developing innovative experimental designs, potentially leading to breakthroughs in diverse fields.

Since its initial launch in November 2022, there has been a significant increase in publications, particularly editorials, related to ChatGPT. Publishers, editors, reviewers, and authors have struggled to keep up with the accelerated pace of publications, opinions, recommendations, and perceived benefits and risks. The aim of this study is to provide a birds view of this dynamic and evolving subject. Our goal is to fully review the literature regarding the use of ChatGPT in academic research and shed light on its potential benefits and risks for scientific research. How can the chatbot be used safely and effectively for scientific research? and what are the pitfalls that should be avoided to minimize harm?

We will attempt to answer these questions based on the published literature and the opinions of leading scientists and experts in the domain.

Materials and Methods

Search strategy

The current review was conducted according to the methodological framework explained by Arksey and O’Malley[6]. The inclusion criteria involved any type of published scientific research or preprints (article, review, communication, editorial, opinion, etc.), addressing the use of ChatGPT in scientific research and publication. The exclusion criteria included: (1) non-English articles; (2) articles addressing ChatGPT in subjects other than scientific research; and (3) articles from non-academic sources (e.g., newspapers, internet websites, magazines, etc.). The search was conducted in PubMed/MEDLINE, SCOPUS, and Google Scholar. The search terms used for PubMed/MEDLINE and SCOPUS were (ChatGPT) AND (Scientific research) which yielded 50 and 11 articles respectively and the search was concluded on April 23^rd, 2023. The search on Google Scholar was conducted using Publish or Perish (Version 8). The search terms were“ChatGPT” and “Scientific research “for the years: 2022–2023. The Google Scholar search yielded 1218 articles and concluded on April 23^rd, 2023.

Summary of the Article Screening Approach

The results were imported to Rayyan software (https://www.rayyan.ai/pricing/) which yielded a total of 1279 articles. Then, screening of the title/abstract was done by two reviewers independently (A.A. And Am.A) and duplicate articles were excluded (n = 151), followed by exclusion of articles published in languages other than English (n = 3). Additionally, the articles that fell outside the scope of the review (articles addressing ChatGPT in a context outside scientific research) were excluded (n = 879).

Afterwards, full screening of the remaining articles (n = 246) was done. Screening completed resulted in the exclusion of an additional 40 articles that fell outside the current review’s scope. An additional 9 articles were excluded due to inability to access the full text. Two articles were excluded due to being a duplicate with different titles but same website and author. Four more articles were excluded due to unavailability of an English version of the full text. One entry was excluded as it was a conference abstract. Five articles were excluded as they assess general AI tools or generative language model other than ChatGPT. An additional six articles were excluded because they were news articles. Details of each included articles were collected such as (1) the year of publication, type of publication, purpose, the filed studied in the publication (2) benefits and expected applications of ChatGPT in scientific research, (3) risk or concerns of ChatGPT in scientific research and (4) the main conclusions and recommendation regarding ChatGPT in scientific research.

Results

Following the full screening process, a total of 181 articles were eligible to be included in the review. The PRISMA flowchart of the articles’ selection process is shown in figure 1.

Figure 1:

Flowchart showing final count of included articles in the review.

Characteristics of the Included Articles

Table 1 (supplementary) illustrates the detailed characteristics of the included papers regarding the type of the article, field, purpose of the study and author.

It is obvious that vast majority of existing literatures are editorial where editors and experts are sharing their thoughts and concerns about the use of ChatGPT in research followed by original research articles where authors are reporting their use of ChatGPT and analyzing its performance at various stages of Scientific research (see figure 2).

Figure 2:

Types of included articles

Benefits and Possible Applications of ChatGPT in scientific Research Based on the Included Articles

Summary of the most perceived benefits and applications of ChatGPT in scientific research is shown figure 3. The most listed benefit reported by researchers was the speed-up of the write-up process [7]– [76]. Many researchers acknowledged that ChatGPT assisted in expediting the writing process for various sections of the manuscript, including titles, abstracts, conclusions, and other parts of the manuscript [7], [9], [10], [15]–[17], [21], [22], [24], [25], [28], [30], [35], [46], [49], [50], [58], [60], [62], [64], [65], [69], [72], [75], [77]–[89]. Additionally, researchers highlighted that ChatGPT proved useful in constructing editorials, case reports, and reviews[10], [11], [34], [69], [77], [87], [90], [91]. Another frequently mentioned benefit was the improvement of writing style and proofreading[5], [10]–[14], [20], [30], [33]–[35], [39], [41], [42], [46], [47], [57], [59], [63], [72], [74], [77], [79], [81], [84], [92]–[98] Researchers found that ChatGPT helped enhance the overall quality of their writing by providing suggestions for improved sentence structure, grammar, and clarity which is thought to be particularly helpful for non-native English speakers, as it provided support in writing and communicating their research effectively [20], [21], [24], [33], [35], [59], [63], [67], [75], [80]–[82], [86], [92], [94]–[97], [99]–[103]. Additionally, ChatGPT served as a valuable tool for summarizing scientific articles, allowing researchers to quickly extract key information from complex texts [7], [9]–[11], [16], [25], [26], [29], [30], [35], [38], [40], [41], [43], [46], [47], [51], [55], [57], [61], [63], [67], [72], [77], [79]–[81], [85], [89], [104]–[113] that is extremely helpful to facilitate literature review, detecting research gaps and construct hypothesis in a timely manner[10], [14], [16], [24], [46], [47], [50], [62]–[64], [67], [72], [74], [80], [81], [104], [109], [110], [112], [114]–[117] and even was reported to be helpful in suggesting study design [59], [72]. Several other benefits were identified through the review which included assistance in data analysis [9], [11], [16], [17], [20], [22], [25], [39], [46], [48], [55], [59], [61], [67], [68], [72], [77], [80], [81], [97], [98], [104], [106], [113], [116], [118]–[121], data augmentation [68], and the construction of protocols for clinical trials [122]. Furthermore, ChatGPT demonstrated its utility as a real-time research assistant [16], [21], [72], [81], [90], [93], [119], [122], assisting researchers in tasks such as literature reviews [5], [10], [11], [16], [23]–[25], [34], [35], [41], [46], [51], [54], [57], [59], [62], [63], [77], [79]–[81], [87], [104], [106], [111], [113], peer reviewing manuscripts [20], [25], [35], [62], [77], [91], [94], [97], [123], and writing and debugging code for data analysis [5], [14], [67], [72], [83], [107], [124], [125]. Its ability to learn from interactions and correct errors, with the guidance of experts, was also noted as a valuable feature[126].

Figure 3:

The most perceived benefits and applications of C[12], [36], [111], [123], [42], [47], [51], [58], [67], [71], [96], [101]hatGPT in scientific research as reported in the included studies.

Providing these services will save the researcher time to focus on real questions and give more time for performing scientific research. ChatGPT is thought to accelerate the pace of scientific discovery and innovation especially in molecular studies and drug discovery studies [12], [36], [42], [47], [51], [58], [67], [71], [96], [101], [111], [123], [113], [116], [120]. ChatGPT could enable researchers to quickly identify new potential targets, design new drugs, and optimize pharmacokinetics and pharmacodynamics. It also supported decision-making in early-stage drug discovery by generating new knowledge[113], [116], [120].

In qualitative research, ChatGPT was found to be useful in extracting meaningful information from text, labeling the answers to open ended questions, and coding the responses [96], [127], [128]. Additionally, it facilitated the generation of synthetic data for experimentation especially in Human computer Interaction and software engineering although with potential limitations in terms of quality [129], [130]. In computer science research, ChatGPT proved valuable in text annotation tasks[131]. It was also reported to assist in sentiment analysis [128]. It was also reported that ChatGPT is a helpful tool to simplify the scientifc findings that will help to disseminating research findings to the community in simple language [41], [101], [105], [132], [133]. It was also reported to be benefecial during the manuscript submission where it can suggest suitable journals according to the title or the abstract of the manuscript [134]and also suggest possibereviewrs for the manuscript which is beleved to be helpful for editors [10], [11], [34], [69], [77], [87], [90], [91], [86], [135].

Risk and concerns of use of ChatGPT in scientific Research Based on the Included Articles

This review has identified several reported concerns regarding the use of ChatGPT in scientific research which are listed in figure 4. The most reported concern was the inaccuracy of the provided information and the superficiality of the generated content [7], [10], [26]–[30], [35]–[39], [12], [40]–[47], [49], [50], [13], [51], [52], [54], [55], [58]–[60], [62]–[64], [14], [66], [67], [72]–[74], [77], [79], [80], [82], [84], [19], [86], [92], [95], [97]–[99], [102]–[105], [21], [106], [107], [110], [111], [113], [115], [117], [120], [125], [126], [23], [130], [132], [136]–[143], [24], [144]–[150], [25]. Researchers expressed apprehension about relying on ChatGPT for accurate and in-depth scientific knowledge. ChatGPT provides superficial knowledge and has been reported to provide very well constructed responses that are completely falsified and misleading known as “hallucinations” [5], [13], [20]–[22], [29], [42], [63], [67], [68], [76], [77], [80], [89], [93], [97], [99], [102], [104], [110], [135], [138], [140]–[143], [151]–[153]. One very alarming concern is that the tool tend to provide citations to the provided falsified response which are themselves are fake and non-existing [19], [20], [22], [24], [28], [29], [34], [39], [41], [43], [45], [52], [53], [59], [63], [64], [69], [72], [77], [81], [84], [93], [102], [103], [110], [117], [135], [137], [138], [141]–[144], [149]–[152], [154]–[157]. For example, in one study, out of the 23 references that were provided by ChatGPT, only 14 were accurate, 6 seems to have been completely made up and 4 existed but were attributed to the wrong author[158]. Therefore, identifying data fabrication or falsification during the peer-review of a manuscript containing text generated by ChatGPT will pose a significant challenge for reviewers and editors. This challenge becomes even more worrisome if the actual authors of a paper are not responsible for careful fact-checking.

Figure 4:

The most perceived risks and concerns of ChatGPT in scientific research as reported in the included studies.

Following this, other major concerns focused on authorship, where many included editorials discussed the eligibility of ChatGPT to be an author. It is obvious that most of the existing literature agreed that ChatGPT does not meet the International Committee of Medical Journal Editors (ICMJ) and the Committee on Publication Ethics (COPE) criteria for authorship [10], [11], [54], [55], [62], [67], [71], [72], [78], [81], [83], [85], [13], [86], [96]–[98], [102], [107], [110], [111], [114], [121], [17], [149], [150], [152], [157]–[163], [20], [164]–[172], [21], [24], [27], [41], [42] unless these criteria are changed or updated [74]. Prior to the launch of ChatGPT, English-editing services were commonly used by authors during the publication process, such services were not included as co-authors in the published work. Therefore if ChatGPT is only used for language editing purposes, then there is no issue with using it to prepare scientific articles without listing it as being a co-author [92]. Another major concern that received attention is the potential for plagiarism [11], [14], [39], [41], [42], [46], [49], [55], [63], [66]–[68], [20], [69], [78], [82]–[84], [98], [103], [104], [107], [110], [21], [111], [121], [122], [125], [126], [135], [138], [141], [157], [158], [22], [24], [25], [27], [30], [35]. ChatGPT relies on existing training data from the internet which is paraphrased without any scientific position, leading to concerns on the originality of the produced text. Moreover, directly adopting the full text written by ChatGPT may constitute plagiarism and violate the code of conduct for scientific publishing, as originality is the foundation of scientific writing. Work that relies solely on ChatGPT’s outputs lacks critical thinking and the reasoning skills of a human being and can potentially be detrimental to the research and impede the science advancement[138]. Researchers were wary of the implications of using ChatGPT in terms of scientific integrity and the appropriate allocation of credit for intellectual contributions [17], [20], [22], [27], [28], [38], [41], [51], [64], [69], [72], [80], [83], [84], [98], [119], [126], [130], [152], [168], [173], [174].

Additional concerns collected during the review are related to accountability [7], [17], [21], [35], [62], [67], [81], [93], [98], [110], [130], [136], [152], [158], [174], copyright issues [20], [24], [130], [158], [175], [30], [35], [41], [54], [63], [72], [111], [122]. Furthermore, the use of ChatGPT raised concerns about the encroachment on the originality of writing styles, potentially erasing the unique characteristics and cultural influences of human authors in favor of a homogeneous style [51], [62], [74], [82], [107], [138], [176]. Researchers questioned whether this loss of individual stylistic traits, while aiming for mutual comprehension, might remove a desired feature in scientific publications.

Moreover, researchers expressed apprehension that the use of ChatGPT could exacerbate existing disparities and inequalities, potentially leading to natural selection of certain research directions [20], [24], [67], [91], [130], [173], [177], [178] where people with access to the tool will be able to use it and speed their publication process compared to those who have no access which has been referered to as a Mathew effect [130], [178] [177]. The overreliance on ChatGPT is perceived as a potential erosion of the skills and expertise required for high-quality academic publication [11], [21], [83], [84], [111], [115], [126], [135], [166], [176], [25], [26], [42], [46], [47], [58], [64], [67]. Additionally, there are concerns about the potential reduction in the frequency of disruptive scientific breakthroughs [9], [11], [14], [21], [35], [42], [67], [80], [82], [107], [115], [132], [135], [138], [176], [179], [180], a decrease in research quality, the presence of bias[13], [20], [21], [24], [26], [30], [32], [41], [46], [47], [51], [55], [62], [63], [66]–[68], [73], [75], [82], [84], [89], [91], [93], [96]–[98], [101], [104]–[107], [113], [116], [121], [125], [126], [128], [130], [139], [148], [157], [166], [174]. This is thought to lead to the rapid production of low-quality scientific articles, highlighting the potential for a decline in the overall quality of research output due to the automated nature of ChatGPT [14], [42], [176], [180] which will facilitate and enhance the growth of predatory journals and papermills [51], [63], [86], [110], [176], [180].

Other issues surrounding privacy and security [24], [71], [106], [130], [139], [174], transparency, credibility, and validity were also raised as concerns [17], [19], [21], [24], [41], [67], [70], [71], [93], [106], [110], [114]–[116], [130], [136], [139], [140], [142], [166], with researchers expressing doubts about the up-to-date nature of the information provided by ChatGPT [7], [17], [21], [34], [63], [73], [79], [110], [111], [142], [157], [180] since it is trained on data up to November 2021.

Discussion

Looking at the five essential stages of conventional research, namely idea generation, literature synthesis, data identification and preparation, method determination and implementation, and results analysis, ChatGPT demonstrates diverse potential applications in each stage. Its greatest perceived advantages lie in the latter phases of scientific research, particularly in constructing the research manuscript where ChatGPT has been reported to accurately and precisely draft an initial manuscript when provided with suitable prompts[181]. As ChatGPT continues to advance, it holds significant promise as an artificial research assistant that can assist in organizing the structure of research, rather than generating the content itself[10]. In this review, it was prominent that much of the existing literature acknowledges the advantages of ChatGPT in accelerating the writing process. Many articles have specifically assessed ChatGPT’s capabilities in constructing abstracts, editorials, conclusions, and even a full article. A recent study examined the ability of human authors and AI detection tools to distinguish between abstracts generated by ChatGPT and those generated by humans. The findings revealed that human reviewers could miss up to 32% of abstracts that were fully fabricated by ChatGPT during a thorough screening process [75] whereas 14% of real abstracts were mistakenly identified as ChatGPT-generated abstracts [182]. Furthermore, none of the journal editors were able to destingush between human generated and AI generated abstracts in neurology field [86].

This clearly indicates the capabilities of ChatGPT to assist in the scientific paper write up with original text but a precaution should be directed toward the content. This is thought to save researchers time from laborious writing processes, allowing them to concentrate on disruptive science and innovation[99]. With time, scientific writing is thought to gradually becomes less important compared to the ability to generate new findings, which will open opportunities for scientists to share their brilliant ideas regardless of their English language capabilities. With this major benefit of ChatGPT, it is thought to bridge language barriers and assist non-English speaking researchers in crafting high-quality texts. Many of the world’s top researchers are non-native English speakers, and they often face challenges as their papers may appear less professional compared to their native English-speaking counterparts. Chen T-J wrote an editorial in Chinese which was translated by ChatGPT into English, then modified little bet and final English editing was done by ChatGPT before the submission[52].

This advantageous aspect is believed to have a dual-edged impact. While it undoubtedly expedites the laborious process of writing and allows researchers to allocate more time to innovative ideas and groundbreaking scientific advancements, there is a concern that it may exacerbate issues such as paper mills. These, in turn, could fuel the proliferation of predatory journals and the dissemination of fake science and fabricated evidence. Alarmingly, a request was made to fabricate a manuscript for a fictitious clinical trial comparing two drugs for the management of a certain condition, and the request was executed proficiently, constructing a paper complete with realistic statistics and numerical data [14]. The author then requested changes in the conclusion to demonstrate the superiority of drug A over drug B. This scenario is highly disconcerting and has the potential to erode trust in scientific evidence.

Furthermore, it is thought researchers may excessively depend on it for generating scientific content, leading to a lack of critical thinking and originality in their writing and some researchers even consider research papers created using ChatGPT may be viewed as unoriginal and potentially problematic. When the ChatGPT outpot was critically evaluated manually, the detected plagiarism was ranged from 5% to 48.9%. It was found that the tool uses both academic and non academic sources to construct a scientific response. The sentences in the scientific section provided by ChatGPT for example were found to be copied word by word from general sources such as Wikipedia, LinkedIn, and Apple App Store without verification of evidences [157]. This neccessiate a careful evaluation of its reponses by a human and critical evaluation of existing evidence for whatever scientific piece it provides.

Authorship has been extensively discussed in literature, with most journals and publishers not supporting ChatGPT as an author. Currently, there are only four articles that have listed ChatGPT as an author [183]–[186], and one of them has submitted a corrigendum to remove ChatGPT from authorship based on the journal’s guide for authors and Elsevier’s Publishing Ethics Policies¹⁴⁵. The consensus among the scientific community is that ChatGPT cannot be held accountable or responsible as it is not a legal entity, and it is treated similarly to other editing tools that were used in the past but not listed as authors. However, there is a belief that in the future, AI chatbots will possess the same rights as human scientific authors. As AI chatbots become increasingly sophisticated and contribute significantly to scientific research, it will be crucial to acknowledge their contributions and give them due credit[171]. In fact, some researchers assert that ChatGPT should be credited as a co-author or even the primary author when it significantly assists in the writing and editing process, enabling the generation of the entire text efficiently[25]. In fact they were four positions appeared in literature regarding authorship inclusion of ChatGPT, those who support its inclusion, those who do not, those who propose to entirely omit authorship when the manuscript is entirely produced by ChatGPT[170] and those who see it is better to be appropriately cited when used [172]

In the other hand, Lin has discussed the term “ substantial contributions” which does not guarantee an AI model an authrship even if construct the whole manuscript but it does guarantee a humon co-author who had a very minor contribution in a 100 authors manuscript [85].

The discussion of authorship issue has in fact higlighted an already existing major issue in academia for years that is evaluating the success of a scientists by the nymber of publications and citation. In the era of generative AI, a scietist can have tens of abstracts and manuscript ready for submission in 24 hours. Instead of giving the attention to the mere number of publication and citation, the criteria should involve measuring the impact of the research in developng new concepts, models and to have an impact [71]

The inclusion of ChatGPT in the research process should be subject to serious scrutiny. Editors should be alerted to potential issues with the manuscript if there is any acknowledgement of ChatGPT’s contribution or even the mention of ChatGPT as a coauthor [156]. In a conversation with ChatGPT regarding its involvement in authorship, it expressed the following response: “Writing a scientific paper entails more than simply generating text. It is highly unlikely for a machine learning model to produce a scientifically valid and publishable paper without significant human intervention and oversight” [86] [171].

Prominent publishers emphasize the exclusive use of technologies like ChatGPT to enhance article readability and language, with proper documentation. Authors are responsible for manually evaluating AI-generated content. AI tools should not be credited as authors or co-authors due to their inability to take on the responsibility and accountability associated with authorship. ChatGPT acknowledges that ethical violations can occur if authors fail to adhere to research and writing standards, such as plagiarism, data fabrication, or improper citation. Authors must ensure their work complies with ethical standards, regardless of the tools used [87].

While technology may reduce the need for certain skills like manual literature search, it also introduces new abilities such as creating prompts for conversational AI models. The loss of some expertise may not be problematic, considering that many researchers no longer perform statistical analysis manually.

However, it is crucial for the academic community to carefully evaluate which skills and attributes are essential for researchers to possess [111].

Another significant risk and concern that has been raised repeatedly, is the occurrence of inaccurate responses and hallucinations in ChatGPT, where it tends to provide fabricated statistics and conclusions that require human evaluation and critical assessment. For instance, when given titles and asked to write abstracts, ChatGPT generates well-structured abstracts that contain fictional numbers and statistics, necessitating careful evaluation. However, it has been shown that improving prompt with more details increases the accuracy of the response [189] [189][124]

The presence of fake citations is another significant concern, and even the developers of ChatGPT may not fully understand the reasons behind it. It has been observed that ChatGPT tends to cite highly cited publications, which contributes to the Matthew Effect in science, where already popular papers receive even more citations, widening the gap. Additionally, ChatGPT predominantly cites reputable journals in the field, with Nature being the most frequently cited journal [177]. Interestingly, ChatGPT relies exclusively on citation count data from Google Scholar, disregarding citation information from other scientific databases like Web of Science or Scopus [177] [156]. These citations often include authors who are experts in the field and cite journals that are highly relevant. Surprisingly, Day et al. suggest potential positive applications of such fake citations, highlighting that the fake titles provided by ChatGPT could potentially generate excellent papers.

As the ethical framework for ChatGPT is still being developed, several journals have established guidelines for its use in scientific research, which can be found on their websites. For example, Nature recommends authors to disclose the usage of large language models (LLMs) in their submissions. On the other hand, the journal Science has taken a step further by updating their license and editorial policies, explicitly stating that text generated by ChatGPT or any other AI tools cannot be incorporated into their work, and figures, images, or graphics must not be products of such tools.

Despite these concerns, many researchers maintain an optimistic outlook and believe that ChatGPT can make a significant and positive contribution to the scholarly community if it is utilized ethically and sensibly (Xames and Shefa, 2023). Day et.al have discussed the five reasrch priorites in the use of chatGPT and other LLMs in research which are maintaining human verification, establishing accountability rules, investing in truly open LLMs, harnessing the benefits of LLMs for scientific paper writing, and broadening the discussion [156].

Recommendation

After reviewing the existing literatures, we suggest three directions to the recommendations on the use of LLMs in scientific research and publications. The first is directed towards journal policies and editors. Authors agree that AI is becoming part of the research process and the benefits of using it outweigh those of rejecting its use. Hence, journals and editors need to establish rigorous policies to ensure the ethical use of LLMs in publications [5], [171]

For the sake of scientific integrity, authors urge journals to request the declaration of LLM use in the manuscript preparation by having check boxes or signing declarations [132], [75], [94], [176], [89] These include declaring the specific model of LLM used and the type and amount of contributions. However, many authors agree that chatbots should not be given author credits due to their lack of ethical responsibility toward the manuscript [161] [190], [107] [137][172][191]. Others recommend setting up plagiarism checks and margin to ensure limited contribution of LLMs and the governance of human experience over the manuscript [75] [103], [55] [146], [141][87], [172], [192], For example, the editors of Pediatric Radiology journal are updating their author guidelines so that authors need to declare the use of AI in the Material and Method section of the manuscript[95]. Few other authors recommend a revised policy on peer reviewers’ selection [16], [20], [155], [193].They urge journals to select highly experienced researcher and field experts to review submitted manuscripts. This is to avoid false information generated by LLMs that might be missed by submitting authors. Another recommendation [193] is to impose penalties by the journal on authors who use LLMs unethically or without declaration.

The second direction of recommendation is towards authors integrity and transparency. Many authors stressed on the ethical responsibility of the authors toward the scientific community. Human authors should cross check any information contributed by LLMs before submission. With LLMs falling short in accuracy, authors who are the experts in their field are responsible for correcting any fallacies in the manuscript (Huh 2023, [107]. To ensure transparency, authors must declare the use of LLMs in their research and manuscript preparation. Ollivier et al recommend the sharing of original AI generated text with the readers[13]. On the other han, Buriak et al. 2023 suggest that authors establish the first draft of the manuscript and later on use AI tools to enhance or complement it [80]. The declaration of using LLMs in preparing the manuscript is a responsibility carried by the authors. Authors need to state how much has the AI tool contribute the research submitted. This transparency is required in the current era of AI.

The third direction of recommendations acknowledges that AI and LLMs are here to stay and they will contribute to the enhancement of scientific method and the advancement of knowledge. Therefore, many authors call of a collaboration between researchers, technology experts and LLMs [155] [89] [21] [41], [130] There is a need to establish well trained LLMs in the scientific field [176]. Researchers could provide well described data while LLMs process them and train on them [55]. This will ensure a better version of LLMs that are capable of providing more accurate information. A collaboration with technology experts is required to establish AI-generated text detection tools. Just like any regular plagiarism checker software used currently, AI-generated text checker tools should be created and verified to ensure accurate detection [119], [41] (suggest training to be provided to students at both undergraduate and postgraduate levels on AI technology, their benefits, pitfalls and ethics of their use in scientific research Dergaa et al. 2023). Moreover, Kuetls recommends that researchers need to be familiar with this emerging technology and its updates to avoid any negative impact on their research output[141][5], [10], [46], [51], [54], [57], [59], [62], [63], [77], [79], [80], [11], [81], [87], [104], [106], [111], [113], [16], [23]–[25], [34], [35], [41].

To maintain the transparency and integrity of scientific research, Sanmarchi et al recommends collaboration of all stockholders to identify the best practice in LLMs use [119].

Conclusion

ChatGPT has the potential to revolutionize scientific writing by assisting researchers in efficiently producing high-quality and well-written articles. However, it is important to acknowledge that ChatGPT cannot replace the expertise of researchers but can effectively streamline the writing process. Researchers should be cautious of the potential risks associated with using ChatGPT in scientific writing and should consider the generated content as a tool to complement their own critical thinking and expertise.

Additionally, relying solely on AI without the involvement of knowledgeable human researchers may perpetuate biases and inaccuracies in the data, resulting in unfair outcomes and hindering scientific progress. Studies have demonstrated that language models like GPT-3, trained on large web-based datasets, can exhibit biases related to gender, race, ethnicity, and disability status. Hence, despite the impressive advancements in AI tools, the presence of experienced experts remains essential in scientific activities and writing to ensure the quality of the work.

In light of these considerations, it is advisable to harness the full potential of ChatGPT as a linguistic tool while upholding rigorous ethical standards. This approach allows researchers to benefit from its capabilities while maintaining the integrity of scientific practices. Regardless of how AI is employed, we strongly believe that involving domain experts in scientific activities and writing is crucial to uphold the quality of the work. Furthermore, the rapid progress of AI tools may lead to some researchers achieving a significant increase in publication numbers without a corresponding growth in their actual expertise in the field. This can raise ethical concerns when academic institutions prioritize publication quantity over quality, potentially impacting the hiring process for professionals. Ultimately, the influence of language models on scientific writing will depend on their adoption and utilization within the scientific community. It is recommended to use ChatGPT as a supplemental tool for constructive writing, reviewing materials, and rephrasing text rather than relying on it for providing an entirely original blueprint.

Hayes, Carol Mullins, Did a Robot Write This Title? Creativity, Ownership, Justice, and Copyright Law (December 15, 2022). Available at SSRN: https://ssrn.com/abstract=4304470 or http://dx.doi.org/10.2139/ssrn.4304470