Abstract
The release of text-generating applications based on interactive Large Language Models (LLMs) in late 2022 generated an unprecedented and ever-growing interest worldwide. The almost instantaneous success of LLMs stimulated lively discussions in public media and in academic fora alike on the value and potentials of such tools in all areas of knowledge and information acquisition and distribution, but also about the dangers posed by their uncontrolled and indiscriminate use. This conversation is now particularly active in the higher education sector, where LLMs are seen as a potential threat to academic integrity at all levels, from facilitating ‘cheating’ in students’ assignments, to plagiarising academic writing for researchers and administrators. Within this framework, we were interested in testing the boundaries of the LLM ChatGPT (www.openai.com) in areas of our scientific interest and expertise, and in analysing the results from different perspectives, i.e. of a final year BSc student, of a research scientist, and of a lecturer in higher education. To this end, in this paper we present and discuss a systematic evaluation on how ChatGPT addresses progressively complex scientific writing tasks and exam-type questions in Carbohydrate Chemistry and Glycobiology. The results of this project allowed us to gain insight on, 1) the strengths and limitations of the ChatGPT model to provide relevant and (most importantly) correct scientific information, and 2) the format(s) and complexity of the query required to obtain the desired output. Ultimately, with the support of practical examples included in this work, we aim to inspire educators on how to successfully integrate ChatGPT in teaching, learning and assessment in higher education.
Competing Interest Statement
The authors have declared no competing interest.
