Title: ChatGPT assists in identifying and recommending research study designs to support study protocol writing

Authors: Gary K.K. Low; Osamudiamen Favour Omosumwen; Sudarshan Subedi; Sam Froze Jiee; Wan-Hsuan Lee; Sirjana Devkota; Selvanaayagam Shanmuganathan; Zelda Doyle

Addresses: Nepean Hospital, Nepean Blue Mountain Local Health District, Kingswood, New South Wales, Sydney, Australia; Sydney Medical School, Faculty of Medicine and Health, University of Sydney, New South Wales, Sydney, Australia ' Alberta Health Services, Edmonton, Alberta, Canada ' Torrens University of Australia, Adelaide, Australia ' Department of Community Medicine and Public Health, Faculty of Medicine and Health Sciences, Universiti Malaysia Sarawak, Kota Samarahan, Sarawak, Malaysia ' Sydney School of Public Health, Faculty of Medicine and Health, University of Sydney, New South Wales, Sydney, Australia ' Royal Preston Hospital, Preston, Lancashire, England, UK ' Menzies Centre for Health Policy and Economics, Faculty of Medicine and Health, The University of Sydney, New South Wales, Sydney, Australia; Ministry of Health, Putrajaya, Malaysia ' Rural Clinical School, School of Medicine, Faculty of Medicine, Nursing, Midwifery and Health Sciences, The University of Notre Dame, Lithgow, New South Wales, Australia

Abstract: This study aimed to validate ChatGPT by identifying the types of study designs in the published scientific literature and recommending the types of research study designs. ChatGPT Version 3.5 was asked to classify the published literature into different study designs. ChatGPT answers were then compared to the blinded independent reviewers' findings. The ChatGPT was then used to provide recommendations for study designs with the research titles from 333 unpublished study protocols. A total of 463 articles were used for validation. The Kappa coefficient was 84.45% agreement for the published literature. The agreement between ChatGPT and reviewers based on whole study protocol was 56.2%. ChatGPT may assist in classifying the types of study design with reasonably good accuracy if an abstract or full text is provided. ChatGPT may also be useful in recommending study designs to a novice researcher but with oversight from experts.

Keywords: study protocol; methodology; generative AI; artificial intelligence.

DOI: 10.1504/IJSMARTTL.2025.150926

International Journal of Smart Technology and Learning, 2025 Vol.4 No.4, pp.314 - 326

Received: 01 Sep 2024
Accepted: 24 Oct 2024
Published online: 05 Jan 2026 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article