Evaluasi Performa ChatGPT dan Gemini AI dalam Menjawab Pertanyaan Akademik Menggunakan Cosine Similarity (Studi Kasus: Data Mining)

Nabila, Nurul Masya (2026) Evaluasi Performa ChatGPT dan Gemini AI dalam Menjawab Pertanyaan Akademik Menggunakan Cosine Similarity (Studi Kasus: Data Mining). Skripsi thesis, UIN Imam Bonjol.

	Text (cover-daftar isi) nurul masya nabila NIM 2217020016 cover-daftar isi.pdf - Published Version Download (2MB)
	Text (BAB I) nurul masya nabila NIM 2217020016 BAB I.pdf - Published Version Download (1MB)
	Text (BAB III) nurul masya nabila NIM 2217020016 BAB III.pdf - Published Version Download (3MB)
	Text (BAB V) nurul masya nabila NIM 2217020016 BAB V.pdf - Published Version Download (2MB)
	Text (Full Text) nurul masya nabila NIM 2217020016 Full Text.pdf - Published Version Restricted to Repository staff only Download (5MB)

Abstract

This study aims to evaluate and compare the performance of ChatGPT version 5.3 and Gemini AI version 3 in answering academic questions in the Data Mining course using the Cosine Similarity method. The rapid development of Large Language Models (LLMs) has increased the use of artificial intelligence in education, particularly in assisting students in obtaining academic information. However, the quality and consistency of AI-generated responses may vary, making objective evaluation necessary. This research employed a quantitative approach with a comparative experimental method. The dataset consisted of 15 academic questions categorized into definition, concept, and analysis questions. Reference answers (ground truth) were prepared based on textbooks and scientific journals and validated by expert lecturers. Each question was tested three times on both AI models to evaluate response consistency. The analysis process included text preprocessing, semantic representation using Sentence-BERT (SBERT), and similarity measurement using Cosine Similarity. The results showed that both AI models achieved high semantic similarity scores with the reference answers. ChatGPT obtained an average Cosine Similarity score of 0,7879, while Gemini AI obtained 0,7824. These findings indicate that both models demonstrated relatively comparable performance on this dataset in generating academic answers that were close to the reference answers.This study is expected to contribute to the evaluation of AI performance in educational contexts and provide insight into the application of semantic similarity methods for assessing AI-generated academic responses.

Actions (login required)

View Item