SETIAWAN, MUHAMMAD SURYA ADHI (2026) ANALISIS PERFORMA REASONING MODELS PADA QUESTION ANSWERING SYSTEM BERBASIS RAG SEBAGAI LAYANAN INFORMASI PPMB UPN “VETERAN” JAWA TIMUR. Undergraduate thesis, UNIVERSITAS PEMBANGUNAN NASIONAL "VETERAN" JAWA TIMUR.
|
Text (Cover)
21082010167_Cover.pdf Download (4MB) | Preview |
|
|
Text (Bab 1)
21082010167_Bab 1.pdf Download (335kB) | Preview |
|
![[img]](https://repository.upnjatim.ac.id/style/images/fileicons/text.png) |
Text (Bab 2)
21082010167_Bab 2.pdf Restricted to Repository staff only until 10 March 2029. Download (510kB) |
|
|
Text (Bab 3)
21082010167_Bab 3.pdf Restricted to Repository staff only until 10 March 2029. Download (1MB) |
|
|
Text (Bab 4)
21082010167_Bab 4.pdf Restricted to Repository staff only until 10 March 2029. Download (11MB) |
|
|
Text (Bab 5)
21082010167_Bab 5.pdf Download (463kB) | Preview |
|
|
Text (Daftar Pustaka)
21082010167_Daftar Pustaka.pdf Download (515kB) | Preview |
|
|
Text (Lampiran)
21082010167_Lampiran.pdf Restricted to Repository staff only until 10 March 2029. Download (951kB) |
Abstract
Accurate information is vital for prospective students, yet conventional services often face high query volumes and model hallucinations. This study analyzes reasoning models in a Retrieval-Augmented Generation (RAG)-based Question Answering System for student admissions at UPN “Veteran” Jawa Timur. The methodology involves building a knowledge base from 30 sources, utilizing VoyageAI embeddings, and a dataset of 353 Instagram comments in Indonesian, English, and Javanese Suroboyoan. Evaluation was conducted using the RAGAS framework across six reasoning (o4-mini, gemini-2.5-flash, deepseek-r1) and non-reasoning models (gpt-4o-mini, gemini-2.0-flash, deepseek-v3). Results show reasoning models consistently outperformed non-reasoning models with average RAGAS scores of 0.7772 versus 0.7289 (+6.63%). The most significant improvement occurred in factual correctness (+15.95%), proving the efficiency of internal reasoning in synthesizing complex data. gemini-2.5-flash emerged as the optimal model with a score of 0.8207 and stable multilingual robustness. The study concludes that integrating reasoning models into RAG architecture effectively minimizes misinformation, although retrieval quality remains a fundamental limiting factor. Keywords: PPMB, Question Answering System, Ragas, Reasoning Models, Retrieval-Augmented Generation (RAG).
| Item Type: | Thesis (Undergraduate) | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Contributors: |
|
||||||||||||
| Subjects: | T Technology > T Technology (General) | ||||||||||||
| Divisions: | Faculty of Computer Science > Departemen of Information Systems | ||||||||||||
| Depositing User: | Muhammad Surya Adhi Setiawan | ||||||||||||
| Date Deposited: | 18 May 2026 06:38 | ||||||||||||
| Last Modified: | 18 May 2026 06:44 | ||||||||||||
| URI: | https://repository.upnjatim.ac.id/id/eprint/50306 |
Actions (login required)
 |
View Item |