Intrusion Detection on ToN-IoT Using XGBoost–LightGBM Stacking Ensemble with Bayesian Optimization

Hakim, Albi Akhsanul (2026) Intrusion Detection on ToN-IoT Using XGBoost–LightGBM Stacking Ensemble with Bayesian Optimization. Undergraduate thesis, UPN Veteran Jawa Timur.

	Text (cover) 22081010194-cover.pdf Download (851kB)
	Text (bab 1) 22081010194-bab1.pdf Download (182kB)
	Text (bab 2) 22081010194-bab2.pdf Restricted to Repository staff only until 18 July 2029. Download (546kB)
	Text (bab 3) 22081010194-bab3.pdf Restricted to Repository staff only until 18 July 2029. Download (983kB)
	Text (bab 4) 22081010194-bab4.pdf Restricted to Repository staff only until 18 July 2029. Download (1MB)
	Text (bab 5) 22081010194-bab5.pdf Download (169kB)
	Text (daftar pustaka) 22081010194-daftarpustaka.pdf Download (183kB)
	Text (lampiran) 22081010194-lampiran.pdf Restricted to Repository staff only until 18 July 2029. Download (218kB)

Abstract

Machine learning-based Intrusion Detection Systems (IDSs) generally achieve strong performance when the test data follow a distribution similar to the training data. However, their performance may degrade under distribution shift, such as the introduction of previously unseen attack types or network endpoints. This study evaluates the robustness of a stacking ensemble model based on XGBoost and LightGBM under distribution shift caused by unseen attacks and unseen endpoints, as well as its performance consistency during streaming inference simulation. The evaluation was conducted using the ToN-IoT Network Subset dataset under three experimental scenarios: Stratified Random, Unseen Attack, and Unseen Endpoint. Model performance was assessed using Recall, Macro-F1, and PR-AUC, together with analyses of false positives, false negatives, Permutation Feature Importance (PFI), and Jensen–Shannon Divergence (JSD). The experimental results show that the stacking ensemble demonstrates greater robustness and more consistent performance than the individual models across the evaluated distribution shift scenarios. Although LightGBM most frequently achieved the highest Recall and Macro-F1 scores, the stacking ensemble consistently ranked among the top-performing models and most frequently achieved the highest PR-AUC. During the streaming inference simulation, the stacking ensemble maintained stable predictive performance, albeit with the trade-off of higher inference latency and lower throughput. These findings indicate that while the stacking ensemble provides improved robustness to distribution shift, its deployment in production environments should carefully consider computational resource constraints and real-time response requirements.

Item Type:

Thesis (Undergraduate)

Contributors:

Contribution	Contributors	NIDN/NIDK	Email
Thesis advisor	Aditiawan, Firza Prima	NIDN0023058605	firzaprima.if@upnjatim.ac.id
Thesis advisor	Junaidi, Achmad	NIDN0710117803	achmadjunaidi.if@upnjatim.ac.id

Subjects:

Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Q Science > QA Mathematics > QA76.9 .A25 Computer Security
T Technology > TK Electrical engineering. Electronics Nuclear engineering > TK5105 Computer Network

Divisions:

Faculty of Computer Science > Departemen of Informatics

Depositing User:

Albi Akhsanul Hakim

Date Deposited:

21 Jul 2026 01:22

Last Modified:

21 Jul 2026 01:57

URI:

https://repository.upnjatim.ac.id/id/eprint/56181