Firmansyah, Mukhamad Aziz (2026) DETECTION OF THE INDONESIAN SIGN LANGUAGE (BISINDO) ALPHABET USING THE DETECTION TRANSFORMER (DETR). Undergraduate thesis, UPN Veteran Jawa Timur.
|
Text (Cover)
22081010030-cover.pdf Download (258kB) | Preview |
|
|
Text (Bab 1)
22081010030-bab1.pdf Download (185kB) | Preview |
|
![[img]](https://repository.upnjatim.ac.id/style/images/fileicons/text.png) |
Text (Bab 2)
22081010030-bab2.pdf Restricted to Repository staff only until 25 May 2028. Download (548kB) |
|
|
Text (Bab 3)
22081010030-bab3.pdf Restricted to Repository staff only until 25 May 2028. Download (846kB) |
|
|
Text (Bab 4)
22081010030-bab4.pdf Restricted to Repository staff only until 25 May 2028. Download (891kB) |
|
|
Text (Bab 5)
22081010030-bab5.pdf Download (131kB) | Preview |
|
|
Text (Daftar Pustaka)
22081010030-daftarpustaka.pdf Download (190kB) | Preview |
|
|
Text (Lampiran)
22081010030-lampiran.pdf Restricted to Repository staff only until 25 May 2028. Download (923kB) |
Abstract
Indonesian Sign Language (BISINDO) is the primary language used by the deaf community in Indonesia. However, the communication gap between BISINDO users and the general public remains a challenge that has not yet been fully resolved. This study develops a real-time BISINDO alphabet detection system using the Detection Transformer (DETR) architecture with a ResNet-50 backbone, featuring a single-layer transformer encoder-decoder and 25 object queries, trained to recognize 26 letters (A–Z) and 17 BISINDO words. The research was conducted under two training scenarios: Scenario 1 without on-the-fly augmentation and Scenario 2 with on-the-fly augmentation, which includes spatial and photometric transformations. Evaluation was performed using standard PyCOCOTools metrics, along with real-time robustness testing under three conditions: normal conditions with the same subject, dynamic lighting variations, and subjects different from the training data. PyCOCOTools evaluation results show that Scenario 2 with augmentation outperforms in nearly all key metrics: AP@[IoU=0.50:0.95] of 0.7188 (vs 0.6772), AP@0.75 of 0.9192 (vs 0.8118), AR@maxDets=100 of 0.7949 (vs 0.7219), with the most significant improvements observed for medium-sized objects, where the medium AP increased by 27.6% and the medium AR by 31.7%. In real-time testing, Scenario 2 maintained a 100% detection success rate across all test conditions, while Scenario 1 only achieved 34.9% under the most challenging condition, different subjects with varying lighting. The developed system runs at 30–60 FPS on a GPU and 5–9 FPS on a CPU, and is implemented in a web-based interface using Streamlit. This study concludes that on-the-fly augmentation plays a crucial role in building models that are not only superior in metrics but also reliable and generalize well to real-world conditions.
| Item Type: | Thesis (Undergraduate) | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Contributors: |
|
||||||||||||
| Subjects: | Q Science > QA Mathematics > QA75 Electronic computers. Computer science | ||||||||||||
| Divisions: | Faculty of Computer Science > Departemen of Informatics | ||||||||||||
| Depositing User: | Mukhamad Aziz Firmansyah | ||||||||||||
| Date Deposited: | 26 May 2026 01:51 | ||||||||||||
| Last Modified: | 26 May 2026 01:51 | ||||||||||||
| URI: | https://repository.upnjatim.ac.id/id/eprint/52476 |
Actions (login required)
 |
View Item |