ANALISIS PERFORMA INDOBERTWEET DAN DISTILBERT PADA ANALISIS SENTIMEN DENGAN DATASET BERLABEL MANUAL DAN OTOMATIS

Performance Analysis of IndoBERTweet and DistilBERT on Sentiment Analysis Using Manually and Automatically Labeled Datasets

Abstract

Tourism in North Lombok, particularly the Gili Islands (Trawangan, Air, and Meno), plays a significant role in the regional economy. Understanding public sentiment through social media is crucial for improving tourism services and management. This study compares the performance of two transformer-based models IndoBERTweet and DistilBERT in sentiment analysis of tourism-related tweets from X (Twitter). The dataset used consists of 3,159 preprocessed Indonesian-language tweets, labeled through both manual annotation and automatic classification using DistilBERT. IndoBERTweet was evaluated on both manual and automatic labels, while DistilBERT was only applied to the manually translated dataset. Experimental results show that IndoBERTweet with manual labeling achieved an F1-score of 72.98% and demonstrated more balanced performance across all sentiment classes. Meanwhile, DistilBERT showed lower F1-scores overall (max. 57%) but performed efficiently in terms of computational time. Automatic labeling showed weak agreement with manual annotation (only 31.8% match), leading to bias in model learning, particularly the failure to detect neutral sentiment. Evaluation using new test sentences yielded 80% prediction accuracy, yet revealed that IndoBERTweet struggles with implicit sentiment or subtle dissatisfaction. This research highlights IndoBERTweet's effectiveness in Indonesian sentiment analysis and emphasizes the trade off between computational efficiency and contextual accuracy in lightweight models like DistilBERT.