Zhang, X., Lim, H., Fu, X., Wang, K., Xiao, Z., & Qin, Z. (2024). Maritime-Context Text Identification for Connecting Artificial Intelligence (AI) Models. 2024 IEEE Conference on Artificial Intelligence (CAI), 899–904. https://doi.org/10.1109/cai59869.2024.00165
Abstract:
This study focuses on identifying texts related to maritime contexts using an advanced Large Language Model (LLM) and cost-sensitive approach for handling data imbalances. Firstly, a comprehensive dataset specifically for maritime-context queries is collected and augmented. Secondly, the dynamic contextual representations of input query considering the context of each word are obtained by a pre-trained LLM which incorporates Bidirectional Encoder Representations from Transformers (BERT) and Convolutional Neural Network (CNN). Thirdly, a Multi-Layer Perceptron (MLP) is constructed as the classifier to fine-tune the whole network on the newly collected dataset. Finally, the Focal loss is introduced for more effective parameter optimization to tackle the challenge of data imbalance between positive and negative samples, Extensive experiments have been conducted and the following promising results have been obtained: 1) The proposed approach achieves an impressive 99.97% F1 score in recognizing maritime-context texts; 2) The ConvBERT model, an enhancement over the original BERT, demonstrates superior performance in text representation while being more computationally efficient; 3) The Focal loss method outperforms other cost-sensitive learning strategies like class weighting and oversampling techniques; and 4) the proposed method surpasses other deep learning and BERT-based methods in text classification tasks.
License type:
Publisher Copyright
Funding Info:
This research / project is supported by the Singapore Maritime Institute (SMI) - Programme of Maritime AI Research
Grant Reference no. : SMI-2022-MTP-06