Sentiment Analysis based on Text with Universal Sentence Encoder and CNN-LSTM Models

Saka S. O., CÖMERT Z.

8th International Artificial Intelligence and Data Processing Symposium, IDAP 2024, Malatya, Türkiye, 21 - 22 Eylül 2024, (Tam Metin Bildiri) identifier

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Doi Numarası: 10.1109/idap64064.2024.10711063
  • Basıldığı Şehir: Malatya
  • Basıldığı Ülke: Türkiye
  • Anahtar Kelimeler: CNN, sentiment analysis, Text processing, USE
  • Samsun Üniversitesi Adresli: Evet

Özet

Text processing has a crucial role to perform in today's information era for interpreting data and uncovering valuable insights. In this context, sentiment analysis is especially critical for evaluating customer satisfaction and feedback. The proposed work performs sentiment analysis on customer reviews about airline experiences. For this purpose, convolutional neural network with Long-Short Term Memory (CNN-LSTM) based and Universal Sentence Encoder (USE) models are proposed. The CNN-LSTM model uses training from scratch, while the Universal Sentence Encoder model uses a transfer learning approach. The dataset comprises 1100 customer reviews related to airline experiences, including 705 negative, 303 positive, and 92 neutral reviews. To ensure the reliability of comparison results, the dataset was divided into training and test sets in a 75% to 25% ratio, and neutral samples were isolated. The classification results show that the USE model outperforms the CNN-LSTM model across all metrics. The USE model achieved an accuracy of 95.63%, sensitivity of 97.74%, specificity of 90.67%, and F1 score of 96.92%, while the CNN-LSTM model achieved an accuracy of 92.06%, sensitivity of 97.18%, specificity of 80.00%, and F1 score of 94.51%. The ROC curves further indicate the superior performance of the USE model with an AUC of 99% compared to the CNN-LSTM model's AUC of 96%. In conclusion, the study demonstrates that the USE model, utilizing a transfer learning approach, provides superior performance in sentiment analysis of customer reviews compared to the CNN-LSTM model. These findings suggest that the USE model is more effective and preferable for text classification tasks, offering more balanced and reliable predictions. This research contributes valuable insights for future applications in natural language processing and text mining, highlighting the potential of combining different learning strategies to enhance sentiment analysis..