Optimizing Multi-Layer Perceptron Performance in Sentiment Classification through Neural Network Feature Extraction

Abstract

There are some problems with using the Multi-Layer Perceptron (MLP) model for complex tasks because it can be hard to understand hierarchical relationships and tends to overfit data with a lot of dimensions. This research proposes an enhanced MLP model for sentiment classification by integrating feature extraction layers from advanced neural networks, specifically the Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), and Bidirectional LSTM (Bi-LSTM). These layers aim to improve the model's representation capabilities by capturing more nuanced features. To evaluate the performance improvements of this augmented MLP model, metrics such as accuracy, precision, recall, F1-score, and the Area Under the Curve for Receiver Operating Characteristics (ROC-AUC) were employed. A key metric focus is the delta value, representing changes in the ROC-AUC, to assess the significance of these enhancements. The integration of CNN as a feature extraction layer yielded optimal ROC-AUC results, achieving values of 93.30% and 93.00%, which reflect an improvement of 0.51% and 4.46% over the baseline model. These findings indicate that adding feature extraction layers significantly enhances MLP performance in sentiment classification tasks. Future research may explore the potential of using alternative neural networks as feature extractors to continue advancing MLP capabilities in complex NLP applications.