Exploring Transfer Learning for News Sentiment Classification: An In-depth Analysis

Abstract

In this paper, two approaches of transfer learning i.e. feature and fine-tuning from pre-trained models are analyzed for the downstream task of banking financial news sentiment classification. In the feature-based approach, the study assesses how well two distinct strategies perform in determining whether news events are neutral, positive, or negative sentiments. The first method, DistilBERT, uses a language representation that understands context fully, whereas the conventional technique, TF-IDF, does not. Both techniques are evaluated with a Random Forest classifier to see whether feature-based transfer learning from pre-trained contextualized embeddings DistilBERT performs better at sentiment classification. The results showed that DistilBERT was better at understanding the semantics of news events and achieved higher accuracy compared to TF-IDF. Moreover, the fine-tuning of the pre-trained BERT-base-uncased model using the same dataset, the accuracy of classification is further improved by 12% compared to the feature-based approach. The study found that fine-tuning BERT on the downstream task of banking news sentiment classification leads to an accuracy of 90%. Whereas the BERT-base-cased variant produces 88% accuracy which means that casing (capitalization) does not play a crucial role in classifying sentiments of banking financial news events.