Publications
Fraunhofer SIT at CheckThat! 2023: Can LLMs Be Used for Data Augmentation & Few-Shot Classification? Detecting Subjectivity in Text Using ChatGPT
| Author | Frick, Raphael |
|---|---|
| Date | 2023 |
| Type | Conference Paper |
| Abstract | The fight against the spread of misinformation and rumors on the Internet has become a difficult issue lately. In some cases, it is difficult to tell whether a news article published on the Internet contains opinions or was written objectively. This year’s CheckThat! 2023 Task 2 dealt with the recognition of such texts. Due to the recent rise of large language models, this work analyzed the extent to which large language models such as ChatGPT can be used to augment unbalanced data sets and whether they can serve as a reliable few-shot classifier. The proposed approaches were trained and evaluated on the English and German subtasks of the challenge. While the models trained with the augmented data were unable to outperform the BERT models trained without the additional data, the few-shot classification scheme was able to outperform across different data set splits, most notably with the English test set. On the private test sets, the proposed ChatGPT-based few-shot classifiers achieved an ? 1 value of 0.73 on the English data and an ? 1 value of 0.68 on the German data. However, they have not been shown to achieve stable performance over multiple data set splits. |
| Conference | Conference and Labs of the Evaluation Forum 2023 |
| Url | https://publica.fraunhofer.de/handle/publica/462594 |
