题名: | 网络舆情治理中的仇恨迷因检测增强方法研究 |
作者: | |
学号: | SX2209020 |
保密级别: | 公开 |
语种: | chi |
学科代码: | 120100 |
学科: | 管理学 - 管理科学与工程(可授管理学、工学学位) - 管理科学与工程 |
学生类型: | 硕士 |
学位: | 管理学硕士 |
入学年份: | 2022 |
学校: | 南京航空航天大学 |
院系: | |
专业: | |
研究方向: | 大数据分析 |
导师姓名: | |
导师单位: | |
完成日期: | 2025-03-17 |
答辩日期: | 2025-03-13 |
外文题名: |
Research on Enhanced Methods for Multimodal Hateful Meme Detection in Online Public Opinion Governance |
关键词: | |
外文关键词: | Hateful Meme Detection ; Public Opinion Governance ; Internet Meme ; Multimodality ; Feature Enhancement |
摘要: |
随着互联网快速普及,网络已成为人们表达观点和情感的重要途径。同时,一种结合了图像和文本的多模态载体——互联网迷因,因其幽默、简短而内涵丰富的特点而广受网民欢迎,在网络舆情中占据越来越高的比例。然而,其病毒式传播的特性也使包含仇恨内容的迷因(即仇恨迷因)成为威胁社会和谐的重要隐患,为网络舆情治理带来了挑战。为了降低仇恨迷因带来的社会影响,在仇恨迷因传播早期进行检测识别是治理仇恨迷因的关键问题之一,而传统的纯文本仇恨言论检测方法对于检测多模态的仇恨迷因效果不佳。研究针对仇恨迷因的自动化检测方法能够更准确地识别海量互联网舆情信息中的仇恨迷因,对网络舆情治理有重要意义。 本文首先分析仇恨迷因检测的挑战性和重要性,梳理仇恨言论检测、互联网迷因以及仇恨迷因检测的研究现状。总结现有仇恨迷因检测方法存在的不足,并针对现有研究的不足提出了两种仇恨迷因检测的特征增强方法。本文的主要研究内容如下: (1)针对检测模型提取仇恨检测相关的迷因图像高层语义特征困难的问题,提出了融合领域图像实体的仇恨迷因检测增强方法。以数据驱动的方式从仇恨言论语料中进行实体发现和实体精炼,构建了与仇恨检测任务相关的高质量的感兴趣实体集合。之后,利用CLIP模型的零样本学习能力从感兴趣实体集合中匹配迷因图像出现的领域图像实体。领域图像实体包含迷因图像中与仇恨检测相关的实体元素,从而帮助检测模型理解迷因图像中与仇恨检测相关的高层语义信息,增强仇恨迷因检测性能。在公开仇恨迷因数据集上对多种检测模型进行对比实验,对比基线模型,在文本的RoBERTa-base模型提高了0.122的AUC分数,在视觉-语言模型BridgeTower-base上提高了0.030的AUC分数。实验结果表明融合领域图像实体对多种检测模型都有增强效果,而且增强效果优于通用图像实体特征,实验验证了方法的稳定性和有效性。 (2)在融合领域图像实体提升检测模型高层语义特征提取能力的基础上,针对检测模型缺乏检测仇恨迷因必要的背景信息的问题,提出了融合实体背景关系的仇恨迷因检测增强方法。将仇恨迷因检测任务所需的背景知识建模为可能存在仇恨的实体间的背景关系信息,通过大语言模型的提示学习和检索增强生成技术为迷因提取实体背景关系。由于大语言模型只能处理文本模态的信息,复用了领域图像实体提取的模块,将迷因图像中的实体转为文本模态的领域图像实体从而囊括了迷因图像中的重要视觉元素。先通过提示学习驱动大语言模型从迷因文本和领域图像实体中找出可能存在仇恨关系的实体集合。之后再通过检索增强生成技术,从实体背景知识库中召回相关的片段生成实体关系信息。在公开仇恨迷因数据集上以视觉-语言模型BridgeTower-base作为基线模型在多种实验设置下进行对比实验。对比实验的结果表明,融合实体背景关系后检测模型相较于基线检测模型有显著的性能提升,尤其是在融合实体背景关系特征后仅使用50%的训练数据可以取得媲美基线模型使用全量数据训练的AUC分数和正确率,证明了融合实体背景关系方法的有效性。 最后,结合本文研究内容总结了对网络舆情治理的管理启示,对不同管理角色提出了治理仇恨迷因的对策建议,并对本文研究的局限以及未来可改进的研究方向进行了展望。 |
外摘要要: |
With the rapid proliferation of the internet, online platforms have become a significant avenue for people to express their opinions and emotions. Meanwhile, internet memes, a multimodal medium combining visual and textual modalities, have gained widespread popularity among netizens due to their humor, brevity, and rich connotations, playing an increasingly prominent role in online public opinion. However, their characteristic of rapid and wide propagation has also made memes containing hateful content (i.e., hateful memes) a significant threat to social harmony, posing challenges to the governance of online public opinion. To mitigate the societal impact of hateful memes, early detection and identification during their initial spread is a key issue in combating them. Traditional text-based hate speech detection methods, however, are less effective at identifying multimodal hateful memes. Research on automated detection methods for hate memes can more accurately identify such content from vast amounts of online information, playing a critical role in the governance of online public opinion. This thesis begins by analyzing the challenges and significance of hate meme detection, reviewing the current researches on hate speech detection, internet memes, and hateful meme detection. It summarizes the limitations of existing hate meme detection methods and proposes two feature enhancement approaches to address these shortcomings. The main research contributions of this thesis are as follows: (1) To address the challenge of extracting high-level semantic features related to hateful meme detection from meme's images, this thesis proposes an enhanced hateful meme detection method that integrates domain-specific visual entities. A data-driven approach is used to discover and refine entities from hate speech corpora, resulting in a high-quality collection of task-relevant entities. Leveraging the zero-shot learning capability of the CLIP model, domain-specific image entities present in meme's image are matched from this collection. These extracted entities contain elements relevant to hate detection within memes, aiding detection models in understanding high-level semantic information associated with hate memes and improving their detection performance. A comparative experiment was conducted on multiple detection models using a publicly available hateful meme dataset. Compared to the baseline models, the RoBERTa-base model for text achieved an AUC score improvement of 0.122, while the BridgeTower-base vision-language model improved by 0.030. The experimental results demonstrate that incorporating domain-specific visual entities enhances various detection models, with a greater enhancement effect than general visual entity features. The results further validate the stability and effectiveness of the proposed approach. (2) Based on improving the high-level semantic feature extraction capabilities of detection models by incorporating domain-specific visual entities, an enhanced method for detecting hate memes by integrating entity-background relationships is proposed to address the issue that the detection model lacks the necessary background information for detecting hateful memes. Specifically, the background knowledge required for hateful meme detection is modeled as contextual relationship information between entities potentially involved in hateful content. Using prompt learning and retrieval augmented generation techniques in large language models (LLMs), the method extracts entity background relationships from memes. Since LLMs can only process textual data, the domain-specific visual entity extraction module is reused to convert visual entities in meme images into textual entities, ensuring that critical visual elements in memes are included. First, prompt learning is utilized to guide the LLM in identifying sets of entities with potential hate relationships from meme text and domain-specific visual entities. Subsequently, retrieval-augmented generation techniques are employed to retrieve relevant fragments from an entity background knowledge base and generate entity relationship information. A comparative experiment was conducted under various experimental settings using the vision-language model BridgeTower-base as the baseline on a publicly available hateful meme dataset. The results indicate that integrating entity-background relationships significantly improves the performance of detection models compared to the baseline. Notably, after incorporating entity-background relationships, the model achieves AUC scores and accuracy comparable to those of the baseline model trained on the full dataset, even when using only 50% of the training data. This demonstrates the effectiveness of the entity-background relationships integration approach. Finally, based on the research content of this thesis, the management implications for online public opinion governance are summarized. Countermeasures and suggestions for different management roles to mitigate the issue of hateful memes are proposed, along with a discussion of the study's limitations and potential directions for future research. |
参考文献: |
[1] 中国互联网络信息中心. 第54次《中国互联网络发展状况统计报告》[R]. 2024. [4] 李华君, 曾留馨, 滕姗姗. 网络暴力的发展研究:内涵类型、现状特征与治理对策[J]. 情报杂志, 2017, 36(9): 139-145. [5] 刘美萍. 重大突发事件网络舆情协同治理机制构建研究[J]. 求实, 2022(5): 64-76+111. [6] 郭小宇, 马静, Zubiaga A, 等. 互联网迷因研究:现状与展望[J]. 情报理论与实践, 2021, 44(6): 199-207. [16] 曾江峰, 高鹏钰, 李玲, 等. 基于BERT和提示学习的网络暴力言论识别研究[J]. 情报杂志: 1-9. [18] 王海涵, 朱焱. 融合反讽机制的攻击性言论检测[J]. 计算机应用, 2024, 44(4): 1065-1071. [19] 王小龙, 王琰慧, 张顺香, 等. 融合帖文属性的性别歧视言论检测模型[J]. 计算机科学, 2024, 51(6): 338-345. [20] 叶瀚, 胡凯茜, 李欣, 等. 基于语义不一致性的网络暴力舆情预警方法[J]. 情报杂志, 2024, 43(4): 135-145+67. [25] 徐磊, 胡亚豪, 陈满, 等. 融合前缀调优和提示学习的仇恨言论检测方法[J]. 计算机科学与探索: 1-13. [27] Dawkins R. The selfish gene[M]. Oxford university press, 1976. [34] 张頔. 互联网迷因的流行病传播模型研究[D]. 天津科技大学, 2018. [53] 刘旭东, 杨亮, 张冬瑜, 等. 结合图卷积网络的多模态仇恨迷因识别研究[J]. 重庆理工大学学报(自然科学), 2024, 38(1): 169-179. [59] 林燕霞, 谢湘生, 张德鹏. 复杂交互行为影响下的网络舆情演化分析[J]. 中国管理科学, 2020, 28(1): 212-221. [60] 杨兴坤, 廖嵘, 熊炎. 虚拟社会的舆情风险防治[J]. 中国行政管理, 2015(4): 16-21. [91] Chase H. LangChain[Z/OL]. (2022-10). https://github.com/langchain-ai/langchain. |
中图分类号: | TP391 |
馆藏号: | 2025-009-0035 |
开放日期: | 2025-09-27 |