题名: | 基于大模型的角色对话生成技术研究 |
作者: | |
学号: | SX2216090 |
保密级别: | 公开 |
语种: | chi |
学科代码: | 081200 |
学科: | 工学 - 计算机科学与技术(可授工学、理学学位) - 计算机科学与技术 |
学生类型: | 硕士 |
学位: | 工学硕士 |
入学年份: | 2022 |
学校: | 南京航空航天大学 |
院系: | |
专业: | |
研究方向: | 自然语言处理 |
导师姓名: | |
导师单位: | |
完成日期: | 2025-01-07 |
答辩日期: | 2025-03-12 |
外文题名: |
Research on Role-playing Dialogue Generation based on Large Language Models |
关键词: | |
外文关键词: | Natural Language Processing ; Large Language Model ; Role-playing ; Few-shot Learning ; Hallucination Mitigation |
摘要: |
大型语言模型的发展极大地增强了角色扮演聊天机器人的性能,推动了角色扮演任务在技术层面的进步。目前,研究者都在努力为聊天机器人赋予独特的人物性格与角色属性。在学术领域,与大模型角色扮演相关的研究工作数量很少,缺乏相应的评测指标和技术方案。在商业领域,尽管已经有一些用户量较多的商业产品,但他们没有公开技术报告或数据集。此外,角色扮演任务对大模型在背景知识准确性和语言风格一致性上提出了更高要求,而大模型在进行对话时会出现事实性有误或是回答笼统、缺乏个性的幻觉回答问题,降低了用户的体验。具体来说,本文的主要研究内容如下: (1)针对角色扮演领域缺乏数据集和评测指标的问题,本文首先构建了包含106个角色的角色扮演任务的数据集,采集自维基百科的知名人物信息,包括背景知识语料库与语言风格语料库,并设计了背景知识准确性和语言风格一致性两个关键评估指标,为大模型在角色扮演领域的表现评估提供量化依据。基于该数据集,探索了零样本学习、少样本学习以及不同微调方法实验设定下对大模型的角色扮演任务上的性能影响。实验表明,目前大模型在背景知识准确性和语言风格一致性上都有提升空间。 (2)针对大模型出现事实性错误、回答笼统缺乏语言风格的幻觉回答的问题,本文设计了背景知识准确性增强和语言风格一致性增强两个模块进行幻觉的缓解。为了增加回答事实性,构建了零样本思维链提示词和少样本思维链提示词引导模型逻辑思考,帮助模型更好理解问题以及提取关键信息,提升背景知识准确性,从而减少事实性幻觉;为了增强回复的语言风格,采用基于职业分类训练并调整输出嵌入层的方式,改变模型输出语气,针对不同职业进行语言风格增强,提升模型回复语言风格一致性,从而减少了语言风格幻觉。实验表明,融合了背景知识准确性增强模块和语言风格一致性增强模块的方法能够提高模型在角色扮演领域的性能。 综上所述,本文为大模型在角色扮演任务中的应用提供了丰富的数据资源、自动化的评估方法以及有效的技术方案,为进一步优化大模型的角色扮演能力打好了基础。 |
外摘要要: |
The development of large language models (LLMs) has significantly enhanced the performance of role-playing chatbots, pushing technological advancements in role-playing tasks. Currently, researchers are striving to endow chatbots with unique character personalities and role attributes. In academic research area, there is a scarcity of research related to role-playing with LLMs, with a lack of evaluation metrics and technical frameworks. In the commercial area, although there exists some commercial products, the lack of detailed technical reports and publicly available datasets has hindered academic progress in the role-playing field. Additionally, role-playing tasks impose higher demands on LLMs in terms of background knowledge consistency and linguistic style consistency. However, hallucinations answers like factual errors and generic or personality-lacking responses reduce the user experience. Specifically, the main contributions of this paper are as follows: (1) To address the lack of datasets and evaluation metrics for the role-playing area, this study first constructed a role-playing dataset containing 106 characters sourced from well-known figures on Wikipedia, including background knowledge corpora and linguistic style corpora. Background knowledge accuracy and linguistic style consistency are also designed to evaluate model performance. Using this dataset, this paper explored the effects of zero-shot learning, few-shot learning, and different fine-tuning methods on LLM performance in role-playing tasks. Experiments show that there is still room for improvement in both background knowledge and language styles consistency for LLMs. (2) To mitigate hallucinatory responses like factual errors, generic answers, and inconsistent styles of the LLMs, this paper designed two modules, namely the module for enhancing the accuracy of background knowledge and the module for enhancing the consistency of language styles, to address these challenges. To improve the factual accuracy of the responses, this paper developed zero-shot chain-of-thought prompts and few-shot chain-of-thought prompts to guide logical reasoning, helping models better understand questions and extract key information, thereby reducing factual hallucinations. To enhance the language style of responses, this paper employed occupation-classification training and adjusted the output embedding layer to alter the output distribution of the model. This improves style consistency for different occupations and diminishes style-related hallucinations. Experiments demonstrate that the proposed approach, which integrates both enhancement modules, enhances LLM performance in role-playing tasks. In summary, this paper provides extensive data resources, automatic evaluation methods, and effective technical solutions for the application of LLMs in role-playing tasks, paving way for further optimizing their role-playing capabilities. |
中图分类号: | TP391 |
馆藏号: | 2025-016-0025 |
开放日期: | 2025-09-28 |