一种轻量级文本蕴含模型
作者:
作者单位:

1.中国电子科技网络信息安全有限公司;2.四川大学计算机学院;3.卫士通信息产业股份有限公司

作者简介:

通讯作者:

中图分类号:

TP391

基金项目:

四川省新一代人工智能重大专项(2018GZDZX0039);四川省重点研发项目(2019YFG0521);JG2020125


A lightweight text entailment model
Author:
Affiliation:

1.China Electronic Technology Cyber Security Co,Ltd;2.College of Computer Science,Sichuan University;3.Westone Information Industry INC

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    现有主流文本蕴含模型大多采用循环神经网络编码,并采用各种注意力推理机制或辅以手工提取的特征来提升蕴含关系识别准确率,由于复杂的网络结构和RNNs网络串行机制导致这些模型训练和推理速度较慢.本文提出轻量级文本蕴含模型,采用自注意力编码器编码文本向量,点积注意力交互两段文本,再采用卷积神经网络对交互特征推理,整个结构可根据不同数据的推理难度叠加不同模块数量.在多个文本蕴含数据集实验表明,本文模型在保持较高识别准确率情况下仅用一个块参数仅为665K,模型推理速度相比其他主流文本蕴含模型至少提升一倍.

    Abstract:

    Most of the existing mainstream textual entailment models adopt recurrent neutral network to encode text, and various complex attention mechanisms or manually extracted text features are used to improve the accuracy of textual entailment recognition. The training and inference speed of the models is usually slow due to the complex network structure and the sequential nature of RNNs. In this paper, Lightweight Text Entailment Model is proposed. In the proposed model, the self-attentional encoder is adopted to encode text vectors; the dot product attention mechanism is adopted to interact two texts; the convolutional neural network is adopted to deduce interactive features, and the module number of the structure can be adjusted according to the reasoning difficulty of data. Experiments on multiple datasets show that the parameter size of single module in the model is only 665 K, and the inference speed of the model is at least twice as high as that of other mainstream models, under the condition of high accuracy.

    参考文献
    相似文献
    引证文献
引用本文

引用本文格式: 王伟,孙成胜,伍少梅,张芮,康睿,李小俊. 一种轻量级文本蕴含模型[J]. 四川大学学报: 自然科学版, 2021, 58: 052001.

复制
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2021-06-28
  • 最后修改日期:2021-07-15
  • 录用日期:2021-07-16
  • 在线发布日期: 2021-10-18
  • 出版日期: