•  
  •  
 

Scientific Information Research

Keywords

knowledge-based question answering; knowledge graph; representation learning; deep learning; machine learning

Abstract

[Purpose/significance] By combining knowledge graph representation learning and word representation learning, this paper explores the question answering model based on knowledge graph, so as to realize the utilization of structured knowledge with high accuracy and wide coverage in the knowledge graph. [Method/process] This paper proposes a knowledge-based QA system, a framework integrating knowledge graph representation learning and word representation learning, and uses comparative and empirical research to explore the influence of different representation learning models and network structure on the effect of knowledge-based QA system. Firstly, the knowledge graph representation learning algorithm is used to generate the vector representation of entities and relations. Secondly, the generated entity and relation vectors are used as supervisory signals to train the vector representation of the problem. Finally, the best answer of the matching problem in the knowledge graph is selected by the triple representation generated by the problem. [Result/conclusion] The experimental results show that different representation learning model and network structure have a significant impact on the effect of knowledge-based question answering. Compared with the baseline method, this method can significantly improve the effect of knowledge base question answering. The research plays an important role in promoting the application of deep learning in the research of knowledge based question answering.

First Page

56

Last Page

70

Submission Date

October 2020

Revision Date

11-3-2020

Publication Date

January 2021

Digital Object Identifier (DOI)

10.19809/j.cnki.kjqbyj.2021.01.005

Reference

[1] MOLLA D, VICEDO J L. Question Answering in Restricted Domains: An Overview[J]. Computational Linguistics, 2007, 33 (01): 41-61.
[2] 欧石燕, 唐振贵.面向图书馆关联数据的自动问答技术研究[J]. 中国图书馆学报, 2015, 41(06): 44-60.
[3] BIZER C, LEHMANN J, KOBILAROV G, et al.DBpedia: A crystallization point for the Web of Data[J]. Web Semantics Science Services & Agents on the World Wide Web, 2009, 7 (03): 154-165.
[4] WMF.Wikidata [EB/OL] (2015-11-11)[2019-11-11]. https://www.wikidata.org/wiki/Wikidata:Main_Page.
[5] SUCHANEK F M, KASNECI G, WEIKUM G.YAGO: A Large Ontology from Wikipedia Andwordnet[J]. Web Semantics Science Services & Agents on the World Wide Web, 2007, 6 (03): 203-217.
[6] 顾琳.基于知识图谱的我国机构知识库研究[J].科技情报研究, 2020, 2 (04): 56-73.
[7] 陈涛, 刘炜, 单蓉蓉, 等.知识图谱在数字人文中的应用研究 [J]. 中国图书馆学报, 2019, 45 (06): 34-49.
[8] 廖开际, 黄琼影, 席运江.在线医疗社区问答文本的知识图谱构建研究[J/OL].情报科学, 1-9 (2020-07-31)[2020-10-04].
http://kns.cnki.net/kcms/detail/22.1264.G2.20200730.1239.010.html.
[9] 陆伟, 戚越, 胡潇戈, 等.图书馆自动问答系统的设计与实现[J].情报工程, 2019, 5 (02): 5-16.
[10] KIM Y. Convolutional Neural Networks for Sentence Classification[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP).Doha, Qatar: Association for Computational Linguistics, 2014, 1746-1751.
[11] CHANG S, ZHANG Y, HAN W, et al.Dilated Recurrent Neural Networks[J/OL].[2020-12-09].
https://www.researchgate.net/publication/320280030_Dilated_Recurrent_Neural_Networks, 2017.
[12] GRAVES A.Supervised Sequence Labelling with Recurrent Neural Networks: Long Short-Term Memory [M]. Berlin: Springer, 2012, 1735-1780.
[13] 罗达, 苏锦钿, 李鹏飞.基于多角度注意力机制的单一事实知识库问答方法[J].计算机科学, 2019, 46 (10): 215-221.
[14] PETERS M, NEUMANN M, IYYER M, et al. Deep contextualized word representations [C] //Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics. New Orleans: NAACL, 2018, 2227-2237.
[15] LUKOVNIKOV D, FISCHER A, LEHMANN J.Pretrained Transformers for Simple Question Answering Over Knowledge Graphs[C]//Proceedings of International Semantic Web Conference.Cham: Springer, 2019, 470-486.
[16] DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding [C] //Proceedings the North American Chapter of the Association for Computational Linguistics.New Orleans: NAACL, 2019, 4171-4186.
[17] GOLUB D, HE X D.Character-Level Question Answering with Attention [C] //Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Stroudsburg, PA: ACL Press, 2016, 1598-1607.
[18] YIN W P, YU M, XIANG B, et al.Simple Question Answering by Attentive Convolutional Neural Network [C] //Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics, New York: ACM, 2016, 1746-1756.
[19] LUKOVNIKOV D, FISCHER A, LEHMANN J. Neural Network-based Question Answering Over Knowledge Graphs on Word and Character Level[C]//Proceedings of International Conference on World Wide Web.Perth: International World Wide Web Conference.2017, 1211-1220.
[20] MOHAMMED S, SHI P, LIN J.Strong Baselines for Simple Question Answering Over Knowledge Graphs with and Without Neural Networks[C]//Proceedings of the North American Chapter of the Association for Computational Linguistics.New Orleans: NAACL, 2018, 291-296.
[21] 安波, 韩先培, 孙乐.融合知识表示的知识库问答系统[J].中国科学: 信息科学, 2018, 48 (11): 1521-1532.
[22] BORDES A, USUNIER N, GARCIA-DURAN A, et al. Translating Embeddings for Modeling Multirelational Data[C]//Proceedings of the Neural Information Processing Systems. Cambridge, MA: MIT Press, 2013 (26): 2787-2795.
[23] HUANG X, ZHANG J Y, LI D C, et al.Knowledge Graph Embedding Based Question Answering [C] //Proceedings of International Conference on Web Search and Data Mining. New York: ACM, 2019, 105-113.
[24] BORDES A, USUNIER N, CHOPRA S, et al. Large-scale Simple Question Answering with Memory Networks[P/OL] (. 2015-06-05)[2019-06-16].https://arxiv.org/pdf/1506.02075.pdf.
[25] BERANT J, CHOU A, FROSTIG R, et al.semantic parsing on freebase from question-answer pairs [C]//Proceedings of Empirical Methods in Natural Language Processing (EMNLP).Seattle: EMNLP, 2013, 1533-1544.
[26] UNGER C, FORASCU C, LOPEZ V, et al.Question answering over linked Data (QALD-4) [C] //Proceedings of Cross-Language Evaluation Forum.United Kingdom: CLEF, 2014: 1172-1180.
[27] DAI Z H, LI L, XU W. CFO: Conditional Focused Neural Question Answering with Large-scale Knowledge Bases [C] //Proceedings of the 54rd Annual Meeting of the Association for Computational Linguistics.Stroudsburg, PA: ACL Press, 2016, 800-810.
[28] DENG Y, XIE Y X, LI Y L, et al.Multi-Task Learning with Multi-View Attention for Answer Selection and Knowledge Base Question Answering[C]//Proceedings of the AAAI Conference on Artificial Intelligence.Menlo Park, CA: AAAI Press, 2019, 33: 6318-6325.
[29] HU S, ZOU L, ZHU Z X.How Question Generation Can Help Question Answering over Knowledge Base [C] //CCF International Conference on Natural Language Processing and Chinese Computing. Cham: Springer, 2019, 80-92.
[30] 周博通, 孙承杰, 林磊, 等.基于LSTM的大规模知识库自动问答[J].北京大学学报 (自然科学版), 2018, 54 (02): 286-292.
[31] 张涛, 贾真, 李天瑞, 等.基于知识库的开放领域问答系统[J].智能系统学报, 2018, 13 (04): 557-563.
[32] 余传明, 王曼怡, 林虹君, 等.基于深度学习的词汇表示模型对比研究[J].数据分析与知识发现, 2020, 4 (08): 28-40.
[33] MIKOLOV T, CHEN K, CORRADO G, et al.Efficient Estimation of Word Representations in Vector Space[C]//Proceedings of the International Conference on Learning Representations.New York: ACM, 2013, 1156-1165.
[34] PENNINGTON J, SOCHER R, MANNING C D. GloVe: Global Vectors for Word Representation [C] //Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Doha, Qatar: ACL, 2014, 1532-1543.
[35] BOJANOWSKI P, GRAVE E, JOULIN A, et al.Enriching Word Vectors with Subword Information [EB/OL]. (2017-06-19)[2018-09-07].https://arxiv.org/abs/1607.04606v2.
[36] PETERS M E, NEUMANN M, IYYER M, et al.Deep contextualized word representations[C] //Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics.New Orleans: NAACL, 2018, 2227-2237.
[37] RADFORD A, SALIMANS T.Improving Language Understanding by Generative Pre-Training[P/OL].[2019-10-13].
https://www.cs.ubc.ca/~amuham01/LING530/papers/radford2018improving.pdf
[38] ALEC R, JEFF W, REWON C, et al.Language Models are Unsupervised Multitask Learners[J/OL]. [2020-06-11]. https://d4mucfpksywv.cloudfront.net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf.
[39] LAN Z Z, CHEN M D, GOODMAN S, et al.ALBERT: A Lite BERT for Self-supervised Learning of Language Representations [EB/OL]. [2020-06-11]. https://arxiv.org/pdf/1909.11942.pdf
[40] LIU Y, OTT M, GOYAL N, et al.RoBERTa: A Robustly Optimized BERT Pretraining Approach [EB/OL]. [2020-06-11]. https://arxiv.org/pdf/1907.11692.pdf
[41] WANG Z, ZHANG J W, FENG J L, et al. Knowledge graph embedding by translating on hyperplanes [C]//Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence. Menlo Park, CA: AAAI Press, 2014: 1112-1119.
[42] LIN Y K, LIU Z Y, ZHU M S, et al.Learning entity and relation embeddings for knowledge graph completion [C]//Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence.Menlo Park, CA: AAAI Press, 2015, 2181-2187.
[43] JI G, HE S, XU L, et al.Knowledge Graph Embedding via Dynamic Mapping Matrix [C] //Proceedings of the Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing.Stroudsburg, PA: ACL Press, 2015, 687-696.
[44] NICKEL M, TRESP V, KRIEGEL H P.A Three-Way Model for Collective Learning on Multi-Relational Data [C] //Proceedings of the 28th International Conference on Machine Learning.New York: ACM, Press, 2011, 809-816.
[45] YANG B S, YIH W T, HE X D, et al. Embedding Entities and Relations for Learning and Inference in Knowledge Bases[C]//In Proceedings of International Conference on Learning Representations ICLR, 2015.
[46] TROUILLON T, WELBL J, RIEDEL S, et al. Complex Embeddings for Simple Link Prediction [C] //Proceedings of the International Conference on Machine Learning.New York: ICML, 2016, 2071-2080.
[47] KAZEMI S, POOLE D.Simple Embedding for Link Prediction in Knowledge Graphs [C] //Proceedings of Advances in Neural Information Processing Systems.Montréal: NIPS, 2018: 4284-4295.
[48] NICKEL M, ROSASCO L, POGGIO T.Holographic Embeddings of Knowledge Graphs [C] //Proceedings of AAAI Conference on Artificial Intelligence.Menlo Park, CA: AAAI Press, 2016, 1955-1961.
[49] DETTMERS T, MINERVINI P, STENETORP P, et al.Convolutional 2D knowledge graph embeddings [C] //Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence. Menlo Park, CA: AAAI Press, 2018, 1811-1818.
[50] BALAEVI I, ALLEN C, HOSPEDALES T M, et al.Hypernetwork Knowledge Graph Embeddings[P/OL]. (2018-08-21)[2020-06-11]. https://arxiv.org/pdf/1808.07018.pdf
[51] SCHLICHTKRULL M, KIPF T N, BLOEM P, et al.Modeling Relational Data with Graph Convolutional Networks [C] //Procedding of European Semantic Web Conference.Cham: Springer, 2018.
[52] KOTNIS B, NASTASE V.Analysis of the Impact of Negative Sampling on Link Prediction in Knowledge Graphs [EB/OL] (.2018-05-02) [2020-06-11]. https://arxiv.org/pdf/1708.06816v2.pdf.
[53] 余传明, 王峰, 安璐.基于深度学习的领域知识对齐模型研究: 知识图谱视角[J].情报学报, 2019, 38 (06): 641-654.
[54] BOLLACKER K, COOK R, TUFTS P. Freebase: A Shared Database of Structured General Human Knowledge [C] //Proceedings of the 22nd AAAI Conf on Artificial Intelligence. Menlo Park, CA: AAAI, 2007, 1962-1963.
[55] AKBIK ALAN, BLYTHE DUNCAN, VOLLGRAF ROLAND.Contextual String Embeddings for Sequence Labeling [C] //Proceeding of the 27th International Conference on Computational Linguistics, Santa Fe: COLING, 2018: 1638-1649.

Download

Share

COinS