•  
  •  
 

Scientific Information Research

Keywords

legalcase matching, Pre-trained model, legal text, semantic tent matching

Abstract

[Purpose/significance]This study aims to solve the problem of traditional short text matching models being difficult to apply to long text matching tasks such as legal case retrieval. [Method/process]For the task of legal case matching, this paper proposes a Legal Text Matching model based on RoFormer （LTMR）. In the coding layer, the legal case is encoded through the RoFormer model and the legal feature extractor. In the reasoning layer, the context and interactive information of long text are further extracted by using interactive attention and self-attention mechanisms. We conducted the empirical research by applying the proposed model to the CAIL2019-SCM dataset. [Result/conclusion]Compared to the baseline methods, the LTMR model achieved the best results. The research sheds light on promoting the application of legal case matching.

First Page

13

Last Page

13

Digital Object Identifier (DOI)

10.19809/j.cnki.kjqbyj.2023.03.002

Reference

[1] 北京市三中院课题组,齐晓丹,史智军,等.类案检索报告制作和运用机制研究[J].法律适用,2020(12):3-14.[2] HU WEIFENG,ZHAO SIWEN,ZHAO QIANG,et al.BERT_LF:A Similar Case Retrieval Method Based on Legal Facts[J].Wireless Communications and Mobile Computing,2022.[3] 高尚. 司法类案的判断标准及其运用[J].法律科学(西北政法大学学报),2020,38(01):24-35.[4] PENNINGTON J,SOCHER R,MANNING C.GloVe:global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing.Stroudsburg:Association for Computational Linguistics,2014:1532-1543.[5] JOULIN A,GRAVE E,BOJANOWSKI P,et al.FastText.zip:Compressing text classification models[J].(2016-12-12)[2022-06-24].https://arxiv.org/pdf/1612.03651.pdf.[6] 陈彦光,刘海顺,李春楠,等.基于刑事案例的知识图谱构建技术[J].郑州大学学报(理学版),2019,51(03):85-90.[7] 王君泽,马洪晶,张毅,等.裁判文书类案推送中的案情相似度计算模型研究[J].计算机工程与科学,2019,41(12):2193-2201.[8] 盛小平,唐筠杰.国内法律法规视角下的个人隐私保护研究[J].科技情报研究,2022,4(04):54-62.[9] 原旭,韩雪姣,陈志奎,等.多模态特征融合的裁判文书推荐方法[J].微电子学与计算机,2020,37(12):42-47.[10] 梁柱,沈思,叶文豪,等.基于结构内容特征的裁判文书自动推荐研究[J].情报学报,2022,41(02):167-175.[11] VU TRAN,MINH LE NGUYEN,SATOSHI TOJO,et al.Encoded summarization:summarizing documents into continuous vector space for legal case retrieval[J].Artificial Intelligence and Law,2020(28):441-467.[12] 陈润好. 公共文化机构参与非遗保护的职责、范畴和对象:基于法规条文的解析[J].图书情报知识,2019(06):59-67.[13] 曹磊,刘晓燕.类案检索应用的困境与破解:以助力法官裁决及文书撰写为视角[J].中国应用法学,2021(05):162-172.[14] 黄承慧,印鉴,侯昉.一种结合词项语义信息和TF-IDF方法的文本相似度量方法[J].计算机学报,2011,34(05):856-864.[15] HE B,OUNIS I.Term Frequency Normalisation Tuning for BM25 and DFR Models[J].European Conference on Information Retrieval,2005(3408):200-214.[16] GRIFFITHS T L,STEYVERS M.Finding scientific topics[J].Proceedings of the National Academy of Sciences of the United States of America,2004,101(suppl1):5228-5235.[17] MIKOLOV T,CHEN K,CORRADO G,et al.Efficient estimation of word representations in vector space [J/OL].(2013-09-07)[2022-06-24].https://arxiv.org/pdf/1301.3781.pdf.[18] Y.KIM.Convolutional Neural Networks for Sentence Classification[C]//(2014-09-03)[2022-06-24].https://arxiv.org/pdf/1408.5882v2.pdf.[19] TAN M,DOS SANTOS C,XIANG B,et al.Improved Representation Learning for Question Answer Matching[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.Stroudsburg:Association for Computational Linguistics,2016:464-473.[20] LU W P,ZHANG X,LU H M,et al.Deep hierarchical encoding model for sentence semantic matching[J].Journal of Visual Communication and Image Representation,2020(71):102794.[21] PETERS M E,NEUMANN M,IYYER M,et al.Deep contextualized word representations[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Stroudsburg:Association for Computational Linguistics,2018:2227-2237.[22] RADFORD A,NARASIMHAN K, SALIMANS T,et al. Improving language understanding by generative pre-training[J/OL].(2020-06-24)[2022-06-24].https://www.cs.ubc.ca/~amuham01/LING530/papers/radford2018 improving.pdf.[23] HU B T,LU Z D,LI H,et al.Convolutional neural network architectures for matching natural language sentences[C]//Proceedings of the Advances in Neural Information Processing Systems.Cambridge:MIT Press,2014:2042-2050.[24] CHEN Q,ZHU X D,LING Z H,et al.Enhanced LSTM for natural language inference[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Stroudsburg:Association for Computational Linguistics,2017:1657-1668.[25] HUANG W Y,QU Q,YANG M.Interactive knowledge-enhanced attention network for answer selection[J].Neural Computing and Applications,2020,32(15):11343-11359.[26] 贾旭东,王莉.基于多头注意力胶囊网络的文本分类模型[J].清华大学学报(自然科学版),2020,60(05):415-421.[27] XU S Y,EE S J,XIANG Y.Enhanced Attentive Convolutional Neural Networks for Sentence Pair Modeling[J].Expert Systems with Applications,2020,151(02):113384.[28] WANG Z,HAMZA W,FLORIAN R.Bilateral multi-perspective matching for natural language sentences[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence(IJCAI 2017),San Francisco:Morgan Kaufmann Press,2017:4144-4150.[29] KIM S,KANG I,KWAK N.Semantic Sentence Matching with Densely-Connected Recurrent and Co-Attentive Information[C]//Proceedings of the 33th AAAI Conference on Artificial Intelligence,Palo Alto:AAAI,2019:6586-6593.[30] YANG R Q,ZHANG J H,GAO X,et al.Simple and Effective Text Matching with Richer Alignment Features[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.Stroudsburg:Association for Computational Linguistics,2019:4699-4709.[31] DEVLIN J,CHANG M W,LEE K,et al.BERT:pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Stroudsburg:Association for Computational Linguistics,2019:4171-4186.[32] SUN Y,WANG S,LI Y,et al.ERNIE 2.0:A continual pre-training framework for language understanding[C]//In Proceedings of the 34th AAAI Conference on Artificial Intelligence.Palo Alto:AAAI,2020:8968-8975.[33] ZENG D,KANG L,CHEN Y,et al.Distant Supervision for Relation Extraction via Piecewise Convolutional Neural Networks[C]//Conference on Empirical Methods in Natural Language Processing.lisbon:Association for Computational Linguistics,2015:1753-1762.[34] 陈志奎,刘杰,丁锋,等.基于案例推理的民间借贷案件适用法律推荐[J].计算机技术与发展,2021,31(05):198-203.[35] 梁鸿翔,吴肇良,杨帅.面向司法案件判定的知识引导智能分析系统[J].数据通信,2021(01):28-32,47.[36] TRAN V,NGUYEN M L,TOJO S,et al.Encoded summarization:summarizing documents into continuous vector space for legal case retrieval[J].Artificial Intelligence and Law,2020,28(03):441-467.[37] HONG Z,ZHOU Q,ZHANG R,et al.Legal Feature Enhanced Semantic Matching Network for Similar Case Matching[C]//2020 International Joint Conference on Neural Networks(IJCNN).Glasgow:IEEE,2020.[38] SHAO Y,MAO J,LIU Y,et al.BERT-PLI:Modeling Paragraph-Level Interactions for Legal Case Retrieval[C]//Twenty-Ninth International Joint Conference on Artificial Intelligence and Seventeenth Pacific Rim International Conference on Artificial Intelligence.Yokohama:2020:3501-3507.[39] ALTHAMMER S,HOFSTÄTTER S,HANBURY A.Cross-Domain Retrieval in the Legal and Patent Domains:A Reproducibility Study[C]//European Conference on Information Retrieval.Springer,Cham,2021.[40] KIM M Y,GOEBEL R,KANO Y,et al.COLIEE-2016:Evaluation of the Competition on Legal Information Extraction and Entailment[C/OL]//International Workshop on Juris-informatics.(2016-11-14)[2022-06-24].https://sites.ualberta.ca/~miyoung2/Papers/2016_COLIEE.pdf.[41] LOCKE D,ZUCCON G.A Test Collection for Evaluating Legal Case Law Search[C]//In Proceedings of SIGIR.Ann Arbor MI:2018.[42] MA Y,SHAO Y,WU Y,et al.LeCaRD:A Legal Case Retrieval Dataset for Chinese Law System[C]//SIGIR'21:The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval.ACM:2021.[43] XIAO C,ZHONG H,GUO Z,et al.CAIL2019-SCM:A Dataset of Similar Case Matching in Legal Domain[EB/OL].(2019-11-25)[2022-06-24].https://arxiv.org/pdf/1911.08962.pdf.[44] 王文广,陈运文,蔡华,等.基于混合深度神经网络模型的司法文书智能化处理[J].清华大学学报(自然科学版),2019,59(07):505-511.[45] WANG P F,FAN Y,NIU S Z,et al.Hierarchical Matching Network for Crime Classification[C]//the 42nd International ACM SIGIR Conference.ACM,2019.[46] HAI YE,XIN JIANG,ZHUNCHEN LUO,et al.Interpretable Charge Predictions for Criminal Cases:Learning to Generate Court Views from Fact Descriptions[J/OL].(2018-02-23)[2022-06-24].https://arxiv.org/pdf/1802.08504.pdf[47] ZHONG H X,GUO Z P,TU C H,et al.Legal Judgment Prediction via Topological Learning[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels:2018.[48] 刘宗林,张梅山,甄冉冉,等.融入罪名关键词的法律判决预测多任务学习模型[J].清华大学学报(自然科学版),2019,59(07):497-504.[49] 潘瑞东,孔维健,齐洁.基于预训练模型与知识蒸馏的法律判决预测算法[J].控制与决策,2022,37(01):67-76.[50] YANG S X,TONG S X,ZHU G X,et al.MVE-FLK:A multi-task legal judgment prediction via multi-view encoder fusing legal keywords[J].Knowledge-Based Systems,2022,239(05):107960.[51] ZHONG H X,ZHANG Z Y,LIU Z Y,et al.Open Chinese Language Pre-trained Model Zoo[EB/OL].(2020-03-18)[2022-06-24].https://github.com/thunlp/openclap.[52] SU J L,LU Y,PAN S F,et al.RoFormer:Enhanced Transformer with Rotary Position Embedding[J/OL].(2022-08-09)[2023-03-23].https://arxiv.org/pdf/2104.09864v4.pdf.[53] LIANG X B,WU L J,LI J T,et al.R-Drop:Regularized Dropout for Neural Networks[C]Proceedings of Neural Information Processing Systems(NeurIPS),Cambridge:MIT Press,2021:1-21.[54] ZHONG H X,XIAO C J,TU C C,et al.How Does NLP Benefit Legal System:A Summary of Legal Artificial Intelligence[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics,2020.

Download

Share

COinS