Paper 4
Constituency-Informed and Constituency-Constrained Extractive Question Answering with Heterogeneous Graph TransformerAuthors: Mingzhe Du, Mouad Hakam, See-Kiong Ng, Stéphane Bressan |
AbstractLarge neural language models are achieving exceptional performance in question answering and other natural language processing tasks. However, these models can be costly to train and difficult to interpret. In this paper, we propose to investigate whether incorporating explicit linguistic information can boost model performance while improving model interpretability. We present a novel constituency-informed and constituency-constrained question answering model called SyHGT-CN. The linguistics-informed model integrates the symbolic information contained in constituency trees with the statistical knowledge of a neural language model. The integration of the linguistics graphic structures with the transformer-based neural language model is achieved by the adjunction to the latter of a heterogeneous graph neural network, in which the former is encoded. We comparatively and empirically show, with the SQuAD2.0 benchmark, that the proposed approach is more accurate than a constituency-oblivious BERT and the constituency-informed SyHGT-C model. Keywords: Question answering, Linguistics-informed natural language processing, Transformer, Graph neural network. |