Co-Attentions are highly effective attention mecha- nisms for text matching applications. Co-Attention enables the learning of pairwise attentions, i.e., learning to attend based on computing word-level affinity scores between two documents. However, text matching problems can exist in either symmet- rical or asymmetrical domains. For example, para- phrase identification is a symmetrical task while question-answer matching and entailment classifi- cation are considered asymmetrical domains. In this paper, we argue that Co-Attention models in asymmetrical domains require different treatment as opposed to symmetrical domains, i.e., a concept of word-level directionality should be incorporated while learning word-level similarity scores. Hence, the standard inner product in real space commonly adopted in co-attention is not suitable. This paper leverages attractive properties of the complex vec- tor space and proposes a co-attention mechanism based on the complex-valued inner product (Hermi- tian products). Unlike the real dot product, the dot product in complex space is asymmetric because the first item is conjugated. Aside from modeling and encoding directionality, our proposed approach also enhances the representation learning process. Extensive experiments on five text matching bench- mark datasets demonstrate the effectiveness of our approach.
License type:
PublisherCopyrights
Funding Info:
Description:
Full paper can be downloaded from the Publisher's URL provided.