15. Pairwise Input

15.1 Overview

NLP领域的任务的输入输出有以下几种：

除7.2外，其他模型结构都是或类似于双胞胎网络(Siamese Network)，2个网络的结构是完全一致的，但其参数，有时共享，有时不同？

Paper

• Neural Network Models for Paraphrase Identification, Semantic Textual Similarity, Natural Language Inference, and Question Answering - Ohio2018

  研究的模型有：InferSent, SSE, DecAtt, ESIM, PWIM

  Article: 基于神经网络模型的释义识别、语义文本相似性、自然语言推理和问题回答

Practice

• https://github.com/brightmart/nlu_sim (Tensorflow)

  all kinds of baseline models for modeling tasks with pair of sentences: semantic text similarity(STS), natural language inference(NLI), paraphrase identification(PI), question answering(QA)

  模型有：DualTextCNN, DualBiLSTM, DualBiLSTMCNN, ESIM, SSE, BiLSTM with Attention

15.2 BiLSTMTextRelation

Structure: Input(Seq EOS Seq) -> Embeddding -> BiLSTM -> Average -> Softmax

Same with TextRNN, but input is special designed.

e.g. input: "How much is the computer ? EOS Price of laptop", where 'EOS' is a special token splitted input1 and input2

Code

• https://github.com/brightmart/text_classification (Tensorflow)

15.3 twoCNNTextRelation - OK

Structure: (Input1 -> Embedding -> TextCNN) * 2 -> Concatenate -> Softmax

产品词关系项目中使用的模型与此类似，在此基础上增加了第3个Input（结构化输入）。

Code

• https://github.com/brightmart/text_classification (Tensorflow)

15.4 BiLSTMTextRelationTwoRNN

又叫 Dual Encoder LSTM Network ?

The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems-2016

Structure: Seq1(Input1 -> Embedding -> BiLSTM) + Seq2(Input2 -> Embedding -> BiLSTM) -> Dot Product -> Softmax

Dot Product作用：To measure the similarity of the predicted response r' and the actual response r by taking the dot product of these two vectors. A large dot product means the vectors are similar and that the response should receive a high score. We then apply a sigmoid function to convert this score into a probability. Similarity --> Probability

Article

• Deep Learning for Chatbots, Part 2 – Implementing a Retrieval-Based Model in Tensorflow

15.5 Others

Paper

• Pairwise relation classification with mirror instances and a combined convolutional neural network - Singapore2016

  Code: https://github.com/jefferyYu/Pairwise-relation-classification (Torch, Lua)