Multi-hop Question Answering

Creating a dataset for multi-hop question answering is more challenging than for a traditional (i.e., single hop) QA setup. The major challenges include:

• •

  Contexts used for questions should form a valid and unambiguous reasoning chain. This implies that a context used for a particular question should have some information overlap or some kind of entailment relation with at least one of the other contexts present in the expected reasoning chain [DP1].

• •

  Since the questions should require reasoning over multiple contexts, the process of question generation itself needs to somehow encapsulate information across those particular contexts.

• •

  The creation process needs to ensure that the question is not answerable by using any single context. For instance, the question “Who was the president of United States in the year in which World War II began?” requires two contexts containing “World War II began in 1939.” and “Franklin D. Roosevelt was the president of United States in 1939.” However, there may be a separate context containing “Franklin D. Roosevelt, president of United States at the start of World War II, was unwilling to…”. This challenge is more prevalent in the setting of Open Domain QA because of the incomplete prior knowledge about the possible contexts.

Therefore, it becomes crucial to look at the methods adopted during creation of existing datasets, along with their advantages and drawbacks and possible ways to mitigate these. In general, the dataset creation task can involve three major steps, which we discuss next.

1. 1.

   Generating reasoning chains

2. 2.

   Question generation by crowd-sourcing

3. 3.

   Automatic/manual filtering

Methods for coming up with reasoning chains are highly specific to the task and domain and thus, differ significantly for each dataset. Thus, we look at a few datasets individually.

1. 1.

   HotpotQA [DP1]: HotpotQA contains only 2-hop questions formed using the first passages of documents from the English Wikipedia dump¹¹1https://en.wikipedia.org/. Two passages are chosen as a reasoning chain (termed candidate passage pair) if they satisfy either of the two conditions:

   • •

     There exists a hyperlink from the first document to the second. The entity which forms the hyperlink is termed as the bridge entity and the questions are termed as bridge questions.

   • •

     The entities for those passages belong to the same category (e.g. Michael Jordan and Kobe Bryant). These are specifically sampled from 42 manually created lists. Such pairs are used for creating comparison questions.

2. 2.

   MultiHop-RAG [LL23]: MultiHop-RAG uses a set of diverse news articles and prompts GPT-4 to extract factual sentences from each article. These articles are again passed to GPT-4 for paraphrasing the factual sentence into a natural language claim. A topic and an entity are also generated in this step which are used as bridge entities for the reasoning chains.

3. 3.

   HybridQA [DP6]: In HybridQA, the definition of a context can be either a passage or a table. It is argued that using a table as context avoids the ambiguity of the questions that could arise when using texts. To ensure at least two hops, the questions are restricted to have reasoning chains containing at least one table and one text document. The tables are filtered from the set of tables released in WikiTables [bhagavatula2013methods], and the Wikipedia hyperlinks present in the cells of the table are used to retrieve relevant passages. At most 12 sentences of the first passage of each Wikipedia page are treated as the passages.

4. 4.

   NarrativeQA [DP3]: The dataset is for answering multi-hop questions on long stories. To ensure that the questions are indeed multi-hop and answering requires non localized reasoning, the annotators are asked to form questions using human-generated summaries of the stories. The stories are collected from books from Project Gutenberg²²2https://www.gutenberg.org/, while movie scripts are scraped from the web.

5. 5.

   OpenBookQA [DP4]: OpenBookQA requires question answering on the scientific domain using a book (in simpler words, a collection) of scientific facts along-with a broad common knowledge (large open-domain scientific sentences). The book and common knowledge are the two contexts required to answer the questions in OpenBookQA. Book is collected by filtering a subset of the WorldTree corpus identified by worldtree where the common knowledge is collected from a collection of 14M scientific facts across Wikipedia, ConceptNet and other scientific corpora.

6. 6.

   QASC [DP6]: QASC is also a dataset for two-hop QA using scientific facts. The process of generating reasoning chains is very similar to that of OpenBookQA. The two contexts of the reasoning chain are chosen from a set of good quality seed facts and a large corpus of auxiliary facts, respectively.

7. 7.

   MultiRC [DP5]: The purpose of this dataset is to create multi-domain multi-hop questions. Documents across various domains are selected from multiple corpora. Here, the multiple contexts are part of the same passage and the task of candidate reasoning chain generation is left to the annotators; each document is given to the annotators and they are asked to form valid multi-hop reasoning questions.

Generating multi-hop QA pairs requires accumulation of information across different contexts which itself is an unsolved problem [SP14, AP1, DP7, QG1, QG2, QG3, QG4, QG5, QG6].

The first step of the open domain retrieval (referred to preliminary retrieval from hereon) can be a single shot retrieval as done in single hop question answering i.e., the model attempts to extract all the relevant contexts in a single retrieval step. We call this strategy single-pass. Another solution would be to retrieve the passages iteratively, where each iteration of retrieval can correspond to each hop in the reasoning. We describe in detail some of these approaches below.

After getting an initial pool of retrieved contexts, many works suggest using a more fine-grained and more sophisticated retrieval on the retrieved contexts to get a better quality of contexts. Based on how the initial pool of retrieval output is processed, we can categorize the approaches into concatenation, supervised re-ranking and unsupervised re-ranking.

Concatenating/union Some of the methods do not filter the input contexts and directly pass a union or concatenation of the contexts to the reasoning model. For example:

• •

  SP7 maintain N-parallel reasoning chains in the preliminary retrieval step and a union of these chains is passed to the answer prediction module.

• •

  SP1, SP4 do not filter the passages and directly perform reasoning at a lower granularity.

Supervised re-ranking Majority of the methods use a supervised module to score and rank the input contexts, and then use some criterion to filter the less relevant documents before passing to the answer prediction module:

• •

  SP2 use a linear layer with sigmoid activation to get a relevance score of each sentence in a paragraph with the question. Max pooling across all sentences is applied to give a relevance score and top-k most relevant paragraphs are picked. Max pooling allows only one of the sentences in a paragraph to be relevant to the question with a high score.

• •

  SP3 use BERT to compute query-aware embeddings for every pair of the first hop and second hop paragraph. The two paragraphs are concatenated and fed to 2-layer Neural Network to get a score. Top $k$ pairs are passed to the reader module.

• •

  SP5 use a bi-LSTM layer [lstm] to predict relevance of each second hop passage with the question.

• •

  SP6 use a RoBERTa encoder [roberta] followed by a fine tuning layer to get the top N paragraphs.

• •

  SP8 pass the node (document) representations to a binary classifier for predicting their relevance and keep the $k$ highest scoring documents.

• •

  SP9 use RoBERTa [roberta] for reranking trained to predict the F-1 score of evidence chain.

• •

  SP16, SP20 concatenate paragraphs with the question and feed it to a BERT followed by a binary classifier and keep N(=3) paragraphs with the highest scores. SP25 follows a similar approach but set a threshold to retrieve a variable number of paragraphs.

• •

  SP10 use the gated memory flow network inspired by the Neural Turing Machine [graves2014neural] to model the probability of a paragraph being the next context in the reasoning chain, conditioned on the question and the previous paragraphs in the reasoning chain. At every time step $t$ BERT is used to compute the question aware embeddings of the paragraphs, $x_{t}$. A KVMemNN architecture models the memory as a set of key-value pairs. The model passes the key vectors and $x_{t}$ to linear layers and applies Softmax to the output of $W_{x}x_{t}\cdot W_{k}k_{i}$. The output after Softmax multiplied with another matrix $W_{v}$ is then used as weights while summing the value vectors $v_{i}$ to give the readout vector $o_{t}$. The memory reading process is similar to computing self-attention [transformer] and the authors concatenate the outputs after computing $o_{t}$ for the $h$ attention heads. $o_{t}$ and $x_{t}$ are passed to another linear layer with tanh and sigmoid activation to give the relevance score $s_{t}$. $x_{t}$ is written to the memory if $s_{t}>gate$ where $gate$ is a hyper-parameter. Hard negative examples are generated by training a BERT based model to predict the relevance score $s_{t}$ and choosing the top-8 of the non-evidence paragraphs.

• •

  SP23 propose generative context selection that learns to predict how the question would have been formed. Mathematically, the model tries to learn $p(a,q|C)$ instead of $p(a|q,C)$ (as in discriminative models). This probability is modeled as

  $$p(a,q|C)=\sum_{c_{ij}}p(a|q,c_{ij})\cdot p(q|c_{ij})\cdot p(c_{ij}|C)$$

  (4.1)

  where $p(c_{ij}|C)$ is the prior that computes compatibility between any two contexts, $p(q|c_{ij})$ is the question generation model that predicts the probability of $q$ being formed from the given contexts, and $p(a|q,c_{ij})$ is the standard answering model. During inference, the model retrieves the contexts as $c^{\ast}_{ij}=argmax_{c_{ij}}p(q|c_{ij})\cdot p(c_{ij}|C).$ To model these probabilities, a pre-trained T5 is used for obtaining the contextual embeddings. The prior and generative models are trained together. Concatenation of every pair of contexts is passed to an encoder and a Pointer Generator Network (PGN) [pgn] decoder is used to predict the question. The training objective is to increase the likelihood of the question for gold context pairs and the unlikelihood [welleck2019neural] for a sample set of negative context pairs. T5 is used for answering using the best pair of contexts. The advantage of using a generative model is that it avoids the annotator biases in the dataset. This hypothesis is tested by running the model on the adversarial set of questions formed by SP32. A multi-label sentence classifier $p(s|q,C)$ that selects relevant sentences is found to have a better performance but at the same time, is more biased.

• •

  DP6 feed each cell along with its neighboring cells to the cell encoder to obtain their representations. The representations are aggregated and further fed to a feed-forward neural network to obtain a score.

• •

  SP22 perform the retrieval in two steps where the first step retrieves a paragraph, and the second step retrieves a sentence from these paragraphs. A weighted sum of paragraph and sentence retrieval scores is computed to find the best retrieved sentence. The retrieval score of the paragraph retrieved from the first hop is broadcast to the sentences in the paragraphs. The best retrieved sentence fed to BERT-large for extractive QA.

Unsupervised re-ranking Some of the methods for supervised re-ranking include:

• •

  SP4 consider all $\binom{n}{k}(k\in[2,5])$ reasoning chains formed by the $n$ retrieved passages and use a simple formula to compute the scores. They define Relevance as the average BM25 scores of the chain, Overlap as the word overlap between each sentence pair in the chain and Coverage as the product of word overlap of sentence with the question and with the answer. The final score is given by $\frac{R\cdot C}{O}$. By experiments, $k=3,4$ are found to be the best values with the justification that getting two correct retrieved passages in a chain of size two is tough while a chain of size 5 will suffer from noise.

• •

  DP6 retrieve cells from the table in the HybridQA dataset in an unsupervised manner. A cell is selected if its value is either mentioned in the question, or is min/max of the corresponding column. A cell is also added if it hyperlinks to one of the passages that are retrieved by a TF-IDF retriever.

Critique: Supervised re-ranking can lead to a model that is more suited to the task at hand but requires some training signals which may not be available for certain tasks. On the other hand, unsupervised methods are more flexible but might not be optimized for a given task. In particular, SP4, SP7 experimentally verify that using a supervised re-ranking method has an implied shortcoming of resulting in a domain dependent performance. The performance for each domain depends on the portion of training data belonging to that domain. Thus, it is attempted to propose unsupervised re-ranking techniques that can lead to similar level of quality as supervised re-ranking.

Another parallel classification of the re-ranking approaches is based on whether the module scores each retrieved context independently or together as constituents of a possible reasoning chain. We call the two categories of methods Context re-ranking and Chain re-ranking respectively. Table 6 lists the methods belonging to the two categories. SP4 show that evaluating the reasoning chain as a whole leads to better performance than evaluating each passage independently.

The general flow of the graph based methods can be summarized in a three-step process: 1) A graph of one or more types of nodes (entity, sentence, paragraph, document nodes, etc.) is constructed and edges are added based on some lexical heuristics. 2) Contextual encodings of the nodes are passed to one or more layers of a Graph Neural Network [gnn] (or a Graph Attention network [gat] or a Graph Convolutional Network [gcn]). These layers update the representation of each node using the representation of each of its neighbouring nodes. In this way, after $n$ such layers, nodes separated by a path of $\leq n$ edges have shared information with each other. Therefore, each layer is expected to perform one hop of the reasoning process. 3) The updated embeddings are passed to the answering module.

As can be expected, some methods slightly deviate from the proposed general flow. We describe these differences below along with some details on graph building and reasoning steps.

• •

  SP1 build an entity graph by starting with the entities extracted during the retrieval process. For each node, a BERT model takes its representation along with the question and a retrieved passage and extracts entities from the paragraph to be added to the graph. The process is repeated until either a) no new nodes can be added or b) a maximum number of nodes are extracted. Sentences containing the extracted entities are considered as ‘clues’ for the respective entities. Clues are then used for updating the node representations using a Graph Neural Network (GNN). Clues are also used later for the auxiliary task of supporting facts prediction.

• •

  SP6 propose constructing a hierarchical graph network with four types of nodes that represent question text, relevant paragraphs, sentences in these paragraphs and entities in these sentences. Bidirectional edges are introduced between the first hop source sentences and second hop target paragraphs. Edges are also introduced between paragraph and its sentences; sentence and the entities it contains; question and the entities it contains; between all paragraphs; between each sentence and its previous and next sentences. RoBERTa is used for encoding and a bi-attention and a Bi-LSTM layer are used for getting initial contextualized representations of the nodes. Graph Attention Network (GAT) [gat] is applied over the hierarchical graph and the updated representations are merged with the original contextual representations using a gated attention mechanism. The merged representations are passed to the answer prediction module. The proposed gated merging is supposed to be effective for dealing with smaller number of hops.

• •

  LL14 build a use abstract meaning representations (AMR) [amr] to build a semantic graph from a passage. Graphs across passages are connected by adding edges between the same concepts in different graphs. A non-parametrized method is employed for generating reasoning chains. A depth first search (DFS) is used to find all paths from the question nodes to the answer nodes and all the edges on the path are used to recover the facts from the context that form the reasoning chain. The reader reads these reasoning chains and the question to predict the answer. A reinforcement learning based chain-aware loss is proposed to boost the performance of the system.

• •

  Similar to LL14 ll13 and LL19 build a semantic graph. However, the entities and relations are extracted using GPT-3.5.

• •

  SP8 construct a graph of documents and add edges if the documents have a shared entity. Question aware embeddings of the document are generated by Albert and used as initial node representations. GAT is used to update the representations of each shared entity. To update the representations of non-entity tokens, they are fed to a transformer along with the updated representations for multi-document fusion. The updated representations are passed to the answer prediction module.

• •

  SP12 make entity graph of all entity mentions of the answer candidate in supporting documents. Cross-document edges are added between the mentions about the same entity and within-document edges are added between every node pair belonging to the same document. For initial representations, GloVe and ELMo are used for token-level and contextualized embeddings respectively. Contextualized embeddings are crucial since graph nodes only contain entity tokens. To deal with entities containing multiple words, average is taken over their embeddings. The two embeddings are passed to a 1 layer NN (replaced by Bi-LSTM for encoding the question) for getting the initial node representations which are concatenated with the NER and POS embeddings before feeding to a GCN layer.

• •

  SP13 compute similarity between GloVe embeddings of each word of the sentence with each word of the query and take the mean among m(=5) closest sentence words to get a sentence score and select top k(=25 or 30) sentences. A graph containing sentence nodes and document nodes is created with only the selected sentences. Edges are added between documents if they have a shared entity and between a paragraph and all its sentences. Adding edges among sentences adds complexity without much improvement. Initial representation of a node is computed by passing the matrix of GloVe embeddings of the entities contained in the sentence or paragraph to a bi-linear attention [kim2018bilinear] layer on nodes and query. To compress the representations into fixed size, self-attention is used. T(=3) layers of Gated Graph Neural Network is used to update the representations.

• •

  SP15 create an entity graph similar to SP12 but have some additional edge types: co-reference edges: between co-reference mentions of the same entity (separate type since these are less reliable) and complement edges which signify no connection between the two nodes. Co-reference edges is a separate edge type since these are less reliable owing to the error in the co-reference system. These are not required for the masked version. The query representations are formed by passing ELMo [elmo] to a Bidirectional RNN [rumelhart1985learning]. Initial query dependent node representations are formed by passing the ELMo contextual embeddings to a feed forward network. L layers of a gated relational-GCN [schlichtkrull2017modeling] R-GCN, a version of GCN are applied to get the final representations which are passed to the prediction module. Relational-GCN is able to accommodate different edge types by using different weight matrices for the neighbouring nodes connected by different edge types. An ablation experiment verifies that, although useful, co-reference edges have the least contribution to performance. Another experiment tries to predict the edge type by training a model but the performance is poor.

• •

  DP6 solve the text-table hybrid MHQA by using the retrieved cells from the previous stage to then decide which neighboring cell to hop to.

• •

  SP25 construct an entity graph and add edges between entities with the same mention text, between entities belonging to the same sentence, and between each entity in a paragraph and each entity in its title. Question and contexts are concatenated and passed to a BERT model followed by a bi-attention layer to get the contextual node representations. The following four steps are performed iteratively: 1) Mean-max pooling over tokens in an entity mention to update the entity embeddings. 2) Attention between query and entity is used to compute the mask weights which are multiplied to the entity representations before feeding to a GAT. 3) Updated entity representations are concatenated to entity tokens and representations of all tokens are updated using an LSTM layer. 4) Bi-directional attention between query and entity representations is used to update the query representation. Updating embeddings of each token is necessary since the answer might not be an entity.

• •

  SP16 create two different graphs: a) sentence graph with edges between sentences belonging to the same paragraph and between sentences that share an entity. b) entity graph with edges between entities appearing in the same sentence, between different mentions of same entities and between each entity in the paragraph to each entity in its title. The intuition is that humans first focus on question-related paragraphs, then sentences, and then the important words. BERT word embeddings are passed to a bidirectional attention layer to get the contextual embedding. They also propose a novel similarity matrix for the layer. Self attention is applied to the query and query aware node embeddings are passed to a GAT. Self attention is applied on the sentence node representations and the output is appended by the representation of each word in the sentence. An LSTM is applied to fuse the output of 2 graphs.

• •

  SP20 form a sentence graph and add an edge between sentences $s_{i}$ and $s_{j}$ with the weight $w_{ij}$ given by:

  $$w_{ij}=\begin{cases}\frac{1}{1+e^{-n+K_{1}}}&\text{$s_{i}$ and $s_{j}$ have $n>0$ shared entities}\\ \frac{1}{1+e^{d+K_{2}}}&\text{$s_{i}$ and $s_{j}$ belong to the same paragraph}\\ &\text{separated by $d$ sentences.}\end{cases}$$

  (4.2)

  These weights allow the model to deal with the edge types in a novel way. The concatenated paragraphs along with the question are fed to a BERT followed by a bi-attention layer. Sentence representations are obtained by extracting token level embeddings from the paragraph encoding and a weighted addition of token embeddings where the weights are calculated using a two layer MLP (Multi Layer Perceptron). Most methods using GNN perform message passing for each node in parallel. This requires the representation of nodes to be updated exactly $L$ times where $L$ needs to be specified as a hyper-parameter. If $L$ is large, it leads to over smoothing. If it is too small, it inhibits long-path reasoning. Moreover, this algorithm performs unnecessary updates leading to inefficiency. Thus, a novel message passing algorithm is proposed which performs a BFS starting from the question and passing the messages through every edge that is visited in the process.

Critique: SP13 argue that the advantage of using graph-structured representations lies in reducing the inference steps necessary to combine multiple pieces of information related in a path-like manner. The fact that the required graphs (or some parts of it) can be created offline gives it a computational advantage. However, graph structure also has certain limitation [SP10], particularly for comparison type questions in HotpotQA where the evidence paragraphs about the two entities are independent. Many of the techniques assume that the relation between nodes is directional which may not always be the case. For gated GNNs, computational efficiency as well as the learning ability of the model degrades with the increasing number of nodes and edge types in the graph used.

While the graph structure is prevalent for multi-hop reasoning, SP26 argue that it is not necessary and that graph attention can be considered as a special case of self-attention. Treating SP25 as the baseline, results are compared after removing the graph fusion module. It is observed that the performance gained by using the graph structure can be easily compensated for by fine-tuning the BERT based encoder. While the graph structure enables the model to focus only on adjacent nodes, a model without any prior knowledge can still learn this behaviour. This is verified by further experiments. When the graph fusion module is replaced by self-attention layers the results are very similar whereas replacing it by a transformer leads to a significant improvement. Graph attention reduces to self-attention for a fully connected graph making it a special case of self-attention. Attention patterns are visualized and observed similar to kovaleva-etal-2019-revealing. It is found that the pre-trained transformers are able to capture several types of attention patterns: a) between entities, b) between co-referenced entities, c) between an entity and its attribute, and d) between an entity and a sentence. Depending on the structure of the graph, these patterns may or may not be covered by graph attention. Therefore, self-attention can be said to be more general and flexible than graph attention.

The key idea here is to decompose the multi-hop question into multiple single hop questions and use a single-hop QA model to answer each of the questions. This approach is particularly effective for questions like the bridge questions in HotpotQA that are constructed using a bridge entity. These questions can be easily decomposed into two sub-questions by finding the bridge entity. LL2 manually label the decomposition for questions across multiple datasets and use GPT-3 and RoBERTa to answer the questions and observe that question decomposition can help fix $60\%$ of the errors made on the original multi-hop questions. However, this technique exploits the knowledge about the structure of question and thus, is difficult to adapt when the structure of the questions can be flexible. Below we list some representatives methods for this approach:

• •

  SP14 use the hypothesis that each sub-question can be formed from a multi-hop question by copying and lightly editing a key span from it. Questions in HotpotQA are categorized into four categories: bridge (47%), comparison (22%), intersection (23%)(questions asking for an entity that satisfy multiple properties) and others (8%). Editing methods are proposed to be dependent on the question type for the first three categories. Question text is segmented into several spans by training a pointer network that predicts $p_{ij}$, the probability of the $i^{th}$ word to be the $j^{th}$ index to split the question. 400 annotations are manually generated for training. For each question, three such indices for bridge questions, two indices for intersection and four indices for comparison questions are predicted by maximizing the joint probability of the decomposition. These spans are modified slightly depending on the reasoning type. Any single-hop QA model can be used for answering each single hop question. Concatenation of the question, the reasoning type, the answer, and the evidence is encoded using BERT and scored using 1 layer feed forward NN with sigmoid activation. The reasoning type is decided as the one resulting in the maximum score. An alternative way where the reasoning type is predicted before decomposing and answering is tried and found to be giving poor results. To verify the original hypothesis, same technique is tested with span-based questions replaced by human written questions. There is a little difference in model performance indicating that the span-based sub-questions are as effective as free-form sub-questions.

• •

  SP21 break down the task of MHQA into two components: Coarse grained decomposition and Fine grained interaction. The decomposition module is responsible for making the high dimensional vector distribution of entity nouns or pronouns more inclined to the intermediate answer to the question. The fine grained interaction module is a modified bidirectional attention module. The resulting context representations are passed to a self-attention layer and then used for supporting facts prediction by passing to a Bi-GRU layer.

• •

  SP22 propose MHQA methods for four different datasets: HybridQA, QASPER, HotpotQA-Long and ShARC-Long. QASPER is originally proposed as single hop QA over long documents whereas ShARC is proposed as a conversational QA task. The paper utilizes the fact that long documents are often structured into sections and subsections which can be used as separate contexts with limited dependence among them. A pre-trained ETC model [ainslie-etal-2020-etc] is used as the question encoder and the context encoder. ETC is a pre-trained Mask Language Model that employs a global-local attention mechanism. ETC assigns to each sentence a special global token that only attends to local tokens in the sentence, and its embedding is trained to summarize the information of local tokens in the sentence. ETC additionally adopts Contrastive Predictive Coding (CPC) [van2018representation] to train the embedding of global tokens to make them aware of other sentences in the context. It takes in a sequence of sentences and outputs the contextualized representations of each sentence. For conversational QA, the question is formed by concatenating the original question with a sentence formed by each pair of follow-up question and answer. ETC is again used to encode the question paragraph. For the 2-hop questions (HotpotQA, HybridQA), a null sentence is passed to ETC along with the question to get two question encodings. The paragraph embeddings are computed by a weighted sum of sentence embeddings, where weights are the attention score of the query vector to that sentence. At every step, each sentence and every paragraph in the document is attended to the query and the output is used to update the query representations. The updated query representation are then combined with the embedding of next query sentence to give the query vector for the next hop.

Critique: SP14 also find that some of the questions are not composed of two single-hop questions but require implicit multi-hop reasoning, hence cannot be decomposed. Secondly, for some questions, answer for each sub-question does not exist explicitly in the text and has to be inferred with commonsense reasoning.

In this section, we list some interesting approaches that do not fall into the above defined categories.

• •

  Question generation: LL18 propose multi-task training of a model to perform both multi-hop question answering and sub-questions generation. The motivation is that a model capable of asking good sub-questions corresponding to a complex question should perform well at MHQA. The system consists of a GNN based QA component and an LM based QG component that are trained together with a shared encoder.

• •

  Entailment based MHQA: Equation 2.1 presents MHQA roughly as an entailment task, where the premise consists of multiple contexts and the hypothesis is "a answers q" (referred as $H_{aq}$ henceforth). This indicates the utility of entailment models for the task. However, modelling MHQA as entailment faces three major challenges: a) A larger set of possible answers makes this method difficult to scale. b) MHQA requires aggregation of multiple contexts and single sentence based entailment models can not be used directly. c) Entailment models are usually not trained to filter irrelevant information. SP28 aim to tackle the two latter challenges on Multiple choice datasets, OpenBookQA and MultiRC. Two simple baselines are proposed: Concatenate that concatenates all sentences as the input to an entailment task, and Max of local decisions that uses entailment on each sentence independently and then aggregate the results with a max operation. The two baselines perform poorly validating the challenges mentioned above. Therefore, a novel method ‘Multee’ comprising of a sentence relevance module and a multi-level aggregation is proposed. The sentence relevance module uses a pre-trained entailment model to produce hypothesis aware representation of each sentence which are passed to a Bi-LSTM layer to give contextual representation of each sentence which is fed to a feed forward layer to get the relevance scores $\alpha_{i}$. The multi-level aggregation module pass each sentence along with the hypothesis to $k$ ESIM [chen-etal-2017-enhanced] entailment stacks to generate $k$ paragraph level vectors which are concatenated and passed to a feed forward layer to predict the entailment. Each entailment stack has $m$ layers, $l$ of these layers process each sentence independently and the outputs are aggregated by using $\alpha_{i}$’s. Rest of the layers process this aggregated output to result in paragraph level embeddings. The aggregation is also done in the cross attention layer to produce a cross attention matrix containing attention between each hypothesis token and each paragraph token. The sentence relevance weights are used here as well. Each entailment stack used is pre-trained on SNLI [bowman2015large] and MultiNLI [williams-etal-2018-broad].

• •

  Commonsense knowledge for MHQA: SP27 argue that a pre-trained model may not be able to capture every grounded common-sense knowledge even with large corpora and propose using ConceptNet [speer2016conceptnet] for extracting the same and using it, form reasoning paths leading to the answer. A tree of various reasoning paths, all starting from the question concepts ($c_{1}^{\prime}s$) is formed by following four steps: a) Selecting relations $r_{1}^{\prime}s$ from ConceptNet that link $c_{1}$ to another concept $c_{2}$ from the context. b) selecting relations $r_{2}^{\prime}s$ that link $c_{2}$ to another concept $c_{3}$ from the context. c) Selecting all neighbouring concepts $c_{4}$ of $c_{3}$ connected by $r_{3}$. d) Selecting relations $r_{4}$ that link $c_{4}$ to another concept $c_{5}$ from the context. This results in a large number of reasoning paths that are scored in two steps: a) $n-score$ is computed by term frequency of each of $c_{2},c_{3},c_{5}$ in the context $C$. $c_{4}$ is scored with its Point-wise Mutual Information (PMI) [church-hanks-1990-word] with $c_{1-3}$. Each node’s score is normalized across all its siblings in the tree using Softmax. b) $c-score$ The node scores are accumulated starting from the leaf node and updating each non-leaf node recursively.

  $$c\_score(c_{i})=\begin{cases}n\_score(c_{i})&c_{i}\text{ is a leaf node}\\ n\_score(c_{i})+\frac{c\_score(c_{i+1}^{\prime})+c\_score(c_{i+1}^{\prime\prime})}{2}&otherwise\end{cases}$$

  (4.3)

  where $c_{i+1}^{\prime}$ and $c_{i+1}^{\prime\prime}$ are the top two scoring children of $c_{i}$. At each level of the tree, for every node, only the two top scoring children are maintained and the rest are pruned resulting in a total of $2^{4}$ reasoning paths. Context representation is attended to the commonsense representations formed by embedding the tokens in each of the reasoning paths. Updated context representation is combined with the original one using a sigmoid gate. The gate incorporates the fact that the commonsense might be optional. Hence, the unit is called NOIC (Necessary and Optional Information Cell).

• •

  Reasoning over tables: DP6 use BERT to encode a cell in a table where a cell is represented by its value, position, hyperlinks etc. Encoding of a cell along with the encodings of its neighbouring cells is fed to a feed forward model and a Softmax layer to find the cell for the next hop. The model can also hop to the same cell. The cell value is prepended to the hyperlinked passage and passed to the answer prediction module.

• •

  Pointer network for reasoning chain prediction: SP19 encode the question and obtain query dependent paragraph encoding by using BERT. Sentence encodings are extracted from the paragraph encoding similar to SP20. Alternative baselines are a) BERT-Sent that obtains query aware encoding of each sentence by using BERT and b) BiDAF-Para that uses BiDAF [bidaf] to encode paragraphs. Results show that BERT-para performs the best. An LSTM Pointer Network is trained to predict probability of each sentence being the $t-{th}$ sentence in the reasoning chain i.e., $P(s_{i}=rc_{t})$. The ground truth reasoning chain is determined using a chain extractor model discussed in Section 4.4. The model is trained using both Negative Log Likelihood (NLL) and Reinforcement Learning (RL). However, RL does not improve the performance significantly. During test time, a beam of possible chains is maintained since the best chain may not be evident until it is entirely retrieved.

• •

  Bi-directional attention baseline: DP1 propose a baseline by modifying the architecture of SP45. Question and the concatenation of paragraphs is passed to an RNN for combining the character and word level embeddings. Bi-directional attention is applied across the question and context embeddings to get query aware context representations. A residual connection is added to the output of another RNN on these representations before passing to a self-attention layer.

This category corresponds to the multi-choice setting of the MHQA task. Methods dealing with candidate answer prediction usually start the retrieval/reasoning process by using a candidate-aware embedding of the question, and output the representations for each candidate independently. We look at some of these methods below:

• •

  A Bi-directional attention layer on query and context [SP12], or a self-attention layer on the embeddings may also be employed.

• •

  Some of the methods use these representations to perform an independent binary classification for each candidate (for multiple-correct type questions such as MultiRC) [SP4, SP7, SP9, SP28].

• •

  On the other hand, some methods perform a multi-way classification for all candidates (for single-correct type questions such as OpenBookQA, WikiHop) [SP7, SP9, SP12, SP15, SP19].

• •

  SP15 use an ensemble of five models, each trained with a different weight initialization.

• •

  SP7 use both the answering methods for QASC and find that the multi-way classification results in an improvement of  5% accuracy.

For tasks requiring the answer as a text span from the contexts, the following methods have been proposed:

• •

  DP1 suggest a span answering approach where the output of the reasoning module is passed to another RNN for supporting facts prediction. Another RNN is used to predict the start and end token of the answer span. Finally, a three-way classifier layer is used to predict the answer type among ’yes’, ’no’ and the extracted span. Many methods focusing on the first two units use this baseline as the answer prediction unit [SP2, SP3, SP10, SP16, SP21, SP25, SP26].

• •

  SP24 make two changes to the above baseline: a) Concatenating passages before encoding makes the representations depend on the order while concatenating. Thus, a shared RNN encoder first encodes each passage independently and then concatenates the representations. b) All the attention layers are replaced by self-attention on the concatenated question and context representation.

• •

  SP1, SP6 replace the RNNs in the above baseline by MLPs and directly feed the node representations for answering. SP5 deal only with bridge question and hence use a single MLP for span prediction on entities. SP13 only deal with supporting facts prediction and use a single MLP for the same.

• •

  SP8 use an AlBERT model to predict the answer span among the $K$ identified passages. The model then predicts if the answer was found in the given passage. If not, the predicted span is used for iterative retrieval for the next hop.

• •

  SP14 use an off-the-shelf single-hop answering module to answer the sub-questions formed by the reasoner.

• •

  SP17 use a simple BERT model to answer the question from the paragraph identified as the answer paragraph. SP23 do the same from the concatenation of the two identified passages.

• •

  SP19 concatenate question and all sentences of all the retrieved reasoning chains and feed to BERT to perform the 4 tasks.

• •

  SP20 propose to use the output of the GNN model. Sentence scores are computed using MLPs on the sentence node representations. Paragraph scores are computed using MLPs on max pooling over the sentence representations. The two scores are combined with the span extraction score to predict the final answer. Separate MLPs are used for predicting the answer type and the supporting facts.

• •

  SP22 Concatenate the embeddings at all retrieved steps $\{k^{0},\cdots,k^{\ast},\cdots,k^{0},\cdots,k^{\ast}\}$ and perform a weighted sum to get $\mathcal{K}$ that is used to make the final prediction. The Softmax weight $\mathcal{Y}_{j}$ is computed across the retrieved sentences from all steps.

Previously, generative answering models had a low prevalence because of the unsatisfactory performance of models to generate relevant texts as well as the lack of reliable evaluation techniques. As a representative of the earlier approaches, SP27 use a self-attention layer followed by a PGN decoder to get the final answer.

However, with the recent advancements relating to large language models (LLMs), generative answering approaches have become a lot more popular. We describe these methods in detail in Chapter 5 designated to LLMs for MHQA.

Reasoning chains are an integral part of explainable MHQA. HotpotQA contains the supporting facts that are required to answer the question, however these are not ordered. Several works have aimed to predict the order among these supporting facts by using simple lexical heuristics [AP6, AP5, AP2] so that the reasoning chain formed can be used to further train or evaluate a model. However, it is also crucial for the models to be able to predict the reasoning chains only by using the question and contexts.

• •

  SP18 propose a semi supervised reinforcement learning for two modules to recover the reasoning chain in a cooperative-game approach. Apart from predicting the reasoning chain, the model is also able to predict the relations among the sentences. The two modules are: a) Ranker module that, given $k$ passages and a question, selects a reasoning chain. Question and passages are encoded by bi-GRU [cho2014learning] and a Match-LSTM [wang2016machine] model is used to get a probability for each passage containing a part of the reasoning chain. A passage is then sampled and used to update the question by passing to an MLP. The process is repeated with the updated question. The ranker module is rewarded for selecting the correct passage at the correct reasoning step. b) Reasoner module that predicts the entity from the current passage to the next passage. Given the first passage (called head) selected by the trained Ranker, the Reasoner uses a Math-LSTM model to predict the probability of each entity appearing in the second passage (called tail). While the ordering of the supporting facts is generated similar to DP1 for HotpotQA, the reasoning chains need to be manually annotated for MedHop. The manual annotation is done by extracting all valid paths such that the first sentence contains an entity in the second sentence and the second contains the answer. A chain is manually labelled as positive if the corresponding passage describes the drug-protein interaction. Reasoning chains extracted in this way may not be unique.

• •

  SP19 derive pseudo gold chains using NER and co-reference resolution. A sentence (only) graph is constructed with edges between sentences that belong to the same paragraph and between those that have a shared entity. Reasoning chains are collected by starting from the question node and finding all possible paths in the graph that lead to an answer containing sentence. These chains are then ranked by two heuristics: a) shorter chains are given higher scores. b) chains whose sentences have a high F-1 ROUGE overlap with the question are given higher scores. Experimental results indicate that b) is a good criteria for scoring chains. Human evaluation shows that the reasoning chains produced are of similar quality compared to the supporting facts present in the HotpotQA. A pointer network is then trained to predict each sentence in the reasoning chains.

SP29 propose a new sub task of MHQA where the model has to figure out and fill the knowledge gap for question answering. It is assumed that the first hop retrieval has been performed ideally and the task is to use the retrieved context to answer the question. A modified version of OpenBookQA is released where the core fact is part of the input and several relations have been modified.

The biggest challenge that the LLMs currently face is that they are known to hallucinate at times, i.e., generating text that is factually incorrect or not derivable from the given information. This poses a threat to accountability of the resulting MHQA system and can produce incorrect results. A large number of works have tried to propose sophisticated and clever ways of overcoming this challenge. We describe few such interesting works in detail below:

• •

  Self-consistency: Self-consistency [LL33] is proposed as an alternative for the naive greedy decoding used in chain-of-thought prompting. Instead of only taking the greedy decoded reasoning path, a diverse set of reasoning paths are sampled, and then the most consistent answer across the reasoning paths is predicted by marginalizing out the sampled reasoning paths. Self-consistency is motivated by the intuition that a complex reasoning problem typically admits multiple different ways of thinking leading to its unique correct answer. Assuming that the hallucinations in the LLM output are random, self-consistency is more likely to lead to the correct answer since it is unlikely that majority of the reasoning paths with hallucinations lead to a common answer. Therefore, self-consistency has been adopted to mitigate hallucinations.

• •

  Self-refine: In self-refine [LL34], the LLM generates an initial response to the prompt, and then iteratively provides feedback for the output and refines it. The same LLM serves as the generator, feedback provider as well as the refiner. During self-refine, the LLM can also evaluate the response on factual correctness, thus fixing the hallucinations.

• •

  [LL21] extend self-consistency and self-refine by fine-tuning external LLMs for providing feedback on the generated reasoning chains. The authors collect labelled data for correctness, error types and descriptions, and the required corrections for each reasoning chain and answer pairs for 2361 examples. They use a weighted self-consistency approach where a fine-tuned Llama2 predicts a score for whether each answer and its corresponding reasoning path are correct or not. This score is used as the weight while voting for the correct answer. They also explore extending self-refine by using a fine-tuned Llama2 model to identify the errors in a reasoning path and refining the generated output.

• •

  [LL6] propose ‘Verify and Edit’ that combines self-consistency and self-refine. It follows a three-step process: finding uncertain predictions using self-consistency, editing their rationales by searching for supporting facts, and using the edited rationales to generate final answers. An answer is considered as uncertain, if none of the generated answers gets the majority agreement. For each of the predictions corresponding to the uncertain answer, each sentence in the reasoning chain is verified by generating corresponding questions. These questions are answered after retrieval and the answer is used to update the particular sentence in the original reasoning chain. Finally, the updated reasoning chain is used to generate the final corrected answer.

• •

  [LL3] propose a claim decomposition based technique for self-evaluation of the LLM generated answers. The complex question is broken down into a set of claims that a correct answer to the question must satisfy. The LLM then identifies which claims mention the same entities and include extra tags for them. For evaluating a generated answer, the answer is replaced in each of the claims and the LLM is prompted to answer true/false based on whether the answer satisfies the claim or not. The answer is then scored on the ratio of claims satisfied by it. Multiple answers are predicted by sampling based generation of the reasoning chains and the one with the highest score is predicted.

[LL24] argue that the challenge of hallucinations is difficult to overcome since the LLMs might always suffer from knowledge gaps. A knowledge gap here refers to some missing or outdated information. Since LLMs heavily rely on knowledge gained during the pre-training phase, outdated or incomplete information in the pre-training corpus can confuse the model, leading to generating factually incorrect text. Further, pre-training LLMs is extremely expensive in terms of time and compute. It is therefore ideal that the LLM is able to detect a knowledge gap and refrain from generating information in such a case. For identifying knowledge gaps, the following approaches can be followed:

• •

  Calibration-based: getting a probability score for an answer by either using token probabilities and temperature during generation [mtlearner, holisticeval] or directly asking the model to generate its confidence in its output [tian2023just].

• •

  Training based: training an extra layer [hidden], or an external module [cobbe2021training] to predict whether the answer produced has a high enough confidence. Another interesting way is to use identifying knowledge gaps as one of the instruction tuning tasks [instr].

• •

  Prompting based: techniques such as self-reflect [selfreflect], self-consistency [LL33] predicting none-of-the-above [selfreflect] or more information needed [moreinfo] instead of predicting the answer.

• •

  Multi-LLM collaboration: using two independently trained LLMs for either providing feedback to each other, or complementing each other to fill the knowledge gaps in both [LL24].

As mentioned in Chapter 4, retrieval is often broken down into two steps. Different metrics can be used to evaluate the two steps:

Based on the type of the answer, the following evaluation metrics have been used for large scale evaluation for the task: