Meta-prompting Optimized Retrieval-augmented Generation

Focusing on Retrieval-Augmented Generation (RAG), this approach seeks to enhance truthfulness and curb hallucinations by expanding the initial prompt, which contains the initial query, with additional content retrieved from sources that are external to the LLM. Such additional content is obtained with the help of an auxiliary Retrieval Model where the retrieval model may be a simple Jacquard model or a vector database that extracts relevant content from external sources and pass it on to a Large Language Model that generates an appropriate response given the original query and the extracted content. If this external content is unstructured text, it may be of different lengths, such as sentences, paragraphs or full documents, among others.

By feeding LLM’s knowledge, and curbing its whim, with further knowledge from external sources, more accurate answers are likely to be provided.

Compared with other techniques, such as fine-tuning or prompt-engineering, RAG key advantage is the ease with which newer, up-to-date content is taken advantage of, as this does not require the costly compute of re-training neural networks (as in fine-tuning) or the costly human labour for the creation of further manually designed prompts (as in prompt-engineering). To be sure, all these techniques can nevertheless be mixed and function together.

Usually, the pieces of content retrieved may be from heterogeneous sources and they tend to lack a connecting thread. They may also be redundant or may be of very high volume. These, among other aspects, may end up having a detrimental effect and eventually jeopardizing the generation task, rather than enhancing it.

To mitigate this problem, we present a method that consists of adding an intermediate step between the retrieval of the external content and the entering of the expanded prompt into the LLM to finally obtain the response to the initial query. Aiming at improving the performance of this generation-LLM, this intermediate step seeks to obtain a refined version of the external knowledge.

This refinement is accomplished by means of an auxiliary transformation-LLM that is entered with a prompt containing the pieces of retrieved contents, preceded by an instruction with the request for the sought refinement.

For example, if several pages of Wikipedia are retrieved as possible relevant content, the transformation-LLM processes this content and may generate a summary or remove unnecessary information from that original content.

Turning to this refinement instruction, this is obtained by an automatic procedure that is preliminary to running the RAG system made of transformation- and generation-LLMs, and it is undertaken by yet a third LLM.

Inspired in Yang et al. [35], in this procedure a meta-prompt is used as input to this third, optimizer-LLM for this to iteratively generate new tentative instructions, score them, and retain, in the meta-prompt itself, a list with the top k ones that induce better performance for the RAG system. By the end of this optimization process, the best scoring instruction in this list is the one retained to be used in the refinement step of the retrieved contents with the transformation-LLM.

This meta-prompt contains a meta-instruction and a list of tentative instructions that is aimed at being updated during this process with new instructions that induce better RAG performance. After a new tentative instruction is generated, its contribution to approximate the gold output to the initial query is scored, and the list of tentative instructions in the meta-prompt is possibly updated so that it retains the top-performing ones so far. This is iterated, and an optimization trajectory is hence accomplished to eventually find the new refinement instruction that maximizes the success of the RAG system.

***

In this paper, we propose a method for RAG to be enhanced with the refinement of the retrieved content, a refinement that is optimized by resorting to iterative meta-prompting. This is a novel method that can be combined with previous approaches aimed at enhancing RAG.

We report on the experiments performed to put this method to the test. This approach is extrinsically evaluated by being embedded in a demanding question-answering downstream task. Its performance demonstrates that it is an effective method to enhance RAG by improving by 30% the performance of a baseline RAG without this method, and that it can be combined with other previous state of the art methods for RAG enhancement proposed in the literature.

The remainder of this paper is structured as follows: Section 2 discusses related work; Section 3 describes the method proposed in this study; Section 4 reports on the models and dataset resorted to; Section 5 presents the experiments undertaken and their evaluation, and discusses the results obtained; Section 6 addresses future research paths; and finally, Section 7 closes this paper with concluding remarks.

Multi-hop question answering requires taking into account disparate pieces of content to get at the answer for a query, which constitutes a most demanding scenario for the task of question answering.

We resorted to a most complex benchmark for multi-hop question-answering available in the literature, the StrategyQA dataset [6, 19, 11, 8], which contains 2,780 queries, each associated with related content made of paragraphs and the respective yes or no answer. Based on Wikipedia content, this dataset covers a range of diverse topics and the task consists in, given a query, to provide an accurate answer to it together with the passages retrieved from Wikipedia with the most relevant content to get at that answer — Table 2 displays an example.

To provide for the evaluation of the proposed method, and isolate the accrued performance induced by it, thus disregarding possible fluctuation or loss of performance due to the retrieval process, only the gold pieces of content from a test set should be taken into account. Since the answers are not provided in the original test set of StrategyQA, a new test set for the present evaluation exercise had to be built. Accordingly, we divided the original training set into two parts: a new test set with 490 of the original training examples, which matches the size of the original test set, and a new training set containing a subset with 1800 such examples. The resulting train and test sets have an average query length of 9.6 words, and 2.33 contents (paragraphs) per query and are almost balanced. The training set contains 834 yes answers and 966 no answers (46.32 % / 53.68 %). The test set contains 237 yes answers and 253 no answers (48.40 % / 51.60 %).

Two Transformer-based language models with 70 Billion parameters were used, a pre-trained Llama-2-70b and an instruct model Llama-2-70b-chat fine-tuned for dialogue use [31]. These models were trained and fine-tuned with a context length of 4k tokens over 2 Trillion tokens on a mix of publicly available data.

In general, the default Llama2 model hyper-parameters were applied and no hyper-parameters search bound was performed. All language models use a temperature value of 1.0, a maximum of 64 generation tokens for the new instructions, a maximum of 128 generation tokens for the refined content, and a maximum of 64 generation tokens for the response to the task. The optimization run was performed for two days on two NVIDIA A100 40GB GPUs.

All software and versioning along with hyper-parameters are fully described in the source code of these experiments.¹¹1For the sake of reproducibility, data and code are available at https://github.com/nlx-group/rag-meta-prompt

Based on empirical experimentation, we arrived at a meta-prompt, presented in Table 1, that indicates the aim of the optimization problem and includes a starting example instruction.²²2The starting instruction is ”Clean and organize the previous text.”

The instruction optimization was iterated over 100 steps. At each step, 3 instructions were generated, each such instruction was evaluated on a random sample of 6 training examples, and the meta-prompt was eventually updated to containing the 8 top scoring queries so far. When this iteration was concluded, the best scoring instruction was retained as the optimized instruction.

We compare against the same generation-LLM using test queries and associated pieces of content, that is without the later being refined by the transformation-LLM under the instruction that was optimized by the optimizer-LLM.

For the StrategyQA task, a Boolean answer is expected. Accuracy is thus the metric used for evaluating the match between the answer output by the system and the gold answer in the data set. Accuracy score is given by the proportion of correct answers, and a generated response was counted as correct if the gold answer was found in the exact beginning of it. The response underwent minimal normalization, with just lowercasing.

As for the instrumental process of instruction optimization, it is worth recalling that the evaluation is performed over sample examples from the training set. For a tentatively generated instruction, a correct answer to it counted 1 point; an incorrect answer, in turn, counted 0.5 points if it was nevertheless in a Boolean format, or counted 0 points otherwise. The maximum possible score was thus 6 points, given each tentative instruction was evaluated against 6 sampled queries as indicated above.

All in all, six experiments were undertaken, two resorting to the model Llama-2-70b, developed with a pre-training regime only, and four resorting to the model Llama-2-70b-chat, which resulted from further fine-tuning it with dialogue data.

Both these models were used in two evaluation scenarios. In one of these scenarios—noted as query in Table 3—, the response to the query entered was provided by the LLM alone, with no further content from external sources being entered. In the other scenario, in turn,—noted as query+contents—, further content from external sources was included in the prompt as well. The performance scores for these two scenarios with the two models are displayed in the top four rows 3.

External, non-parametric content improves generation As expected, and in line with results in the literature, the retrieval-augmented generation (26.12%) outperforms the plain generation based solely on the query (16.53%).

Fine-tuning improves generation Also as expected, and by a very large margin, better performance scores were obtained with Llama-2-70b-chat, which had been fine-tuned on dialogue tasks, namely 16.53% against 3.46$\%$, with the query only, and 26.12% against 6.73%, with the query and external content.

Retrieved content refinement via meta-prompting optimization improves RAG — the proposed method is effective The model Llama-2-70b-chat was thus retained and two further evaluation scenarios were considered.

A scenario with the application of the proposed method—noted as refined query+contents—, where the external content was refined with the help of an instruction optimized with meta-prompting.

The performance scores indicate that, with 34.69% accuracy, our proposed method of enhanced RAG outperforms plain RAG, with 26.12%, thus contributing for a large improvement of over 8.5 percentage points, that represents here an improvement rate of almost 33%.

Retrieved content refinement via "brute force" optimization does not improve RAG A sixth scenario—noted as refined query+contents no iter—was also considered. Here the external content was refined as in the proposed method, but the instruction was refined under an alternative way that dispensed with iterative meta-prompting.

All in all, 300 tentative instructions are generated during all optimization steps —recall that we had 100 iteration steps with 3 tentative instructions generated per step with meta-prompting optimization. To dispense with this iterative meta-prompting, the same number of 300 new tentative instructions were generated at once, in a "brute force" fashion. By the end of this process, all tentative instructions were scored with the same scoring function as in the proposed method, and the top instruction was evaluated on the test set.

This "brute force" optimization approach, scoring 25.92%, is outperformed not only by the proposed method of meta-prompting optimization, with 34.69%, but even also by the baseline, plain RAG, with 26.12%.

Statistical significance To assess the statistical significance of the improvements by our method, we employed the unpaired t-test.³³3The unpaired t-test evaluates if there exists a statistically significant distinction between the means of two independent samples by comparing them. We evaluated the baseline system with three seeds and did the same for the meta-prompting optimized system. Both samples are independent and one may assume the samples are normally distributed. Applying the unpaired t-test, a two-tailed P value equal to 0.0004 was obtained, which is considered statistically significant.

Table 4 presents the top generated prompts. The best scoring prompt (last row), with 5.5 (out of a maximum of 6), was obtained at iteration step 46 (out of 300 steps in total), and a good prompt (first row) can be obtained with only 28 steps.

When reading the best prompt (last row), one realizes that it aims to improve the task through the summarization of the retrieved contents, considering their broader context, and identifying the main theme or message. It appears thus like a reasonable prompt a human might have thoughtfully arrived at if aiming at improving the performance of the task.

It is reasonable to assume that the meta-prompt iteration in subsequent steps used this query and its score in its search for further tentative instructions, with the generated instructions in three subsequent steps (58, 65 and 72) being some derivation of it (second, fourth and fifth rows).

When taking a look at the entire set of generated queries, a high fluctuation of the evaluation scores can be observed along the iteration steps. This is likely due to some interim, generated instructions happening to perform poorly.

To gain insight about where our method outperformed the plain RAG baseline, we examined the first 10 instances where our method correctly provided the answer while the baseline failed. Among these, in six cases, the baseline provided a verbose response and might have failed the exact-match evaluation criterion used.⁴⁴4An example of a verbose response: [query] Was Superhero fiction invented in the digital format? [response] The answer is no; superhero fiction did not originate in digital format. Superheroes have their roots in pulp magazines, comic strips, and comic books, which were all print media formats before the advent of digital technology. In the remaining four cases, the baseline either answered incorrectly, responded with a query, or failed to provide an answer.

Conversely, we reviewed the first 10 instances where our method failed to provide the correct answer while the baseline succeeded. We observed that our method exhibited a verbose response behavior in five cases that eventually arrived at the correct answer but failed the exact-match evaluation criterion. In two other cases, our method gave a verbose response without providing an answer, while in two remainder cases, it provided an incorrect response. Finally, in one instance, our method did not provide any response.

Both methods seem thus to be similarly penalized by the evaluation criterion for not providing straight answers when the correct answer may happen to be included in the verbose response.