LLMs as Function Approximators: Terminology,
Taxonomy, and Questions for Evaluation

What is peculiar about this functional relationship is that the function does not need to be learned specifically by the model, at least not in the heretofore common sense. Rather, the function needs to be found in the vast and “latent” space that is opened by the encompassing “sequence to sequence” function that is the LLM.⁴⁴4Alternatively, you can think of what is happening here as induction of the function based on the prompt. The proposal here is meant to be agnostic as to what the process is; in fact, it is meant to provide a clear way of talking about what the issue here is (finding / retrieving vs. inducing). Techniques for doing so have been suggested from the time when this property was first observed Brown et al. (2020) and are by now somewhat better understood, or at least catalogued Schulhoff et al. (2024). The following is not meant as advice on formulating prompts (which is what the textual means for what we analyse here as function induction are now commonly called); it is meant as a proposal for naming the informational components present in such prompts.⁵⁵5Following the distinction between intensional and extensional task description I introduced in Schlangen (2021).

We can now categorise prompt-induced functions (or, equivalently, the task that a given prompt is meant to pose to the model) according to the type of semantic relation between domain and co-domain; that is, between the $x$ and the corresponding $y$, yielding a distinction between:

• •

  transformation tasks, where the information that is contained in $y$ is also contained in $x$ (that is, $x$ entails $y$). E.g., summarisation, translation, paraphrasing.

• •

  categorisation tasks, where $y$ is a category (typically, out of a small set of candidates) into which $x$ falls.

• •

  additive tasks, where $y$ contains information not entailed by $x$. This can be further classified into
  - recall-additive, where the additional information is based on “recalled” information from the training data (and is assumed to be factually true); e.g., where $y$ is meant to be an answer to a factual question $x$; and
  - creative-additive, where the additional information in $y$ is not (necessarily) meant to have been encountered in the training data (but is still based in some sense on $x$). If $y$ is meant to fulfil certain constraints (e.g., be executable code), we can call this grounded creative-additive; if not, free creative-additive (e.g., generation of a story based on the prompt).

The boundaries between these classes are not necessarily sharp—for example, one might want to understand the “text to code” task as a form of translation (and hence, as a transformation task), if the text is very specific; or as grounded creative-additive, if it is more abstract—but the taxonomy shall suffice to discuss some differences between tasks in the section below. Finally, some tasks, like for example summarisation, are of course better modelled as a mapping from a source text into a set of summaries (or, even better, a fuzzy set / a pair of text + indicator of task-based goodness, interpretable as degree of set membership). Our concern here, however, is not with modelling all cases in all details; it is with framing the discussion, for which these details can remain unresolved for now.

To make the above a bit more concrete, we now go through three example use cases of LLMs and show how they would be described using the terminology introduced above. The first that we will look at, evaluation using HELM Liang et al. (2023), maps rather directly onto our terminology. In order to use (test splits of) existing datasets, the authors define prompt templates that “explain the task” to the model, and then insert the actual instance into this template. Figure 1 shows the example they give for a multiple-choice dataset. We can see that what they label task instruction corresponds closely to our intensional task description. (Although as such a better formulation would be something like: “of the given answer options, select the correct one for the given question about anatomy”.) What is labelled train input / reference / output in the example then corresponds to our $(x_{i},y_{i})$, and the test input / reference to our $x_{t}$. We also note that the paper discusses various metrics, making distinctions roughly along the lines of our task taxonomy from the previous section.

In the previous example, LLMs were purposely made to act like previous types of machine learning model, in order to evaluate them in much the same way as those were evaluated (performance on test set). Figure 2 shows how the proposed analysis can also be applied to the type of interactions often found in interactive use with an LLM-“chatbot”. The aim of the user here is less to find a function for re-use and more to find a formulation that solves one given task; but still the process can be usefully seen as an attempt to induce, via several steps, the desired mapping.

Lastly, we just point out that the so-called system prompt often used in systems aimed at chit chat (see example in the Appendix) can be understood as (part of) the intensional task description constraining the general “reply appropriately to the context”-function that drives the ‘conversation’ forwards.

This brief discussion was meant to illustrate the concepts introduced in the previous sections. Their real worth needs to show in how they bring out commonalities in different questions one can ask about LLM use. To do this is the task of the next section.

f,p: How closely does $\hat{f}$ approximate $f^{*}$? This is the most basic question that we used to ask of machine learned models, and it can be explored with the usual instruments: a test set of $x,y$ mappings, and a metric for comparing predicted values to these reference values. The nature of this metric will likely differ significantly depending on the task type as described above in Section 2.2. This is what the paradigm of reference-based evaluation, represented above by HELM, addresses.

There are also new types of questions, however, owing to the fact that the function is approximated without additional (weight-based) learning:
f,p: How well is $f$ protected against so-called “prompt-injection” attacks Schulhoff et al. (2023), where the $x_{t}$ (coming from a user) contains text that might be understood as being part of, or even replacing, the $\mathrm{td}$, turning the function $f$ into a function $f^{\prime}$ outside of the control of the feature designer. Related to this, but not quite identical, are questions of how well the domain and co-domain of the function is protected against undesirable in- and outputs (e.g., a ‘give me instructions for doing x’ function, where this by design is meant to only respond to activities deemed appropriate for a certain user group; or, on the output side, avoiding certain types of language in additive tasks).

f,t: What is the relation of $\hat{f}$ to the training set of the underlying model? Is the resulting function best understood as an interpolation between similar examples seen in during training (or even memorisation), or is it genuine extrapolation / generalisation? Related to this, what would make an observer grant “understanding” or “intelligence” to the induced function? E.g., if the function is of the complexity of the example in Figure 1, and $\hat{f}$ performs well on a test set, is that evidence that the question text is understood? That the examined medical knowledge is understood? (We will come back to this type of question.)

f,t: What force does the application of a function have? If the function is something like “produce an answer to the question”, what is the status of the generated text? When or how would it get assertoric force?⁷⁷7 Some appear to believe that this is question of overall retrieval accuracy (e.g., inter alia, Heinzerling and Inui (2021)), or that simple disclaimers (“AI models can make up facts; check everything yourself”) can leave this status unclear; I think that is wrong Schlangen (2022, 2023b). In any case, this is an issue worth being discussed more explicitly.

i,p: How natural can the $\mathrm{td}$ be? How stable is what is being induced against semantically insignificant variations in how $\mathrm{td}$ is formulated? That current models are not doing particularly well here is the whole raison d’être of tools like DSPy Khattab et al. (2023); how they do is now beginning to be investigated systematically Lu et al. (2024). This is also one of the factors that the paradigm of preference-based evaluation, represented above by Chatbot Arena, likely captures, in that the relative ease with which a task is described should figure in the user’s relative preference.

i,p: Related to the previous question is the question of whether functions induced through intuitively similar task descriptions (e.g., “summarize this news article from the domain of sports” / “…from the domain of entertainment”) can be prompt-induced similarly.

i,p: How important are the formatting instructions for recovering the answer $y$ in the model response $r$? There is a lot of informal knowledge about how best to get models to produce responses from which a desired answer format can easily be extracted (e.g., by asking for structured output instead of free text); what is the influence of decisions made here Yu et al. (2024)?

i,t: How is the process that goes from $\mathrm{td}$ to $\hat{f}$ best described – as induction or as retrieval? Is that process even a uniform one (always induction, or always retrieval), or does its nature depend on contextual factors? (This is related to the question about generalisation vs. memorisation above, but gets at the issue from a different perspective.) Datasets like ARC Chollet (2019) are designed to probe this question; current models are not faring well.⁸⁸8 https://arcprize.org/leaderboard

F,p: Continuing with a theme from above, another potentially desirable feature is to be able to block certain functions from being reachable via (user) prompt at all; this is particularly relevant if the generality of the model is exposed to users (as in a chatbot-style interface). As discussed above, one way this is currently achieved is by formulating lengthy ‘system prompts’ (see Section A below).

F,p: Do models perform similarly on similar tasks? This is related to the induction question above, but here getting at it from the perspective of how “evenly” the space of functions is covered. From a practical perspective, this kind of homogeneity helps with forming a mental model of which feature should work, and how well. From a theoretical perspective, this leads over to the next question.

F,t: What is the relation between tasks that can successfully be prompt-induced (in the sense that they perform well; let us call this set $\hat{\mathcal{F}}$) and tasks that humans can do? What is the relation between performance differences shown by models and by humans? Imagine that a model performs equally well (measured via reference-based evaluation) on the function ‘answer this maths question targeted at 10 year old students’ and the function ‘answer this maths question targeted at 17 year old students’—what would that tell us about the likely underlying mechanism with which the model answers these questions? This line of inquiry brings us to questions of the construct validity of tests, insofar as they meant to support statements about general abilities of models Raji et al. (2021); Schlangen (2023a), and hence to the heart of the question about the (artificial) ‘intelligence’ of these function approximators.