Infer Human’s Intentions Before Following Natural Language Instructions

In order for AI to be useful to people in our homes and natural environments, non-experts need to be able to communicate with AI agents using natural language. This issue has long captured the attention of researchers (Winograd 1972; Siskind 1994), and the primary challenge involves mapping natural language to concrete meanings within the physical environment. Several studies explore language-conditioned task completion in specific environments (Shridhar et al. 2020; Suglia et al. 2021; Kojima, Suhr, and Artzi 2021). With the emergence of LLMs, many works discussed grounding language by leveraging pre-trained LLMs  (Blukis et al. 2021; Nair et al. 2022; Zellers et al. 2021). A prominent example is SayCan (Ahn et al. 2022), which proposed extracting the knowledge in LLMs by using them to score the likelihood that a subtask available to the robot will help complete a high-level instruction. Although the above studies may incorporate common sense reasoning about language as well as information within the physical environment, their instructions explicitly express human intentions. For example, the most ambiguous instruction solved by SayCan is, “I spilled my coke, can you bring me something to clean it up?” where the ambiguity can still be easily resolved given that the sponge is the only cleaning tool in the environment. In contrast, we address the problem that realistic human instructions omit certain information for efficiency, making them much more ambiguous, and necessitating inferring human intentions to fill in the gaps.

We consider the case where the human and the robot are working in a shared environment, which is closely related to the literature on collaborative communication (e.g. Two Body Problem (Jain et al. 2019)). CerealBar (Suhr et al. 2019), DialFRED (Gao et al. 2022) and TEACh (Padmakumar et al. 2022) introduce collaborative tasks where the human works as a disembodied issuer of instructions, possibly responding to robot’s questions via explicit messages. In contrast, we consider the problem in which the AI assistant needs to consider both explicit messages in natural language and the implicit information in observed human actions. Further, we assume that instructions are not exhaustively describing the required information, but are generated based on a trade-off between informativeness and communication cost. To this end, we focus on the HandMeThat (Wan, Mao, and Tenenbaum 2022) (HMT) benchmark, that calls for the ability to consider both explicit and implicit messages when following ambiguous instructions. The previous state-of-the-art work (Cao et al. 2024) on HMT performs iterated goal inference over the goal space in symbolic representation. However, it requires hand-crafted, pre-defined structures and extensive domain knowledge, which is not applicable in real-world scenarios.

In our method, we hope to infer the human’s intentions based on the observed historical actions, which is related to goal recognition problem (Lesh and Etzioni 1995; Baker, Tenenbaum, and Saxe 2007; Levesque 2011; Meneguzzi and Pereira 2021). Most of the works are based on the assumption of rationale that an agent should make (approximately) optimal decisions towards the goals every step (Dennett 1987; Gergely et al. 1995). Understanding human intentions in embodied environments has also been studied in many works; for example in Watch-and-Help (Puig et al. 2021) the AI must infer the human’s goal from demonstrations, but no natural language is involved. Some recent works (Ying et al. 2024; Zhang et al. 2024) leverage LLMs to conduct goal inference based on the observed human actions or messages. In this work, we employ a small language model trained from scratch to undertake this part of the reasoning, since the aim is not to achieve precise goal recognition but to assist with the step-by-step social reasoning process.

This work is also inspired by the research that uses intermediate steps to solve complex reasoning problems, including formal and mathematical reasoning and program synthesis (Roy, Vieira, and Roth 2015; Amini et al. 2019; Chiang and Chen 2019; Chen et al. 2020; Nye et al. 2021). Specifically, Nye et al. shows that step-wise prediction method performs better than directly predicting the final outputs in program synthesis when prompting LLMs. Chain-of-Thought (CoT) (Wei et al. 2022) thoroughly explores how generating intermediate reasoning steps improves the performances of prompting LLMs to deal with complex reasoning tasks. In this paper, we show that social reasoning tasks benefit from the same approaches, and demonstrate that inferring human intentions is a critical component of successful human-robot collaboration.

A human-robot Markov Decision Process is described as a tuple $\langle{\mathcal{S}},{\mathcal{A}}^{h,r},{\mathcal{T}},{\mathcal{U}},R^{r},\gamma,T\rangle$. $s\in{\mathcal{S}}$ are object-oriented states including the locations, status and type of each object and agent. ${\mathcal{A}}^{h,r}$ is the joint action space with ${\mathcal{A}}^{h},{\mathcal{A}}^{r}$ being the sets of actions available to the human and the robot, respectively. ${\mathcal{T}}:{\mathcal{S}}\times{\mathcal{A}}^{h,r}\times{\mathcal{S}}\rightarrow\{0,1\}$ is the transition function where ${\mathcal{T}}(s,a^{h,r},s^{\prime})=1$ if and only if taking actions $a^{h,r}$ at state $s$ gives $s^{\prime}$ as the next state. ${\mathcal{U}}$ is a set of instructions that the human can give to the robot. $R^{r}:{\mathcal{S}}\rightarrow\mathbb{R}$ is a reward function for the robot, $\gamma$ is the discount factor, and $T$ is the horizon. Throughout the paper, we consider a scenario with only a single round of instruction following for the robot. In each episode, starting from an initial state $s_{0}$, the human begins working in the environment, and the robot is waiting. Human stops at a time step $t^{\prime}\leq T$, leading to a trajectory $\tau_{t^{\prime}}=(s_{0},a^{h}_{0},s_{1},a^{h}_{1},\ldots,s_{t^{\prime}-1},a^{h}_{t^{\prime}-1})$ and a final state $s_{t^{\prime}}$. Then the human produces a natural language instruction $u\in{\mathcal{U}}$ that asks the robot for help. Given $\tau_{t^{\prime}},s_{t^{\prime}}$ and $u$, the robot needs to interact with the environment by taking a sequence of actions $\{a_{t}^{r}\}_{t\geq t^{\prime}}$ to maximize its discounted rewards $\sum_{t=t^{\prime}}^{T}[\gamma^{t-t^{\prime}}R^{r}(s_{t+1})]$.

A straightforward solution to the human-robot MDP may treat $\tau_{t^{\prime}}$ and $u$ as additional state information. However, in reality, $\tau_{t^{\prime}}$, $u$, and $R^{r}$ have important correlations: when the human is taking actions and producing instructions, their behavior can be modeled as optimizing for an internal reward function $R^{h}:{\mathcal{S}}\rightarrow\mathbb{R}$, which is not revealed to the robot. Our insight into this broad problem class is to leverage the causal relation between the human’s behavior and their instruction by explicitly modeling hidden, unobserved variables representing the human’s goals and intentions. This enables making better use of the human’s trajectory to disambiguate the instruction, by recognizing the task the human is assigning to the robot.

We start by assuming the reward functions $R^{h}$ can be parameterized by a set of possible goals ${\mathcal{G}}^{h}$. The human’s goal $G^{h}\in{\mathcal{G}}^{h}$ is sampled from an underlying distribution over ${\mathcal{G}}^{h}$ at the beginning of an episode, and is fixed across the horizon. However, it is not revealed to the robot directly. The goal in ${\mathcal{G}}^{h}$ is usually global and complex, such as “organize the bedroom.” We assume the human trajectory $\tau_{t}$ is rational, in the sense that it was produced to maximize the reward $R^{h}(\cdot\mid G^{h})$. Based on $G^{h}$ and the current progress $\tau_{t^{\prime}}$, the human then selects a subgoal $p^{*}\in G^{h}$ (a part of the human’s overall plan) for which the human would like the robot’s help, such as “having all books put in the box,” and then specifies a task $G^{r}$ for the robot (e.g., asking the robot to hand over a specific book). The instruction $u$ is generated based on $G^{r}$. The relations between these variables are illustrated in Fig. 2.

Although we do not put specific assumptions over the structure of goals in ${\mathcal{G}}^{h}$ and how $\tau_{t}$ is generated, we illustrate them with a simplified example in Fig. 2. We define $P$ as a set of predicates where each $p\in P$ is a classifier over states (to say whether the predicate is satisfied or not). For example, one predicate can be written as $\langle\exists y,\text{box}(y),\forall x,\text{book}(x)\Rightarrow\text{in}(x,y)\rangle$, which describes putting all books in a box. Now we assume that the human goal $G^{h}=\{p_{1},p_{2},\cdots,p_{l}\}$ is a set of predicates. The human chooses to work on predicates one by one and has been working on all $p_{k}$’s ($1\leq k\leq l$) before stopping at time $t^{\prime}$, and then the subgoal $p^{*}$ is chosen from the set of remaining predicates: $p^{*}\in\{p_{k},\ldots,p_{l}\}$. Next, the human specifies a robot’s task $G^{r}$ such that the robot actions will result in a state $s^{\prime}$ where $R^{h}(s^{\prime}\mid\{p^{*}\})>0$ (i.e., $G^{r}$ is a useful step towards $p^{*}$, a remaining subgoal to accomplish). Note that neither $p^{*}$ nor $G^{r}$ is accessible to the robot, since the robot can only get access to the natural language instruction $u$. For example, “could you pass that from the sofa” could be an utterance for $G^{r}=\langle\text{human-holding}(\text{book\#0})\rangle$ and $p^{*}=\langle\exists y,\text{box}(y),\forall x,\text{book}(x)\Rightarrow\text{in}(x,y)\rangle$.

Our model, FISER, builds on top of the factorized human-robot MDP formulation above. We formulate the problem into the social and embodied reasoning phases.

Since small-scale language models cannot process excessively long inputs, we divide the information into four parts, including world state description ($s_{t^{\prime}}$), human’s trajectory ($\tau_{t^{\prime}}$), language instruction ($u$), and the model’s past outputs.

The proposed model consists of $3N$ ($N=3$) encoder layers that are used to update the representation over all four parts of the inputs, in order to predict the human’s intentions or the robot’s actions. Specifically, we use the embeddings at Layer $2N$ to predict the robot’s task $G^{r}$ (social reasoning phase). Then, we replace the instruction input for Layer $2N+1$ by the predicted $G^{r}$ and then use the last layer embeddings to predict robot’s actions. If the Human’s Plan Recognition stage is further included, we use the embeddings at Layer $N$ to predict the selected human subgoal $p^{*}$ and then replace the trajectory input for Layer $N+1$ by the predicted $p^{*}$.

The model is trained in either a multi-staged (MS) or an end-to-end (E2E) manner. The E2E models are trained to directly output robot’s actions, but an auxiliary loss is applied over their intermediate predictions of robot’s task (and human’s plan). The MS models, however, disentangle the social reasoning (functions TR and PR) from embodied reasoning (function GP), and train them as two separate modules. The latter module is trained with the ground-truth $p^{*}$ or $G^{r}$, but is evaluated using the predictions from the former module.

Each encoder layer is composed of four Transformer layers and a Modality Interaction module.

Now we introduce our predictors for intermediate reasoning steps and the final actions. All the predictions are trained with cross entropy loss over corresponding supervisions.

We evaluate our models over the HandMeThat (version 2) dataset (Wan, Mao, and Tenenbaum 2022). It introduces a household ambiguous instruction following task rendered in text. HandMeThat instructions are split into four difficulty levels, and the gaps between levels correspond to different challenges. The instruction in a Level 1 task has no ambiguity—it is a pure planning task. A Level 2 task requires social reasoning where a robot can successfully accomplish the task if it can also infer the goal from the human trajectory. On Level 3, the robot needs to further consider pragmatic reasoning in language use. For example, if there are multiple books everywhere in the room and only one coat, which is on the sofa, and both the book and the coat are helpful to the human’s goal, the human might say “Could you pass that from the sofa?”. Using pragmatics, we can understand the human is asking for the book, since “from the sofa” is required to disambiguate which book the human is referring to, but if the human wanted the coat they could simply ask for it. The final Level 4 contains tasks with inherent ambiguities that cannot be resolved with the existing information, but can potentially be resolved with a strong prior over what human is likely to do. For example, taking one more fruit will complete the goal of packing picnics, but there are many kinds of fruits in the refrigerator to choose from. From the perspective of completing the goal, any fruit will do, but human preferences may make a difference–the human may want apples instead of bananas at this time.

We evaluate all the models on their success rates in achieving the robot’s goal. Note that the original evaluation metric in HandMeThat additionally considers the number of robot’s steps. An agent can trivially improve success rate with increased steps, by simply searching all objects in a brute force, trial-and-error fashion. We believe that enumeration over objects is not realistic in the real world, so we restrict our experiments to one trial (to be completed within 4 or 5 steps).