Show, Don’t Tell: Evaluating Large Language Models Beyond Textual Understanding with ChildPlay

Typically, LLMs are transformer-based models that process input text and generate output text in a coherent and contextually appropriate manner. They utilize the self-attention mechanism to weigh the importance of different words in a sentence relative to each other [36, 6]. Input text is tokenized, converted into vectors using embeddings, and processed through transformer layers that calculate attention scores to dictate focus on relevant tokens [36, 6, 12]. The model then selects the next token based on learned distributions, iteratively generating an arbitrarily long sequence of text [36, 6, 12]. With their enormous parameter counts, from Alpaca with 7 billion parameters [31], to LLaMA with 65 billion [33] or even PaLM and its 540 billion parameters [11], these neural networks have learned to model complex linguistic abstractions, capturing patterns in syntax, semantics, pragmatics, and even elements of style and tone [6, 7, 23].

Benchmarks for evaluating Large Language Models (LLMs) have been designed to assess comprehension, generation, and adaptability across a wide range of language tasks. Datasets like SQuAD, GLUE, BIG-bench, and the lm-evaluation-harness offer various test types, including multiple-choice questions, reading comprehension exercises, and dialogue completion tasks. These benchmarks deploy metrics such as response correctness, language generation fluency, and the ability to maintain contextually relevant dialogue [24, 37, 2, 14]. Other benchmarks like SuperGLUE, ANLI, TruthfulQA, and HellaSwag have been developed to evaluate different aspects of LLM performance, such as natural language understanding, commonsense reasoning, and factual knowledge about diverse topics [37, 22, 20, 40].

Recent studies have explored alternative approaches to evaluate LLMs’ reasoning abilities in non-linguistic modalities. Liga and Pasetto modeled the game Tic-Tac-Toe using ASCII characters, pitting LLMs against the minimax algorithm to observe emergent features, which, according to the authors, might be akin to consciousness [19]. The minimax algorithm is widely considered the optimal algorithm for playing tic-tac-toe, as it guarantees a win or draw against a perfect opponent [29, 1]. While LLMs performed well in some instances, they generally failed to win against the minimax algorithm, often resulting in a draw [19]. Topsakal and Harper [32] used Tic-Tac-Toe encoded with list and illustration prompts in their study. They found that while GPT-4 secured the most wins, it did not always win, indicating that GPT models cannot play Tic-Tac-Toe optimally. This contradiction raises the question: can we truly say the model knows how to play Tic-Tac-Toe if it can explain optimal strategies (see Appendix A.5) but does not consistently win? Or is its performance merely the result of probabilistic outcomes?

Some critical studies have highlighted the need for caution in interpreting LLMs’ capabilities through benchmarking. Lappin et al. assessed their strengths and weaknesses, finding that they excel at many language tasks but struggle with deeper reasoning, world knowledge integration, and context understanding beyond local co-occurrences [18]. And Zečević et al. argued that LLMs may discuss causality but lack true causal reasoning based on interventions and counterfactuals [41].

Bender et al. argue that the lack of transparency and potential risks associated with these large, opaque models raise concerns about their trustworthiness and accountability [3]. While the criticism of Bender et al. focuses on the social dimension of the problem of interpretability and trustworthiness, recent work by Schaeffer et al. critics emergent capabilities and the perceived intelligence of LLMs. They suggest that some claimed "emergent abilities" of LLMs may be an artifact of the choice of evaluation metric, rather than fundamental changes in model behavior [25]. Their analyses demonstrate how the use of nonlinear or discontinuous evaluation metrics can create the illusion of emergent abilities, even when the underlying model performance changes smoothly and predictably with scale.

This critique of the evaluation metrics used in assessing LLMs invites a deeper exploration of general intelligence - specifically how it can be reliably measured and observed in AI through rigorous and realistic tests that extend beyond linguistic prowess to include broader cognitive functions. If we must define general intelligence (GI), one is to use the "g factor," which refers to the ability to reason, plan, solve problems, think abstractly, and learn quickly across a wide range of domains [39, 4, 38, 9, 8]. GI then involves higher-order cognitive processes that go beyond specific skills or knowledge domains [15, 16].

A critical issue that arises in analysing the reasoning capabilities of large and opaque models like the GPT series, is training-test set cross-contamination, which becomes increasingly problematic for the most advanced models [6]. The massive training datasets used, comprising extensive portions of the internet, are often untraceable and completely anonymous to researchers outside the initial developer groups, to some extent even to the developers themselves, making replication studies impossible [6, 13]. The exact amount and identity of data used to train models like GPT-3.5 or GPT-4 has not been publicly disclosed, posing a risk of rendering current benchmarking efforts meaningless due to cross-contamination.

Researchers have attempted to counter the contamination problem using N-Gram Overlap as a metric for detection, by eliminating or withholding results for tests where answers were present in the training data [6]. However, this method has been criticized. Blodgett et al. point out, for example, that such heuristic approaches to mitigating biases in NLP systems can be problematic and may not fully address the underlying challenges [5]. The method is also limited in that it fails to consider the context in which N-Grams appear and may discount synonymous or analogous text worded differently. Additionally, the decision to use a 200-character window around detected N-Grams during training of GPT-3.5 is arbitrary and may not accurately reflect the influence of surrounding text on model learning [6].

We argue that there is a need for nuance in current debates and a pragmatic perspective on understanding LLMs’ capabilities. In order to approximate some measurement of GI in an AI system, it is important that we build benchmarks that allow measurements that can truly gauge generalization and reasoning in a human-like manner, rather than relying solely on pattern matching and statistical correlations [35].

In this work we introduce ChildPlay, a suite of non-language-based games like Tic-Tac-Toe, Connect-Four, Battleship, LEGO Connect Language, and the game of Shapes, to assess reasoning, strategic capabilities, symbolic reasoning, and pattern recognition abilities of large language models (LLMs) beyond traditional linguistic modalities. Games provide structured environments with clear success criteria, making them suitable for evaluating strategic thinking, planning, and long-term decision-making of LLMs. Their dynamic and adversarial nature resembles real-world scenarios, assessing generalized intelligence and reasoning beyond the training domain [28, 19, 32]. We encode these games using ASCII representations to minimize dataset contamination issues prevalent in contemporary LLM benchmarks [6, 19].

Specific tasks in the BIG-bench benchmark [2], among others, are categorized as either zero-shot, one-shot, or multi-shot [6]. Our tasks fit the zero-shot category, as models are given only a brief explanation at inference time with no examples for playing beyond the explained formalism. To demonstrate the reasoning capabilities of LLMs beyond their training data, we focus on a modality not explicitly trained for: spatial reasoning about ASCII sequences. An agent capable of true abstraction should be able to encode and interpret these sequences if the rules are explained or known.

For this purpose, we developed several tasks, including LEGO assembly, ASCII games of Tic-Tac-Toe, Connect-Four, and Battleship, as well as identifying simple geometrical shapes represented as 1s in 15-sided grids of 0s. The same models were deployed over all experiments, namely gpt-3.5-turbo-1106, and gpt-4-1106-preview, which in this paper are referred to as GPT-3.5 and GPT-4, respectively. Every experiment was tested across different temperature settings (t) per model, namely t=0, t=0.5, t=1, and t=1.5. When asked about their understanding of the tasks, GPT-3.5 and GPT-4 were able to generate board states and explain the queried games, including their rules and optimal play. Thus, we consider the tests valid: if the models are truly capable of reasoning, they should be able to play these games optimally given that they "know" and are capable of explaining what playing optimally means (see Appendix A.5). Experiments ran over night, at minimum taking a couple of minutes and at most taking a few hours.

Lego Connect Language (LCL)  We invented a formal language we call LEGO Connect Language (LCL). More specifically, we propose $LCL_{2}$ as a language to instruct assembly in 2D on the x and y axis (this can easily be generalised to $LCL_{3}$ - instructions along the x, y, and z axis). The models were given instructions and their output was fed through a visualizer script to generate the images contained in this work. Only 2x4 pieces were allowed. A piece $P$ (see Fig 1) is then defined as a tuple $P=(l,w,(x,y),c,h)$. A construction, $M$, is then a valid construction in $LCL_{2}$ if no pieces are overlapping and all pieces are connected to other pieces. Namely, a Lego piece is connected through interlocking pegs, not by merely touching sides. And secondly, two Lego pieces overlap when they share the same y-coordinate and any part of their length has the same x-coordinate.

Game 1: Validity Testing  In this experiment, we evaluate the ability of different models to validate the correctness of a given Lego construct. The constructs are generated to be either valid or invalid. A construct is considered valid if there is no horizontal overlap between pieces, and pieces must connect via overlapping pegs such that the whole assembly is connected (no floating pieces). The models, namely GPT-4 and GPT-3.5, are then tasked with predicting the validity of each construct. The evaluation metric for this experiment was the proportion of correct validations, measured across different temperature settings.

Game 2: Construct Generation  In this experiment, the models attempt to generate valid LCL constructs. Each construct description consists of a list of tuples, where each tuple specifies the coordinates and color of a Lego piece. The models generated these constructs based on prompts and the validity of the constructs was automatically evaluated. The metric for this experiment was the proportion of valid constructs generated, measured across different temperature settings.

We automatically produced 800 images for the validity test, half valid and half invalid ones. Then each model was queried to produce 100 images at each temperature setting, which we then checked for validity. We believe our use of LCL is related to the tests found in Bubeck et al. [7], where JavaScript or LaTeX was used to prompt GPT-4 to produce images. However, while the images in Bubeck et al. [7] included common examples such as letters, a car, a truck, a cat, a dog, a person, a pig, a house, and a unicorn, all of which are likely represented in the training data in JavaScript or LaTeX, LCL challenges the model to step outside of its learned data distributions by remaining abstract.

Three Board Games: Tic-tac-toe, Connect-four, and Battleship  In the case of the three board games, each new board state was accompanied by the introductory game explanation sent through the OpenAI API in a zero-context testing environment. The models were provided with the current board state and an opponent making moves at random, with the LLM always playing as the first player, which is advantageous in all three games. Context beyond the initial instruction and the current board state was deemed irrelevant since these games are fully observable, meaning every board state contains all the necessary information to play optimally. The input to the game was simply two scalars for the row-column pair or just a scalar for the column number in the case of connect-four.

For the battleship game, ships (’S’) were randomly initialized, always horizontally, with varying sizes spanning between 2 and 5 cells. When there is a hit by either player, the position is marked with an ’X’ on both players’ boards. If the guess was a miss, an ’O’ is placed on the position instead.

The Game of Shapes  In the case of the game of shapes, preliminary work involved probing the models to determine what geometric shapes they consider basic by prompting them multiple times. The first three shapes consistently mentioned were square, circle, and triangle (not necessarily in that order). The game then consists of finding a basic geometric shape "hidden" behind 1s within a matrix of 0s in a multiple-choice fashion. Four shapes were used as options: the circle, the rectangle, the triangle, and the cross, but only the latter three were ever shown to the model (cf. Fig. 3).

As previously stated, Tic-Tac-Toe as a benchmark has been tackled before [19, 32]. Since it is quite popular, we decided to replicate it before creating new games. But this time using an ASCII encoding instead of a list of moves such that we can gauge spatial reasoning through symbolic reasoning. For comparison with the model’s performance, Fig. 4 presents the Tic-Tac-Toe match results of the minimax algorithm against the same random player the models played against. This outcome creates a baseline for optimal play against a random player.

Tic-tac-toe, Connect-four, and Battleship  To check for a win, we determine if the player has successfully connected the winning number of pieces in a row on the board, which could be horizontally, vertically, or diagonally. To detect missed and blocking moves, we simulate all potential moves for the player by checking if placing a piece in any column leads to a win. If such a move is found, and the player does not execute it on their turn, it is recorded as a missed win, if such a move is found for the opponent and the player does not block it, we register it as missed blocking move. We define incorrect moves to mean a move that was illegal, such as playing a position that has already been played. This results in an immediate loss.

Fig. 5 encompasses comparative results from playing Connect-Four, Tic-Tac-Toe, and Battleship. Each subfigure, 5(a), 5(b), and 15, respectively, outlines the number of games won by the models.

Unfortunately, the models were incapable of following the rules for the Battleship game, that is, regardless of temperature, the models lose the large majority of games, with GPT-4 not winning a single game due to incorrect moves (cf. Fig. 17). GPT-3.5 wins around 10% of the matches at low temperatures, but none at higher temperatures, we refer to Fig. 15 in the Appendix A.3.3 instead.

It is notable that both GPT-3.5 and GPT-4 exhibit their poorest performance in both Connect-Four and Tic-Tac-Toe at a temperature setting of 0, indicative of deterministic play that reflects the models’ learned strategies (Appendix A.3). The Random Player’s normal distribution across columns (Fig. 13) suggests a lower likelihood of countering GPT’s central strategies, in both games, but particularly at Tic-Tac-Toe where GPT-3.5 commits more errors than GPT-4, significantly impacting outcomes due to incorrect moves (Fig. 5(b)). These errors generally increase with temperature, probably due to enhanced choice randomness (Fig. 11). This explains the lack of direct model losses from final defeating moves since losses often result from illegal moves.

Average game moves, missed wins, and blocks in both Tic-Tac-Toe and Connect-Four are further illustrated in Figs. 6(a) and 6(b), highlighting a decrease in these metrics as temperature rises, suggesting that higher settings potentially broaden the explored moves within the models’ strategies. Conclusively, neither model plays the games optimally, as evidenced by the considerable number of missed wins and blocks. Both subfigures demonstrate that, as temperature increases, the number of missed wins and blocks decreases. This might suggest that higher temperature settings potentially increase the explored moves in the models’ learned strategy, in case there is any. We can conclude the same as before, namely that neither model can play Tic-Tac-Toe optimally given the number of missed wins and missed blocks.

The number of moves of GPT-3.5 and GPT-4 per game (see Fig. 6 ²²2Error bars for LCL results as well as for average moves in the board games are computed using the standard deviation: $\sigma=\sqrt{\frac{p(1-p)}{n}}$, where $p$ is the proportion of correct identifications, and $n$ is the total number of trials or identifications.) can be thought of as a measurement of stability in gameplay, not just against the random player, but in general, given that a longer game entails that the model is not losing to illegal moves or to its oponnent. It increases linearly with temperature, inversely correlated with performance measured by the decrease in missed wins and blocks. Tic-Tac-Toe shows a linear improvement, whereas Connect-Four experiences an exponential boost in performance from temperature 0 to 0.5, followed by a linear decline. The random player consistently performs better against GPT-3.5 in Tic-Tac-Toe but loses more frequently in Connect-Four. Both models struggle with blocking or seizing winning moves from the random player. An analysis of the move heatmaps (cf. Appendix A.3) explains why winning Connect-Four against a random player appears straightforward. As the model consistently places pieces in the same column, the probability of the random player losing increases with the board size. However, even under these challenging conditions, the random player still secures wins in at least 20% of the games played against GPT-4.

Shapes  In the game of Shapes, a correct detection happens when the player’s selected shape corresponds with the shape shown on the board. Players have four choices: "circle," "triangle," "square," and "cross". Notably, a circle is never actually displayed to the model, and the positions of these choices are not randomized to test if the model displays any inherent bias for the question order. This does not affect the outcome, since the game does not change across different sessions as it is designed to operate within a single question-response framework.

In the shape detection test results in Fig 7 we see that GPT-3.5’s performance was approximately equivalent to that of random chance when identifiying triangles and crosses, yet it completely failed to recognize squares. In contrast, GPT-4 performed remarkably well, successfully identifying shapes with an accuracy of $\approx$ 80%, demonstrating particularly prociency at recognizing triangles .

LCL  In the game of LCL, both models systematically failed to respect the two rules, namely that Lego pieces must be connected through interlocking pegs, not by merely touching sides, and secondly, that no Lego pieces may overlap, which occurs when they share the same y-coordinate and any part of their length has the same x-coordinate. For example, Figs. 8(a)¹¹1Fig. 8(a) generated for the validity test., and 8(b)²²2Fig. 8(b) generated for the validity test. show valid LCL assemblies, while Fig. 8(c)³³3Fig 8(c) generated for the validity test. shows an invalid LCL structure. While subfigs. 8(d)⁴⁴4Fig. 8(d) generated by GPT-3.5 at temperature = 0. and 8(f)⁵⁵5Fig. 8(f) generated by GPT-3.5 at temperature = 1. show invalid output from GPT-3.5, and Fig.8(e)⁶⁶6Fig. 8(e) generated by GPT-4 at temperature = 1.5. shows a valid output from GPT-4 and Fig. 8(h)⁷⁷7Fig. 8(h) generated by GPT-4 at temperature 1. shows an example of an invalid output by GPT-4.

Fig. 9 ²²footnotemark: 2 shows a roughly linear increase in the proportion of correct answers for GPT-3.5 during the validity test as a function of temperature. While GPT-4 peaks at temperature 0.5 and then declines. However, only GPT-4 produced a small minority of valid LCL constructs (namely 0.04 of a total of 400 = 16), while GPT-3.5 did not manage to produce a single valid LCL construct.

In Tic-Tac-Toe, both models underperform relative to the minimax algorithm baseline, while showing mixed performance at Connect-Four. GPT-4 performs unexpectedly well at the Shapes game, but GPT-3.5 does very poorly. Also unexpectedly, both models fail to assemble or detect valid Lego structures in the LCL game. In Battleship, the models’ failure to follow game rules, especially at higher temperature settings, indicates a significant limitation in their ability to understand and apply structured game rules. The linear increase in the number of moves with temperature suggests that higher temperatures lead to greater exploration of possible moves, but do not improve strategic performance. The increase in missed wins and blocks with temperature further supports this, as greater randomness in decision-making does not enhance the models’ strategic play.

Overall, these results show that while GPT-3.5 and GPT-4 can play simple games to some extent, they struggle with more complex tasks and do not consistently apply optimal strategies. The performance gap between the models and the minimax algorithm highlights the limitations of current language models in tasks requiring precise strategic reasoning and the failure to play Battleship and LCL demonstrates a failure in rule adherence.

The primary aim of contemporary benchmarks for LLMs has been to assess these models through adaptations of Turing’s test [34], evaluating their capability to process and respond to language inputs comparably to humans. However, defining the language problem solely in these terms may overlook deeper complexities. While the transformer architecture in deep neural networks has enabled models smaller than GPT-4 to exhibit what Wilhelm von Humboldt described as the "infinite use of finite means" [21] or their ability to generate a potentially unlimited number of contextually relevant sentences [30] (an idea popularised by Chomsky [10]), this does not necessarily imply that these models have mastered a form of reasoning. Rather, they may simply be engaging in an advanced form of pattern imitation.

Non-language-based tasks are important as they challenge models to demonstrate generalization across different information encodings or forms of input, and, most importantly, to delve into out-of-training-distribution topologies. Testing LLMs like GPT-4 (according to OpenAI, the current contender to AGI [7]) beyond the text they were primarily trained on via our "show, don’t tell" strategy, we demonstrate that it is still mediocre at best at even very simple reasoning tasks that are outside of its training data. The models fail to play optimally at very simple games, such as tic-tac-toe, battleship, and connect-four. We also experimented with LEGO assembly, finding the LLMs still performing poorly. Mixed results were found at the task of interpreting geometric shapes from binary grids. These tasks are then designed to test reasoning without relying on language skills, such that the model cannot get by through parroting - it must be capable of playing the game. Currently, the "non-language" category of the BigBench benchmark shows 16 active tasks, including explicit ASCII recognition tasks, chess, and Sudoku, but, to the best of our knowledge, no task like ours [2]. Hence, we believe that ChildPlay is a useful addition to the suite of current established LLM benchmarks.

In general, this work shows that developing games allows us to critically examine claims regarding a models’ ability to reason and solve problems regardless of the persistent issue of data contamination. In other words, we explore what the model knows by making it play games instead of asking it how to play them. Our results suggest that current LLMs show disappointing performance in terms of problem solving capabilities and reveal important aspects to be considered for future improvements.