Show, Don’t Tell: Evaluating Large Language Models Beyond Textual Understanding with ChildPlay

Typically, Large Language Models (LLMs) are transformer-based - they process input text and generate output text in a coherent and contextually appropriate manner [8]. Modern versions use a self-attention mechanism to weigh the importance of different words in a sentence relative to each other [53, 8]. Input text is tokenized, converted into vectors using embeddings, and processed through transformer layers that calculate attention scores to dictate focus on relevant tokens [53, 8, 16]. The model then selects the next token based on learned distributions [53, 8, 16]. With their large parameter count, from Alpaca with 7 billion parameters [46], to LLaMA with 65 billion [49] or PaLM with 540 billion parameters [13], these neural networks have learned to model complex linguistic abstractions, capturing patterns in syntax, semantics, pragmatics, and elements of both style and tone [8, 9, 37].

Current Benchmarks  Benchmarks consist of tests or tasks along with their associated metrics, used to comparatively evaluate a system. They help identify the relative state of the art by comparing systems based on specific measurements. Datasets like SQuAD, GLUE, BIG-bench, SuperGLUE, ANLI, TruthfulQA, HellaSwag, and the lm-evaluation-harness offer various test types, including multiple-choice questions, reading comprehension tasks, natural language understanding, common sense reasoning, factual knowledge, and dialogue completion tasks [38, 54, 2, 54, 34, 28, 59, 18].

Recent studies have evaluated LLMs’ reasoning in non-linguistic tasks too. Liga and Pasetto used Tic-Tac-Toe in ASCII form, pitting LLMs against the minimax algorithm to explore emergent features, suggested to be akin to consciousness [27]. Although LLMs performed well, they often failed to win against the minimax algorithm, resulting in draws and losses [27]. Topsakal and Harper [48] found GPT-4 secured more wins than GPT-3.5 but did not consistently play optimally.

Some studies, such as those by Lappin et al. and Zečević et al., caution against overestimating LLMs’ capabilities, stating that LLMs may excel at language tasks but seem to struggle with reasoning and causal understanding [26, 60]. Bender et al. and Schaeffer et al. further critique the transparency of evaluation metrics, arguing that some "emergent abilities" may be artifacts of evaluation rather than genuine model behavior [3, 39].

Another issue in evaluating LLMs is that of training-test set cross-contamination [8]. Massive training datasets, often untraceable, make replication studies difficult and risk rendering benchmarks meaningless due to contamination [8, 17]. N-Gram Overlap is one method to detect contamination, but it has limitations, such as failing to account for context and synonymous text [6]. Furthermore, the arbitrary 200-character window classically used during GPT-3.5 training may not accurately reflect the influence of surrounding text [8].

This critique of the evaluation metrics used in assessing LLMs invites a deeper exploration of general intelligence - specifically how it can be reliably measured and observed in AI through rigorous and realistic tests that extend beyond linguistic prowess to include broader cognitive functions. Typically, when defining general intelligence (GI), the "g factor" is used as a measure of the ability to reason, plan, solve problems, think abstractly, and learn quickly across a wide range of domains [57, 5, 55, 11, 10]. GI then involves higher-order cognitive processes that go beyond specific skills or knowledge domains [19, 23].

We argue that there is a need for nuance in current debates surrounding AGI (Artificial General Intelligence) and a pragmatic perspective on understanding LLMs’ capabilities. In order to approximate some measurement of GI in an AI system, it is important that we build benchmarks that allow measurements that can truly gauge generalization and reasoning in a human-like manner, rather than relying solely on pattern matching and statistical correlations [52].

This Work  We introduce ChildPlay, a light and scalable suite of non-language-based games like Tic-Tac-Toe, Connect-Four, Battleship, LEGO Connect Language, and Shapes, to assess LLMs’ reasoning, strategy, symbolic reasoning, and pattern recognition beyond traditional linguistic tasks. The benchmarks can be executed, taking only a few hours to complete across all tasks for a given model. Furthermore, we have proposed this benchmark suite for inclusion in the BIG-bench initiative [2], which aims to standardize challenging benchmarks for LLMs.

We chose games because they provide structured environments with clear success criteria, suitable for evaluating strategic thinking and decision-making in dynamic, adversarial scenarios [43, 27, 48]. We encode these games using ASCII representations to minimize dataset contamination issues prevalent in testing generalized intelligence beyond the training domain [8, 27].

While our benchmarks attempt to probe the model’s understanding indirectly through win ratios, gameplay is inherently limited by the model’s reliance on probabilistic patterns rather than genuine rule-based reasoning, something that is contentious in the field at the moment and that we attempt to probe. The game rules are explicitly given to the models in the prompts, which we believe is a critical component of evaluating how well LLMs can generalize from given instructions. While this does not fully guarantee that the models are learning and applying these rules as a human would, it still probes generalization ability.

General Methodology  Specific tasks in the BIG-bench benchmark [2] are categorized as zero-shot, one-shot, or multi-shot [8]. Our tasks, available through the ChildPlay GitHub Repository, fit the zero-shot category, where models receive only a brief explanation at inference time with no examples. To test LLMs’ reasoning beyond their training data, we focused on spatial reasoning with ASCII sequences. An agent capable of true abstraction should encode and interpret these sequences when the rules are explained.

We developed tasks including LEGO assembly, ASCII games (Tic-Tac-Toe, Connect-Four, Battleship), and identifying shapes represented as 1s in 15-sided grids of 0s. The models tested were gpt-3.5-turbo-1106 (GPT-3.5) and gpt-4-1106-preview (GPT-4), across temperatures t = 0, 0.5, 1, and 1.5 (refer to Appendix G for a preliminary analysis on models other than GPT-3.5 and GPT-4). Temperature is a hyperparameter that controls the randomness of a model’s output in text generation. Lower temperatures make the model more deterministic, favoring the most probable tokens. Importantly, when queried, both GPT-3.5 and GPT-4 were able to explain the tasks and generate valid board states, validating our benchmarks. We hypothesize that models capable of reasoning should play optimally, given their ability to explain optimal strategies when prompted (see Appendix B.5). In general, these experiments took between a few minutes to a few hours to run. Error bars are based on the binomial distribution and computed using the standard deviation.

Lego Connect Language (LCL)  We invented a formal language we call LEGO Connect Language (LCL). More specifically, we propose $LCL_{2}$ as a language to instruct assembly in 2D on the x and y axis (this can be extended to $LCL_{3}$ or 3D). Only 2x4 pieces are allowed when assembling a construction, $M$, which is then valid if no pieces are overlapping and all pieces are connected to other pieces. Namely, a Lego piece is connected through interlocking pegs, not by merely touching sides. And secondly, two Lego pieces overlap when they share the same y-coordinate and any part of their length has the same x-coordinate (see Appendix D.1 for a formal definition of LCL).

Game 1: Validity Testing  This test evaluates the ability to assess constructs as valid or invalid.

Game 2: Construct Generation  Models generate LCL constructs from prompts. Each construct is described by tuples specifying the coordinates and colors of pieces. Construct validity is then automatically evaluated.

In both games, the evaluation metric is the proportion of correct validations across different temperatures. We generated 800 images for the validity test (half valid, half invalid) and queried each model to produce 100 images at each temperature. These were checked for validity. An example for data used in these tasks is $[(1,1,^{\prime}red^{\prime}),(2,1,^{\prime}blue^{\prime}),(3,1,^{\prime}green^{\prime})]$, where each tuple represents a brick’s position and color. Our use of LCL is related to Bubeck et al. [9], where GPT-4 generated images using JavaScript or LaTeX. However, unlike Bubeck et al.’s examples, which are prone to training data contamination, LCL remains abstract, challenging the model to operate outside its learned distributions.

Board Games: Tic-tac-toe, Connect-four, and Battleship (BTC)  For this set of games, which we will refer to as BTC from now on, each new board state was introduced with an initial game explanation via the OpenAI API in a zero-shot setup. The models, always playing as the first player, were provided with the current board state and faced an opponent making random moves. Since these games are fully observable, only the initial instruction and the current board state were needed for optimal play. Inputs required were two scalars for the row-column pair or a single scalar for the column in Connect-Four.

In Battleship, ships (’S’) were randomly initialized horizontally, varying from 2 to 5 cells. Hits were marked with an ’X’ and misses with an ’O’ on both players’ boards. The symbol ’~’ denoted the sea - an unexplored square.

The Game of Shapes  For the game of shapes, we first probed the models about what geometric shapes they consider basic. The three shapes consistently mentioned were square, circle, and triangle. The game consists of finding a basic geometric shape "hidden" behind 1s in a matrix of 0s in a multiple-choice fashion. Four shapes were used as options: the circle, the rectangle, the triangle, and the cross. Only the latter three were ever shown to the model (cf. Fig. 3). The choice positions are not randomized to test if the model displays any inherent bias for the question order. This does not affect the outcome, as the game is designed to operate within a single question-response framework.

GPT move biases when playing BTC are shown in the heatmaps in Appendix B.3. The Random Player’s uniform distribution of moves across squares and columns appears to make it less likely to counter GPT’s "strategies", especially in Connect-Four. According to these heatmaps, GPT-3.5 makes more mistakes than GPT-4 at Tic-Tac-Toe, namely because it seems to play the middle row even when occupied, which produces an automatic loss. Furthermore, the heatmaps reveal that GPT models often play the middle column in Connect-Four, even if already full, increasing the likelihood of connecting four pieces before the Random Player, who has just a $1/7$ chance of blocking. Errors, especially from GPT-3.5, generally increase with temperature, likely due to increased randomness in move selection.

As shown in Table 1, Performances vary significantly across games and temperatures. In Battleship, GPT-3.5’s win rate drops from 10% at $t=0.0$ to 3% at $t=1.0$ and $t=1.5$. Surprisingly, GPT-4 performs even worse, with no wins at $t=0.0$, $t=0.5$, and $t=1.5$, and only 4% at $t=1.0$. In Tic-Tac-Toe, GPT-3.5’s win rate decreases from 53% at $t=0.0$ to 19% at $t=1.5$. GPT-4 performs better, peaking at 77% at $t=0.5$ but dropping to 50% at $t=1.5$. In Connect-Four, GPT-3.5 shows an improvement from 64% at $t=0.0$ to 76% at $t=0.5$, maintaining around 75% at higher temperatures, while GPT-4 starts strong with 80% at $t=0.0$ but falls to 39% at $t=1.5$.

Overall, GPT-4 outperforms GPT-3.5 in Tic-Tac-Toe and Connect-Four at lower temperatures ($t=0.0$ and $t=0.5$), but GPT-3.5 shows better performance in Battleship at $t=0.0$ and tends to be more stable in Connect-Four at higher temperatures. Both models struggle as temperature increases, with significant declines in win rates, especially in Battleship and Tic-Tac-Toe. Importantly, both models often underperform compared to a random player, particularly at higher temperatures, which underscores the challenges they face in maintaining effective strategies as the temperature rises.

Tic-Tac-Toe  For comparison with the model’s performance, Fig. 12 presents the Tic-Tac-Toe match results of the minimax algorithm against the same random player the models played against. The algorithm won 996 games out of 1000, tied 4, and lost none. This also replicates the results of previous research on Tic-Tac-Toe and we use that as a baseline for optimal play against a random player (see Appendix A.7.1) [44, 1].

Both GPT-3.5 and GPT-4 perform poorly in Tic-Tac-Toe at t=0, reflecting deterministic play based on learned strategies (see Appendix B.3). The Random Player’s uniform move distribution (see Fig. B.3.2) reduces the likelihood of countering GPT’s central strategies, with GPT-3.5 making more errors than GPT-4. These errors significantly impact outcomes due to incorrect moves (cf. Fig. 4). Errors increase with temperature, likely due to an increase in randomness (cf. Fig. 14), leading to losses from illegal moves rather than direct defeats.

Average game moves, missed wins and missed blocks in Tic-Tac-Toe decrease as temperature rises (see Fig. 5), because of an increase in losses due to random play leading to illegal moves. Regardless, neither model plays optimally, as indicated by the high number of missed wins and blocks. As temperature increases, missed wins and blocks decrease as models move away from deterministic play illustrated in the heatmaps of moves in Appendix B.3.

Connect-Four  Both GPT-3.5 and GPT-4 exhibit their poorest performance in Connect-Four at temperature 0, indicative of deterministic play reflecting the models’ learned behavior (see Appendix B.3.3). The Random Player’s uniform distribution across columns (Fig. 16) implies a lower chance of countering GPT’s central strategies in Connect-Four. Neither model plays Connect-Four optimally, as evidenced by the many missed wins and blocks. Fig. 7 demonstrates that, as temperature increases, missed wins and blocks increase for GPT-3.5 and remains roughly the same for GPT-4.

Battleship  The models were incapable of following the Battleship rules. Regardless of temperature, the models lose most games, with GPT-4 not winning any game due to incorrect moves (cf. Fig. 19). GPT-3.5 wins 10% of the matches at low temperatures, but none at higher ones (cf. Fig. 8).

Shapes  In the Shapes game, we see that GPT-3.5’s performance was approximately equivalent to random chance when identifiying triangles and crosses, yet, it failed to recognize most squares (cf. Fig 9). In contrast, GPT-4 performed remarkably well, successfully identifying shapes with an accuracy of $\approx$ 80%, demonstrating particular proficiency at recognizing triangles. At higher temperatures, some of GPT-4’s responses were discarded by our parser when the model generated invalid Unicode output, and, thus, were not included in the final evaluation. This discrepancy is evident in Fig. 9, for instance, where the sum of correct and incorrect choices does not total 25 at high temperatures.

LCL  In LCL, both models failed to respect the two rules (cf. Section 2). Figs. 10(a), and 10(b) generated for the validity test show valid LCL, while Fig. 10(c) shows an invalid LCL structure. These were used in their formal encoding (cf. Appendix D.1) as model input. Figs. 10(d) and 10(f) show examples for invalid output from GPT-3.5 and Fig. 10(e) shows the only type of valid output generated by GPT-4. The second invalid example for GPT-3.5 was also generated by GPT-4 at $t=1$.

Fig. 11 shows a roughly linear increase in the proportion of correct answers for GPT-3.5 during the validity test as a function of temperature. While GPT-4 peaks at temperature 0.5 and then declines. Regardless of these increases, the percentage of correct responses is minimal. Only GPT-4 produced a small fraction of valid LCL constructs (namely 0.04 out of 400 = 16). GPT-3.5 did not produce a single valid LCL construct.

Just as the presently ongoing ARC-AGI competition [21] attempts to test systems under a definition of GI akin to the one we offer in our introduction (processes equivalent to higher-order cognition enabling general problem solving that goes beyond specific skills or knowledge domains), our work too explores AI reasoning capabilities through methods that differ from typical benchmarks. Interestingly, both currently highlight deep learning limitations (at the moment no solution, with or without deep learning, goes beyond 43% accuracy, defined as the percentage of correct predictions [24]), particularly in abstract reasoning. The failure of these systems in ARC tasks aligns with our ChildPlay findings, where LLMs struggle with strategic reasoning and rule adherence, underscoring challenges in achieving true generalization beyond trained data distributions.

In Tic-Tac-Toe, both models underperform compared to the minimax algorithm, with mixed results in Connect-Four. GPT-4 performs well in Shapes, but GPT-3.5 does poorly. Both fail in LCL, and Battleship, at any temperatures, showing significant limitations in rule understanding. Increased temperature leads to more naive exploration but does not improve strategic performance, as seen in the rise of missed wins and blocks.

Overall, GPT-3.5 and GPT-4 can produce output for simple games but struggle to do so sensibly, often failing to apply optimal strategies such as blocking winning moves or going for winning move. In summary, the performance difference to minimax underscores LLM limitations in strategic reasoning. Failures in Battleship and LCL highlight severe issues in rule adherence.

Modern LLM benchmarks primarily adapt Turing’s test [50] to assess language processing, potentially overlooking deeper complexities. While smaller models like GPT-4 exhibit what Humboldt described as the "infinite use of finite means" [29] as popularized by Chomsky [12], generating contextually relevant sentences [45], this does not necessarily mean they have mastered reasoning - it is more likely to be advanced pattern imitation.

Limitations and Future Work  Just as in ARC-AI [21], in the future we intend to generate a holdout game dataset that is kept hidden, only providing tools for evaluation, such that overfitting on it before testing becomes unlikely. Regarding our general approach, we pursued a zero-shot condition, but future work should test Chain-of-Thought (CoT), Tree-of-Thoughts (ToT), and even Few-Shot learning [56, 58, 8]. These methodologies reduce task complexity and may lead to improved LLM performance. LLMs might just need more clues. The current technical disadvantage of any testing setting is that they allow LLMs to resort to probabilistic patterns rather than genuine reasoning. This might not be an issue of the class of methods, but simply a direct product of the underlying transformer algorithm which narrows in on the correct data manifold by using input tokens as features for arriving at the most likely answer given the training data, allowing the models to approximate reasoning patterns, likely without true understanding.

Regarding input tokens, ChildPlay sometimes uses single character output, leading to what we term single character token myopia. This is problematic when encoding state information as single characters because GPT has a multi-character tokenizer as do most LLMs [8].

On the subject of evaluation, our benchmark’s binary outcomes (win/loss) can bias perceived capabilities. To avoid this, we also assess move choices and missed wins/blocks (see Appendix B.3 and Figs. 5, 7). Using discontinuous metrics in strategic games may cause sharp evaluation transitions, potentially misrepresenting gradual improvements. Shape recognition and LCL tasks might benefit from continuous metrics, providing smoother performance gradients and more accurate reflections of reasoning abilities. If the model outputs continuous values (e.g., coordinates of a shape’s vertices), the mean squared error, for example, can be used to quantify the difference between predicted and actual values.

Furthermore, following Schaeffer et al.’s perspective, one might argue that the ChildPlay games may not fully reflect true generalization or emergent abilities [39]. If benchmarks resemble nonlinear metrics, they could exaggerate LLM weaknesses or strengths. For instance, a sharp failure in the Battleship game might not indicate poor strategic reasoning universally but rather in specific conditions. We believe this not to be the case as the experiments consist of simple fully observable games. Additionally, we aimed to diminish this effect by producing heatmaps of the moves and keeping track of illegal moves, missed blocks, and missed wins. Regarding the performance difference between the models at the shapes game, we believe this is best rationalized through data-contamination. We think it is likely that GPT-4’s training data contained examples similar to the ones used. This would explain why GPT-4 only improved dramatically at that game - it already knew the answers.

Future ChildPlay enhancements could involve feedback on actions, reducing task complexity. We have translated a small dataset from the Lego corporation (see Appendix D) for future reference and development of LCL. The LCL dataset can also be simulated in both 2D and 3D as a multimodal benchmark. Another important aspect is our ASCII representation. It might not be well-suited to convey the meaning of a board state. More complex symbolic representations might benefit the models’ performance. We also plan to benchmark additional models (cf. Appendix G for a short preliminary analysis on models other than GPT-3.5 and GPT-4), including open-source LLMs, and explore deep reinforcement learning as in Schrittwieser et al., Kaiser et al., and Silver et al. [40, 25, 42].

Non-language tasks challenge models to generalize across different encodings and delve into out-of-training-distribution scenarios. Testing LLMs like GPT-4 (according to OpenAI, a contender for AGI [9]) beyond text, we show they are mediocre at best at simple reasoning tasks outside their training data. Both GPT-4 and GPT-3.5 fail to play simple games optimally, and struggle with following rules, namely assembly instructions. Mixed results were found in interpreting geometric shapes from binary grids, where GPT-4 did particularly well. These tasks test reasoning without relying on language, forcing the model to play the game rather than describe it, which we believe is akin to repeat learned information, also known as "parroting" [7, 14, 4, 20, 60, 3]. . The "non-language" category of the BigBench benchmark includes tasks like ASCII recognition, chess, and Sudoku, but none like ours [2]. Hence, we believe ChildPlay is a valuable addition to current LLM benchmarks.

Developing games allows us to critically assess claims about a model’s reasoning and problem-solving abilities, avoiding data contamination issues. Our results suggest current LLMs show disappointing problem-solving capabilities, highlighting key areas for future improvement.

The key takeaway of our work is not just the models’ performance in specific games, but understanding their ability to generalize and reason in novel contexts. This has broader implications for AI, particularly in developing models capable of true reasoning and adaptive thinking. Our research highlights how LLMs handle tasks requiring higher-order reasoning, which is crucial for advanced applications. Practically, if GPT or other LLMs cannot reason accurately on novel data, they should not be trusted to do so in standard applications. This is critical, especially in scientific contexts, as many users may be biased toward believing LLM outputs because they appear credible. Our benchmarks illustrate these pitfalls, showing that if LLMs struggle with simple games, users should be more critical of their outputs.