-
Human-Centered Design Recommendations for LLM-as-a-Judge
Authors:
Qian Pan,
Zahra Ashktorab,
Michael Desmond,
Martin Santillan Cooper,
James Johnson,
Rahul Nair,
Elizabeth Daly,
Werner Geyer
Abstract:
Traditional reference-based metrics, such as BLEU and ROUGE, are less effective for assessing outputs from Large Language Models (LLMs) that produce highly creative or superior-quality text, or in situations where reference outputs are unavailable. While human evaluation remains an option, it is costly and difficult to scale. Recent work using LLMs as evaluators (LLM-as-a-judge) is promising, but…
▽ More
Traditional reference-based metrics, such as BLEU and ROUGE, are less effective for assessing outputs from Large Language Models (LLMs) that produce highly creative or superior-quality text, or in situations where reference outputs are unavailable. While human evaluation remains an option, it is costly and difficult to scale. Recent work using LLMs as evaluators (LLM-as-a-judge) is promising, but trust and reliability remain a significant concern. Integrating human input is crucial to ensure criteria used to evaluate are aligned with the human's intent, and evaluations are robust and consistent. This paper presents a user study of a design exploration called EvaluLLM, that enables users to leverage LLMs as customizable judges, promoting human involvement to balance trust and cost-saving potential with caution. Through interviews with eight domain experts, we identified the need for assistance in developing effective evaluation criteria aligning the LLM-as-a-judge with practitioners' preferences and expectations. We offer findings and design recommendations to optimize human-assisted LLM-as-judge systems.
△ Less
Submitted 3 July, 2024;
originally announced July 2024.
-
Lightweight Multi-System Multivariate Interconnection and Divergence Discovery
Authors:
Mulugeta Weldezgina Asres,
Christian Walter Omlin,
Jay Dittmann,
Pavel Parygin,
Joshua Hiltbrand,
Seth I. Cooper,
Grace Cummings,
David Yu
Abstract:
Identifying outlier behavior among sensors and subsystems is essential for discovering faults and facilitating diagnostics in large systems. At the same time, exploring large systems with numerous multivariate data sets is challenging. This study presents a lightweight interconnection and divergence discovery mechanism (LIDD) to identify abnormal behavior in multi-system environments. The approach…
▽ More
Identifying outlier behavior among sensors and subsystems is essential for discovering faults and facilitating diagnostics in large systems. At the same time, exploring large systems with numerous multivariate data sets is challenging. This study presents a lightweight interconnection and divergence discovery mechanism (LIDD) to identify abnormal behavior in multi-system environments. The approach employs a multivariate analysis technique that first estimates the similarity heatmaps among the sensors for each system and then applies information retrieval algorithms to provide relevant multi-level interconnection and discrepancy details. Our experiment on the readout systems of the Hadron Calorimeter of the Compact Muon Solenoid (CMS) experiment at CERN demonstrates the effectiveness of the proposed method. Our approach clusters readout systems and their sensors consistent with the expected calorimeter interconnection configurations, while capturing unusual behavior in divergent clusters and estimating their root causes.
△ Less
Submitted 12 April, 2024;
originally announced April 2024.
-
Materials science in the era of large language models: a perspective
Authors:
Ge Lei,
Ronan Docherty,
Samuel J. Cooper
Abstract:
Large Language Models (LLMs) have garnered considerable interest due to their impressive natural language capabilities, which in conjunction with various emergent properties make them versatile tools in workflows ranging from complex code generation to heuristic finding for combinatorial problems. In this paper we offer a perspective on their applicability to materials science research, arguing th…
▽ More
Large Language Models (LLMs) have garnered considerable interest due to their impressive natural language capabilities, which in conjunction with various emergent properties make them versatile tools in workflows ranging from complex code generation to heuristic finding for combinatorial problems. In this paper we offer a perspective on their applicability to materials science research, arguing their ability to handle ambiguous requirements across a range of tasks and disciplines mean they could be a powerful tool to aid researchers. We qualitatively examine basic LLM theory, connecting it to relevant properties and techniques in the literature before providing two case studies that demonstrate their use in task automation and knowledge extraction at-scale. At their current stage of development, we argue LLMs should be viewed less as oracles of novel insight, and more as tireless workers that can accelerate and unify exploration across domains. It is our hope that this paper can familiarise material science researchers with the concepts needed to leverage these tools in their own research.
△ Less
Submitted 11 March, 2024;
originally announced March 2024.
-
SAMBA: A Trainable Segmentation Web-App with Smart Labelling
Authors:
Ronan Docherty,
Isaac Squires,
Antonis Vamvakeros,
Samuel J. Cooper
Abstract:
Segmentation is the assigning of a semantic class to every pixel in an image and is a prerequisite for various statistical analysis tasks in materials science, like phase quantification, physics simulations or morphological characterization. The wide range of length scales, imaging techniques and materials studied in materials science means any segmentation algorithm must generalise to unseen data…
▽ More
Segmentation is the assigning of a semantic class to every pixel in an image and is a prerequisite for various statistical analysis tasks in materials science, like phase quantification, physics simulations or morphological characterization. The wide range of length scales, imaging techniques and materials studied in materials science means any segmentation algorithm must generalise to unseen data and support abstract, user-defined semantic classes. Trainable segmentation is a popular interactive segmentation paradigm where a classifier is trained to map from image features to user drawn labels. SAMBA is a trainable segmentation tool that uses Meta's Segment Anything Model (SAM) for fast, high-quality label suggestions and a random forest classifier for robust, generalizable segmentations. It is accessible in the browser (https://www.sambasegment.com/) without the need to download any external dependencies. The segmentation backend is run in the cloud, so does not require the user to have powerful hardware.
△ Less
Submitted 7 December, 2023;
originally announced December 2023.
-
Mechanic Maker 2.0: Reinforcement Learning for Evaluating Generated Rules
Authors:
Johor Jara Gonzalez,
Seth Cooper,
Matthew Guzdial
Abstract:
Automated game design (AGD), the study of automatically generating game rules, has a long history in technical games research. AGD approaches generally rely on approximations of human play, either objective functions or AI agents. Despite this, the majority of these approximators are static, meaning they do not reflect human player's ability to learn and improve in a game. In this paper, we invest…
▽ More
Automated game design (AGD), the study of automatically generating game rules, has a long history in technical games research. AGD approaches generally rely on approximations of human play, either objective functions or AI agents. Despite this, the majority of these approximators are static, meaning they do not reflect human player's ability to learn and improve in a game. In this paper, we investigate the application of Reinforcement Learning (RL) as an approximator for human play for rule generation. We recreate the classic AGD environment Mechanic Maker in Unity as a new, open-source rule generation framework. Our results demonstrate that RL produces distinct sets of rules from an A* agent baseline, which may be more usable by humans.
△ Less
Submitted 4 October, 2023; v1 submitted 18 September, 2023;
originally announced September 2023.
-
Active Learning for Classifying 2D Grid-Based Level Completability
Authors:
Mahsa Bazzaz,
Seth Cooper
Abstract:
Determining the completability of levels generated by procedural generators such as machine learning models can be challenging, as it can involve the use of solver agents that often require a significant amount of time to analyze and solve levels. Active learning is not yet widely adopted in game evaluations, although it has been used successfully in natural language processing, image and speech r…
▽ More
Determining the completability of levels generated by procedural generators such as machine learning models can be challenging, as it can involve the use of solver agents that often require a significant amount of time to analyze and solve levels. Active learning is not yet widely adopted in game evaluations, although it has been used successfully in natural language processing, image and speech recognition, and computer vision, where the availability of labeled data is limited or expensive. In this paper, we propose the use of active learning for learning level completability classification. Through an active learning approach, we train deep-learning models to classify the completability of generated levels for Super Mario Bros., Kid Icarus, and a Zelda-like game. We compare active learning for querying levels to label with completability against random queries. Our results show using an active learning approach to label levels results in better classifier performance with the same amount of labeled data.
△ Less
Submitted 8 September, 2023;
originally announced September 2023.
-
Game Level Blending using a Learned Level Representation
Authors:
Venkata Sai Revanth Atmakuri,
Seth Cooper,
Matthew Guzdial
Abstract:
Game level blending via machine learning, the process of combining features of game levels to create unique and novel game levels using Procedural Content Generation via Machine Learning (PCGML) techniques, has gained increasing popularity in recent years. However, many existing techniques rely on human-annotated level representations, which limits game level blending to a limited number of annota…
▽ More
Game level blending via machine learning, the process of combining features of game levels to create unique and novel game levels using Procedural Content Generation via Machine Learning (PCGML) techniques, has gained increasing popularity in recent years. However, many existing techniques rely on human-annotated level representations, which limits game level blending to a limited number of annotated games. Even with annotated games, researchers often need to author an additional shared representation to make blending possible. In this paper, we present a novel approach to game level blending that employs Clustering-based Tile Embeddings (CTE), a learned level representation technique that can serve as a level representation for unannotated games and a unified level representation across games without the need for human annotation. CTE represents game level tiles as a continuous vector representation, unifying their visual, contextual, and behavioral information. We apply this approach to two classic Nintendo games, Lode Runner and The Legend of Zelda. We run an evaluation comparing the CTE representation to a common, human-annotated representation in the blending task and find that CTE has comparable or better performance without the need for human annotation.
△ Less
Submitted 28 June, 2023;
originally announced June 2023.
-
HRTF upsampling with a generative adversarial network using a gnomonic equiangular projection
Authors:
Aidan O. T. Hogg,
Mads Jenkins,
He Liu,
Isaac Squires,
Samuel J. Cooper,
Lorenzo Picinali
Abstract:
An individualised head-related transfer function (HRTF) is very important for creating realistic virtual reality (VR) and augmented reality (AR) environments. However, acoustically measuring high-quality HRTFs requires expensive equipment and an acoustic lab setting. To overcome these limitations and to make this measurement more efficient HRTF upsampling has been exploited in the past where a hig…
▽ More
An individualised head-related transfer function (HRTF) is very important for creating realistic virtual reality (VR) and augmented reality (AR) environments. However, acoustically measuring high-quality HRTFs requires expensive equipment and an acoustic lab setting. To overcome these limitations and to make this measurement more efficient HRTF upsampling has been exploited in the past where a high-resolution HRTF is created from a low-resolution one. This paper demonstrates how generative adversarial networks (GANs) can be applied to HRTF upsampling. We propose a novel approach that transforms the HRTF data for direct use with a convolutional super-resolution generative adversarial network (SRGAN). This new approach is benchmarked against three baselines: barycentric upsampling, spherical harmonic (SH) upsampling and an HRTF selection approach. Experimental results show that the proposed method outperforms all three baselines in terms of log-spectral distortion (LSD) and localisation performance using perceptual models when the input HRTF is sparse (less than 20 measured positions).
△ Less
Submitted 27 February, 2024; v1 submitted 9 June, 2023;
originally announced June 2023.
-
Level Assembly as a Markov Decision Process
Authors:
Colan F. Biemer,
Seth Cooper
Abstract:
Many games feature a progression of levels that doesn't adapt to the player. This can be problematic because some players may get stuck if the progression is too difficult, while others may find it boring if the progression is too slow to get to more challenging levels. This can be addressed by building levels based on the player's performance and preferences. In this work, we formulate the proble…
▽ More
Many games feature a progression of levels that doesn't adapt to the player. This can be problematic because some players may get stuck if the progression is too difficult, while others may find it boring if the progression is too slow to get to more challenging levels. This can be addressed by building levels based on the player's performance and preferences. In this work, we formulate the problem of generating levels for a player as a Markov Decision Process (MDP) and use adaptive dynamic programming (ADP) to solve the MDP before assembling a level. We tested with two case studies and found that using an ADP outperforms two baselines. Furthermore, we experimented with player proxies and switched them in the middle of play, and we show that a simple modification prior to running ADP results in quick adaptation. By using ADP, which searches the entire MDP, we produce a dynamic progression of levels that adapts to the player.
△ Less
Submitted 26 April, 2023;
originally announced April 2023.
-
Two approaches to inpainting microstructure with deep convolutional generative adversarial networks
Authors:
Isaac Squires,
Samuel J. Cooper,
Amir Dahari,
Steve Kench
Abstract:
Imaging is critical to the characterisation of materials. However, even with careful sample preparation and microscope calibration, imaging techniques are often prone to defects and unwanted artefacts. This is particularly problematic for applications where the micrograph is to be used for simulation or feature analysis, as defects are likely to lead to inaccurate results. Microstructural inpainti…
▽ More
Imaging is critical to the characterisation of materials. However, even with careful sample preparation and microscope calibration, imaging techniques are often prone to defects and unwanted artefacts. This is particularly problematic for applications where the micrograph is to be used for simulation or feature analysis, as defects are likely to lead to inaccurate results. Microstructural inpainting is a method to alleviate this problem by replacing occluded regions with synthetic microstructure with matching boundaries. In this paper we introduce two methods that use generative adversarial networks to generate contiguous inpainted regions of arbitrary shape and size by learning the microstructural distribution from the unoccluded data. We find that one benefits from high speed and simplicity, whilst the other gives smoother boundaries at the inpainting border. We also outline the development of a graphical user interface that allows users to utilise these machine learning methods in a 'no-code' environment.
△ Less
Submitted 13 October, 2022;
originally announced October 2022.
-
MicroLib: A library of 3D microstructures generated from 2D micrographs using SliceGAN
Authors:
Steve Kench,
Isaac Squires,
Amir Dahari,
Samuel J Cooper
Abstract:
3D microstructural datasets are commonly used to define the geometrical domains used in finite element modelling. This has proven a useful tool for understanding how complex material systems behave under applied stresses, temperatures and chemical conditions. However, 3D imaging of materials is challenging for a number of reasons, including limited field of view, low resolution and difficult sampl…
▽ More
3D microstructural datasets are commonly used to define the geometrical domains used in finite element modelling. This has proven a useful tool for understanding how complex material systems behave under applied stresses, temperatures and chemical conditions. However, 3D imaging of materials is challenging for a number of reasons, including limited field of view, low resolution and difficult sample preparation. Recently, a machine learning method, SliceGAN, was developed to statistically generate 3D microstructural datasets of arbitrary size using a single 2D input slice as training data. In this paper, we present the results from applying SliceGAN to 87 different microstructures, ranging from biological materials to high-strength steels. To demonstrate the accuracy of the synthetic volumes created by SliceGAN, we compare three microstructural properties between the 2D training data and 3D generations, which show good agreement. This new microstructure library both provides valuable 3D microstructures that can be used in models, and also demonstrates the broad applicability of the SliceGAN algorithm.
△ Less
Submitted 12 October, 2022;
originally announced October 2022.
-
tile2tile: Learning Game Filters for Platformer Style Transfer
Authors:
Anurag Sarkar,
Seth Cooper
Abstract:
We present tile2tile, an approach for style transfer between levels of tile-based platformer games. Our method involves training models that translate levels from a lower-resolution sketch representation based on tile affordances to the original tile representation for a given game. This enables these models, which we refer to as filters, to translate level sketches into the style of a specific ga…
▽ More
We present tile2tile, an approach for style transfer between levels of tile-based platformer games. Our method involves training models that translate levels from a lower-resolution sketch representation based on tile affordances to the original tile representation for a given game. This enables these models, which we refer to as filters, to translate level sketches into the style of a specific game. Moreover, by converting a level of one game into sketch form and then translating the resulting sketch into the tiles of another game, we obtain a method of style transfer between two games. We use Markov random fields and autoencoders for learning the game filters and apply them to demonstrate style transfer between levels of Super Mario Bros, Kid Icarus, Mega Man and Metroid.
△ Less
Submitted 15 August, 2022;
originally announced August 2022.
-
Label Sleuth: From Unlabeled Text to a Classifier in a Few Hours
Authors:
Eyal Shnarch,
Alon Halfon,
Ariel Gera,
Marina Danilevsky,
Yannis Katsis,
Leshem Choshen,
Martin Santillan Cooper,
Dina Epelboim,
Zheng Zhang,
Dakuo Wang,
Lucy Yip,
Liat Ein-Dor,
Lena Dankin,
Ilya Shnayderman,
Ranit Aharonov,
Yunyao Li,
Naftali Liberman,
Philip Levin Slesarev,
Gwilym Newton,
Shila Ofek-Koifman,
Noam Slonim,
Yoav Katz
Abstract:
Text classification can be useful in many real-world scenarios, saving a lot of time for end users. However, building a custom classifier typically requires coding skills and ML knowledge, which poses a significant barrier for many potential users. To lift this barrier, we introduce Label Sleuth, a free open source system for labeling and creating text classifiers. This system is unique for (a) be…
▽ More
Text classification can be useful in many real-world scenarios, saving a lot of time for end users. However, building a custom classifier typically requires coding skills and ML knowledge, which poses a significant barrier for many potential users. To lift this barrier, we introduce Label Sleuth, a free open source system for labeling and creating text classifiers. This system is unique for (a) being a no-code system, making NLP accessible to non-experts, (b) guiding users through the entire labeling process until they obtain a custom classifier, making the process efficient -- from cold start to classifier in a few hours, and (c) being open for configuration and extension by developers. By open sourcing Label Sleuth we hope to build a community of users and developers that will broaden the utilization of NLP models.
△ Less
Submitted 31 October, 2022; v1 submitted 2 August, 2022;
originally announced August 2022.
-
Lessons Learnt from Deploying ARI in Residential Care
Authors:
Sara Cooper,
Francesco Ferro
Abstract:
This paper describes the final prototype of an assistive robot used for increasing engagement of older adults in the context of SHAPES project. It then highlights lessons learned from hands-on training during the first phases of the pilots at Clinica Humana and Can Granada residence in Mallorca (Spain).
This paper describes the final prototype of an assistive robot used for increasing engagement of older adults in the context of SHAPES project. It then highlights lessons learned from hands-on training during the first phases of the pilots at Clinica Humana and Can Granada residence in Mallorca (Spain).
△ Less
Submitted 6 July, 2022;
originally announced July 2022.
-
Latent Combinational Game Design
Authors:
Anurag Sarkar,
Seth Cooper
Abstract:
We present latent combinational game design -- an approach for generating playable games that blend a given set of games in a desired combination using deep generative latent variable models. We use Gaussian Mixture Variational Autoencoders (GMVAEs) which model the VAE latent space via a mixture of Gaussian components. Through supervised training, each component encodes levels from one game and le…
▽ More
We present latent combinational game design -- an approach for generating playable games that blend a given set of games in a desired combination using deep generative latent variable models. We use Gaussian Mixture Variational Autoencoders (GMVAEs) which model the VAE latent space via a mixture of Gaussian components. Through supervised training, each component encodes levels from one game and lets us define blended games as linear combinations of these components. This enables generating new games that blend the input games as well as controlling the relative proportions of each game in the blend. We also extend prior blending work using conditional VAEs and compare against the GMVAE and additionally introduce a hybrid conditional GMVAE (CGMVAE) architecture which lets us generate whole blended levels and layouts. Results show that these approaches can generate playable games that blend the input games in specified combinations. We use both platformers and dungeon-based games to demonstrate our results.
△ Less
Submitted 20 December, 2023; v1 submitted 28 June, 2022;
originally announced June 2022.
-
On Linking Level Segments
Authors:
Colan Biemer,
Seth Cooper
Abstract:
An increasingly common area of study in procedural content generation is the creation of level segments: short pieces that can be used to form larger levels. Previous work has used basic concatenation to form these larger levels. However, even if the segments themselves are completable and well-formed, concatenation can fail to produce levels that are completable and can cause broken in-game struc…
▽ More
An increasingly common area of study in procedural content generation is the creation of level segments: short pieces that can be used to form larger levels. Previous work has used basic concatenation to form these larger levels. However, even if the segments themselves are completable and well-formed, concatenation can fail to produce levels that are completable and can cause broken in-game structures (e.g. malformed pipes in Mario). We show this with three tile-based games: a side-scrolling platformer, a vertical platformer, and a top-down roguelike. Additionally, we present a Markov chain and a tree search algorithm that finds a link between two level segments, which uses filters to ensure completability and unbroken in-game structures in the linked segments. We further show that these links work well for multi-segment levels. We find that this method reliably finds links between segments and is customizable to meet a designer's needs.
△ Less
Submitted 22 August, 2022; v1 submitted 9 March, 2022;
originally announced March 2022.
-
Fusion of complementary 2D and 3D mesostructural datasets using generative adversarial networks
Authors:
Amir Dahari,
Steve Kench,
Isaac Squires,
Samuel J. Cooper
Abstract:
Modelling the impact of a material's mesostructure on device level performance typically requires access to 3D image data containing all the relevant information to define the geometry of the simulation domain. This image data must include sufficient contrast between phases to distinguish each material, be of high enough resolution to capture the key details, but also have a large enough field-of-…
▽ More
Modelling the impact of a material's mesostructure on device level performance typically requires access to 3D image data containing all the relevant information to define the geometry of the simulation domain. This image data must include sufficient contrast between phases to distinguish each material, be of high enough resolution to capture the key details, but also have a large enough field-of-view to be representative of the material in general. It is rarely possible to obtain data with all of these properties from a single imaging technique. In this paper, we present a method for combining information from pairs of distinct but complementary imaging techniques in order to accurately reconstruct the desired multi-phase, high resolution, representative, 3D images. Specifically, we use deep convolutional generative adversarial networks to implement super-resolution, style transfer and dimensionality expansion. To demonstrate the widespread applicability of this tool, two pairs of datasets are used to validate the quality of the volumes generated by fusing the information from paired imaging techniques. Three key mesostructural metrics are calculated in each case to show the accuracy of this method. Having confidence in the accuracy of our method, we then demonstrate its power by applying to a real data pair from a lithium ion battery electrode, where the required 3D high resolution image data is not available anywhere in the literature. We believe this approach is superior to previously reported statistical material reconstruction methods both in terms of its fidelity and ease of use. Furthermore, much of the data required to train this algorithm already exists in the literature, waiting to be combined. As such, our open-access code could precipitate a step change by generating the hard to obtain high quality image volumes necessary to simulate behaviour at the mesoscale.
△ Less
Submitted 30 September, 2022; v1 submitted 21 October, 2021;
originally announced October 2021.
-
A Reasoning Engine for the Gamification of Loop-Invariant Discovery
Authors:
Andrew Walter,
Seth Cooper,
Panagiotis Manolios
Abstract:
We describe the design and implementation of a reasoning engine that facilitates the gamification of loop-invariant discovery. Our reasoning engine enables students, computational agents and regular software engineers with no formal methods expertise to collaboratively prove interesting theorems about simple programs using browser-based, online games. Within an hour, players are able to specify an…
▽ More
We describe the design and implementation of a reasoning engine that facilitates the gamification of loop-invariant discovery. Our reasoning engine enables students, computational agents and regular software engineers with no formal methods expertise to collaboratively prove interesting theorems about simple programs using browser-based, online games. Within an hour, players are able to specify and verify properties of programs that are beyond the capabilities of fully-automated tools. The hour limit includes the time for setting up the system, completing a short tutorial explaining game play and reasoning about simple imperative programs. Players are never required to understand formal proofs; they only provide insights by proposing invariants. The reasoning engine is responsible for managing and evaluating the proposed invariants, as well as generating actionable feedback.
△ Less
Submitted 2 September, 2021;
originally announced September 2021.
-
Procedural Content Generation using Behavior Trees (PCGBT)
Authors:
Anurag Sarkar,
Seth Cooper
Abstract:
Behavior trees (BTs) are a popular method for modeling NPC and enemy AI behavior and have been widely used in commercial games. In this work, rather than use BTs to model game playing agents, we use them for modeling game design agents, defining behaviors as content generation tasks rather than in-game actions. Similar to how traditional BTs enable modeling behaviors in a modular and dynamic manne…
▽ More
Behavior trees (BTs) are a popular method for modeling NPC and enemy AI behavior and have been widely used in commercial games. In this work, rather than use BTs to model game playing agents, we use them for modeling game design agents, defining behaviors as content generation tasks rather than in-game actions. Similar to how traditional BTs enable modeling behaviors in a modular and dynamic manner, BTs for PCG enable simple subtrees for generating parts of levels to be combined modularly to form complex trees for generating whole levels as well as generators that can dynamically vary the generated content. We refer to this approach as Procedural Content Generation using Behavior Trees, or PCGBT, and demonstrate it by using BTs to model generators for Super Mario Bros., Mega Man and Metroid levels as well as dungeon layouts and discuss several ways in which this paradigm could be applied and extended in the future.
△ Less
Submitted 7 October, 2021; v1 submitted 24 June, 2021;
originally announced July 2021.
-
Dungeon and Platformer Level Blending and Generation using Conditional VAEs
Authors:
Anurag Sarkar,
Seth Cooper
Abstract:
Variational autoencoders (VAEs) have been used in prior works for generating and blending levels from different games. To add controllability to these models, conditional VAEs (CVAEs) were recently shown capable of generating output that can be modified using labels specifying desired content, albeit working with segments of levels and platformers exclusively. We expand these works by using CVAEs…
▽ More
Variational autoencoders (VAEs) have been used in prior works for generating and blending levels from different games. To add controllability to these models, conditional VAEs (CVAEs) were recently shown capable of generating output that can be modified using labels specifying desired content, albeit working with segments of levels and platformers exclusively. We expand these works by using CVAEs for generating whole platformer and dungeon levels, and blending levels across these genres. We show that CVAEs can reliably control door placement in dungeons and progression direction in platformer levels. Thus, by using appropriate labels, our approach can generate whole dungeons and platformer levels of interconnected rooms and segments respectively as well as levels that blend dungeons and platformers. We demonstrate our approach using The Legend of Zelda, Metroid, Mega Man and Lode Runner.
△ Less
Submitted 17 June, 2021;
originally announced June 2021.
-
Robot to support older people to live independently
Authors:
Sara Cooper,
Óscar Villacañas,
Luca Marchionni,
Francesco Ferro
Abstract:
This paper presents an overview on how the PAL Robotics ARI robot is participating in the European SHAPES project to promote healthy and active living among older people, by integrating digital solutions from project partners and adapting the system in order to improve human-robot interaction and user acceptability in a wide range of tasks.
This paper presents an overview on how the PAL Robotics ARI robot is participating in the European SHAPES project to promote healthy and active living among older people, by integrating digital solutions from project partners and adapting the system in order to improve human-robot interaction and user acceptability in a wide range of tasks.
△ Less
Submitted 15 April, 2021;
originally announced April 2021.
-
Generating and Blending Game Levels via Quality-Diversity in the Latent Space of a Variational Autoencoder
Authors:
Anurag Sarkar,
Seth Cooper
Abstract:
Several works have demonstrated the use of variational autoencoders (VAEs) for generating levels in the style of existing games and blending levels across different games. Further, quality-diversity (QD) algorithms have also become popular for generating varied game content by using evolution to explore a search space while focusing on both variety and quality. To reap the benefits of both these a…
▽ More
Several works have demonstrated the use of variational autoencoders (VAEs) for generating levels in the style of existing games and blending levels across different games. Further, quality-diversity (QD) algorithms have also become popular for generating varied game content by using evolution to explore a search space while focusing on both variety and quality. To reap the benefits of both these approaches, we present a level generation and game blending approach that combines the use of VAEs and QD algorithms. Specifically, we train VAEs on game levels and run the MAP-Elites QD algorithm using the learned latent space of the VAE as the search space. The latent space captures the properties of the games whose levels we want to generate and blend, while MAP-Elites searches this latent space to find a diverse set of levels optimizing a given objective such as playability. We test our method using models for 5 different platformer games as well as a blended domain spanning 3 of these games. We refer to using MAP-Elites for blending as Blend-Elites. Our results show that MAP-Elites in conjunction with VAEs enables the generation of a diverse set of playable levels not just for each individual game but also for the blended domain while illuminating game-specific regions of the blended latent space.
△ Less
Submitted 22 July, 2021; v1 submitted 24 February, 2021;
originally announced February 2021.
-
Generating 3D structures from a 2D slice with GAN-based dimensionality expansion
Authors:
Steve Kench,
Samuel J. Cooper
Abstract:
Generative adversarial networks (GANs) can be trained to generate 3D image data, which is useful for design optimisation. However, this conventionally requires 3D training data, which is challenging to obtain. 2D imaging techniques tend to be faster, higher resolution, better at phase identification and more widely available. Here, we introduce a generative adversarial network architecture, SliceG…
▽ More
Generative adversarial networks (GANs) can be trained to generate 3D image data, which is useful for design optimisation. However, this conventionally requires 3D training data, which is challenging to obtain. 2D imaging techniques tend to be faster, higher resolution, better at phase identification and more widely available. Here, we introduce a generative adversarial network architecture, SliceGAN, which is able to synthesise high fidelity 3D datasets using a single representative 2D image. This is especially relevant for the task of material microstructure generation, as a cross-sectional micrograph can contain sufficient information to statistically reconstruct 3D samples. Our architecture implements the concept of uniform information density, which both ensures that generated volumes are equally high quality at all points in space, and that arbitrarily large volumes can be generated. SliceGAN has been successfully trained on a diverse set of materials, demonstrating the widespread applicability of this tool. The quality of generated micrographs is shown through a statistical comparison of synthetic and real datasets of a battery electrode in terms of key microstructural metrics. Finally, we find that the generation time for a $10^8$ voxel volume is on the order of a few seconds, yielding a path for future studies into high-throughput microstructural optimisation.
△ Less
Submitted 10 February, 2021;
originally announced February 2021.
-
Conditional Level Generation and Game Blending
Authors:
Anurag Sarkar,
Zhihan Yang,
Seth Cooper
Abstract:
Prior research has shown variational autoencoders (VAEs) to be useful for generating and blending game levels by learning latent representations of existing level data. We build on such models by exploring the level design affordances and applications enabled by conditional VAEs (CVAEs). CVAEs augment VAEs by allowing them to be trained using labeled data, thus enabling outputs to be generated con…
▽ More
Prior research has shown variational autoencoders (VAEs) to be useful for generating and blending game levels by learning latent representations of existing level data. We build on such models by exploring the level design affordances and applications enabled by conditional VAEs (CVAEs). CVAEs augment VAEs by allowing them to be trained using labeled data, thus enabling outputs to be generated conditioned on some input. We studied how increased control in the level generation process and the ability to produce desired outputs via training on labeled game level data could build on prior PCGML methods. Through our results of training CVAEs on levels from Super Mario Bros., Kid Icarus and Mega Man, we show that such models can assist in level design by generating levels with desired level elements and patterns as well as producing blended levels with desired combinations of games.
△ Less
Submitted 12 October, 2020;
originally announced October 2020.
-
Game Level Clustering and Generation using Gaussian Mixture VAEs
Authors:
Zhihan Yang,
Anurag Sarkar,
Seth Cooper
Abstract:
Variational autoencoders (VAEs) have been shown to be able to generate game levels but require manual exploration of the learned latent space to generate outputs with desired attributes. While conditional VAEs address this by allowing generation to be conditioned on labels, such labels have to be provided during training and thus require prior knowledge which may not always be available. In this p…
▽ More
Variational autoencoders (VAEs) have been shown to be able to generate game levels but require manual exploration of the learned latent space to generate outputs with desired attributes. While conditional VAEs address this by allowing generation to be conditioned on labels, such labels have to be provided during training and thus require prior knowledge which may not always be available. In this paper, we apply Gaussian Mixture VAEs (GMVAEs), a variant of the VAE which imposes a mixture of Gaussians (GM) on the latent space, unlike regular VAEs which impose a unimodal Gaussian. This allows GMVAEs to cluster levels in an unsupervised manner using the components of the GM and then generate new levels using the learned components. We demonstrate our approach with levels from Super Mario Bros., Kid Icarus and Mega Man. Our results show that the learned components discover and cluster level structures and patterns and can be used to generate levels with desired characteristics.
△ Less
Submitted 22 August, 2020;
originally announced September 2020.
-
Towards Game Design via Creative Machine Learning (GDCML)
Authors:
Anurag Sarkar,
Seth Cooper
Abstract:
In recent years, machine learning (ML) systems have been increasingly applied for performing creative tasks. Such creative ML approaches have seen wide use in the domains of visual art and music for applications such as image and music generation and style transfer. However, similar creative ML techniques have not been as widely adopted in the domain of game design despite the emergence of ML-base…
▽ More
In recent years, machine learning (ML) systems have been increasingly applied for performing creative tasks. Such creative ML approaches have seen wide use in the domains of visual art and music for applications such as image and music generation and style transfer. However, similar creative ML techniques have not been as widely adopted in the domain of game design despite the emergence of ML-based methods for generating game content. In this paper, we argue for leveraging and repurposing such creative techniques for designing content for games, referring to these as approaches for Game Design via Creative ML (GDCML). We highlight existing systems that enable GDCML and illustrate how creative ML can inform new systems via example applications and a proposed system.
△ Less
Submitted 25 July, 2020;
originally announced August 2020.
-
Sequential Segment-based Level Generation and Blending using Variational Autoencoders
Authors:
Anurag Sarkar,
Seth Cooper
Abstract:
Existing methods of level generation using latent variable models such as VAEs and GANs do so in segments and produce the final level by stitching these separately generated segments together. In this paper, we build on these methods by training VAEs to learn a sequential model of segment generation such that generated segments logically follow from prior segments. By further combining the VAE wit…
▽ More
Existing methods of level generation using latent variable models such as VAEs and GANs do so in segments and produce the final level by stitching these separately generated segments together. In this paper, we build on these methods by training VAEs to learn a sequential model of segment generation such that generated segments logically follow from prior segments. By further combining the VAE with a classifier that determines whether to place the generated segment to the top, bottom, left or right of the previous segment, we obtain a pipeline that enables the generation of arbitrarily long levels that progress in any of these four directions and are composed of segments that logically follow one another. In addition to generating more coherent levels of non-fixed length, this method also enables implicit blending of levels from separate games that do not have similar orientation. We demonstrate our approach using levels from Super Mario Bros., Kid Icarus and Mega Man, showing that our method produces levels that are more coherent than previous latent variable-based approaches and are capable of blending levels across games.
△ Less
Submitted 17 July, 2020;
originally announced July 2020.
-
Pores for thought: The use of generative adversarial networks for the stochastic reconstruction of 3D multi-phase electrode microstructures with periodic boundaries
Authors:
Andrea Gayon-Lombardo,
Lukas Mosser,
Nigel P. Brandon,
Samuel J. Cooper
Abstract:
The generation of multiphase porous electrode microstructures is a critical step in the optimisation of electrochemical energy storage devices. This work implements a deep convolutional generative adversarial network (DC-GAN) for generating realistic n-phase microstructural data. The same network architecture is successfully applied to two very different three-phase microstructures: A lithium-ion…
▽ More
The generation of multiphase porous electrode microstructures is a critical step in the optimisation of electrochemical energy storage devices. This work implements a deep convolutional generative adversarial network (DC-GAN) for generating realistic n-phase microstructural data. The same network architecture is successfully applied to two very different three-phase microstructures: A lithium-ion battery cathode and a solid oxide fuel cell anode. A comparison between the real and synthetic data is performed in terms of the morphological properties (volume fraction, specific surface area, triple-phase boundary) and transport properties (relative diffusivity), as well as the two-point correlation function. The results show excellent agreement between for datasets and they are also visually indistinguishable. By modifying the input to the generator, we show that it is possible to generate microstructure with periodic boundaries in all three directions. This has the potential to significantly reduce the simulated volume required to be considered representative and therefore massively reduce the computational cost of the electrochemical simulations necessary to predict the performance of a particular microstructure during optimisation.
△ Less
Submitted 4 May, 2020; v1 submitted 17 February, 2020;
originally announced March 2020.
-
Controllable Level Blending between Games using Variational Autoencoders
Authors:
Anurag Sarkar,
Zhihan Yang,
Seth Cooper
Abstract:
Previous work explored blending levels from existing games to create levels for a new game that mixes properties of the original games. In this paper, we use Variational Autoencoders (VAEs) for improving upon such techniques. VAEs are artificial neural networks that learn and use latent representations of datasets to generate novel outputs. We train a VAE on level data from Super Mario Bros. and K…
▽ More
Previous work explored blending levels from existing games to create levels for a new game that mixes properties of the original games. In this paper, we use Variational Autoencoders (VAEs) for improving upon such techniques. VAEs are artificial neural networks that learn and use latent representations of datasets to generate novel outputs. We train a VAE on level data from Super Mario Bros. and Kid Icarus, enabling it to capture the latent space spanning both games. We then use this space to generate level segments that combine properties of levels from both games. Moreover, by applying evolutionary search in the latent space, we evolve level segments satisfying specific constraints. We argue that these affordances make the VAE-based approach especially suitable for co-creative level design and compare its performance with similar generative models like the GAN and the VAE-GAN.
△ Less
Submitted 26 February, 2020;
originally announced February 2020.
-
Generalized minimum distance functions and algebraic invariants of Geramita ideals
Authors:
Susan M. Cooper,
Alexandra Seceleanu,
Stefan O. Tohaneanu,
Maria Vaz Pinto,
Rafael H. Villarreal
Abstract:
Motivated by notions from coding theory, we study the generalized minimum distance (GMD) function $δ_I(d,r)$ of a graded ideal $I$ in a polynomial ring over an arbitrary field using commutative algebraic methods. It is shown that $δ_I$ is non-decreasing as a function of $r$ and non-increasing as a function of $d$. For vanishing ideals over finite fields, we show that $δ_I$ is strictly decreasing a…
▽ More
Motivated by notions from coding theory, we study the generalized minimum distance (GMD) function $δ_I(d,r)$ of a graded ideal $I$ in a polynomial ring over an arbitrary field using commutative algebraic methods. It is shown that $δ_I$ is non-decreasing as a function of $r$ and non-increasing as a function of $d$. For vanishing ideals over finite fields, we show that $δ_I$ is strictly decreasing as a function of $d$ until it stabilizes. We also study algebraic invariants of Geramita ideals. Those ideals are graded, unmixed, $1$-dimensional and their associated primes are generated by linear forms. We also examine GMD functions of complete intersections and show some special cases of two conjectures of Tohăneanu--Van Tuyl and Eisenbud-Green-Harris.
△ Less
Submitted 11 September, 2019; v1 submitted 16 December, 2018;
originally announced December 2018.
-
The Art of Drafting: A Team-Oriented Hero Recommendation System for Multiplayer Online Battle Arena Games
Authors:
Zhengxing Chen,
Truong-Huy D Nguyen,
Yuyu Xu,
Chris Amato,
Seth Cooper,
Yizhou Sun,
Magy Seif El-Nasr
Abstract:
Multiplayer Online Battle Arena (MOBA) games have received increasing popularity recently. In a match of such games, players compete in two teams of five, each controlling an in-game avatars, known as heroes, selected from a roster of more than 100. The selection of heroes, also known as pick or draft, takes place before the match starts and alternates between the two teams until each player has s…
▽ More
Multiplayer Online Battle Arena (MOBA) games have received increasing popularity recently. In a match of such games, players compete in two teams of five, each controlling an in-game avatars, known as heroes, selected from a roster of more than 100. The selection of heroes, also known as pick or draft, takes place before the match starts and alternates between the two teams until each player has selected one hero. Heroes are designed with different strengths and weaknesses to promote team cooperation in a game. Intuitively, heroes in a strong team should complement each other's strengths and suppressing those of opponents. Hero drafting is therefore a challenging problem due to the complex hero-to-hero relationships to consider. In this paper, we propose a novel hero recommendation system that suggests heroes to add to an existing team while maximizing the team's prospect for victory. To that end, we model the drafting between two teams as a combinatorial game and use Monte Carlo Tree Search (MCTS) for estimating the values of hero combinations. Our empirical evaluation shows that hero teams drafted by our recommendation algorithm have significantly higher win rate against teams constructed by other baseline and state-of-the-art strategies.
△ Less
Submitted 26 June, 2018;
originally announced June 2018.
-
Q-DeckRec: A Fast Deck Recommendation System for Collectible Card Games
Authors:
Zhengxing Chen,
Chris Amato,
Truong-Huy Nguyen,
Seth Cooper,
Yizhou Sun,
Magy Seif El-Nasr
Abstract:
Deck building is a crucial component in playing Collectible Card Games (CCGs). The goal of deck building is to choose a fixed-sized subset of cards from a large card pool, so that they work well together in-game against specific opponents. Existing methods either lack flexibility to adapt to different opponents or require large computational resources, still making them unsuitable for any real-tim…
▽ More
Deck building is a crucial component in playing Collectible Card Games (CCGs). The goal of deck building is to choose a fixed-sized subset of cards from a large card pool, so that they work well together in-game against specific opponents. Existing methods either lack flexibility to adapt to different opponents or require large computational resources, still making them unsuitable for any real-time or large-scale application. We propose a new deck recommendation system, named Q-DeckRec, which learns a deck search policy during a training phase and uses it to solve deck building problem instances. Our experimental results demonstrate Q-DeckRec requires less computational resources to build winning-effective decks after a training phase compared to several baseline methods.
△ Less
Submitted 25 June, 2018;
originally announced June 2018.
-
The Importance of Computing Education Research
Authors:
Steve Cooper,
Jeff Forbes,
Armando Fox,
Susanne Hambrusch,
Andrew Ko,
Beth Simon
Abstract:
Interest in computer science is growing. As a result, computer science (CS) and related departments are experiencing an explosive increase in undergraduate enrollments and unprecedented demand from other disciplines for learning computing. According to the 2014 CRA Taulbee Survey, the number of undergraduates declaring a computing major at Ph.D. granting departments in the US has increased 60% fro…
▽ More
Interest in computer science is growing. As a result, computer science (CS) and related departments are experiencing an explosive increase in undergraduate enrollments and unprecedented demand from other disciplines for learning computing. According to the 2014 CRA Taulbee Survey, the number of undergraduates declaring a computing major at Ph.D. granting departments in the US has increased 60% from 2011-2014 and the number of degrees granted has increased by 34% from 2008-2013. However, this growth is not limited to higher education. New York City, San Francisco and Oakland public schools will soon be offering computer science to all students at all schools from preschool to 12th grade, although it will be an elective for high school students. This unprecedented demand means that CS departments are likely to teach not only more students in the coming decades, but more diverse students, with more varied backgrounds, motivations, preparations, and abilities.
This growth is an unparalleled opportunity to expand the reach of computing education. However, this growth is also a unique research challenge, as we know very little about how best to teach our current students, let alone the students soon to arrive. The burgeoning field of Computing Education Research (CER) is positioned to address this challenge by answering research questions such as, how should we teach computer science, from programming to advanced principles, to a broader and more diverse audience? We argue that computer science departments should lead the way in establishing CER as a foundational research area of computer science, discovering the best ways to teach CS, and inventing the best technologies with which to teach it. This white paper provides a snapshot of the current state of CER and makes actionable recommendations for academic leaders to grow CER as a successful research area in their departments.
△ Less
Submitted 12 April, 2016;
originally announced April 2016.
-
The Machine as Data: A Computational View of Emergence and Definability
Authors:
S. Barry Cooper
Abstract:
Turing's (1936) paper on computable numbers has played its role in underpinning different perspectives on the world of information. On the one hand, it encourages a digital ontology, with a perceived flatness of computational structure comprehensively hosting causality at the physical level and beyond. On the other (the main point of Turing's paper), it can give an insight into the way in which hi…
▽ More
Turing's (1936) paper on computable numbers has played its role in underpinning different perspectives on the world of information. On the one hand, it encourages a digital ontology, with a perceived flatness of computational structure comprehensively hosting causality at the physical level and beyond. On the other (the main point of Turing's paper), it can give an insight into the way in which higher order information arises and leads to loss of computational control - while demonstrating how the control can be re-established, in special circumstances, via suitable type reductions. We examine the classical computational framework more closely than is usual, drawing out lessons for the wider application of information-theoretical approaches to characterizing the real world. The problem which arises across a range of contexts is the characterizing of the balance of power between the complexity of informational structure (with emergence, chaos, randomness and 'big data' prominently on the scene) and the means available (simulation, codes, statistical sampling, human intuition, semantic constructs) to bring this information back into the computational fold. We proceed via appropriate mathematical modelling to a more coherent view of the computational structure of information, relevant to a wide spectrum of areas of investigation.
△ Less
Submitted 20 June, 2015;
originally announced June 2015.
-
Incomputability after Alan Turing
Authors:
S. Barry Cooper
Abstract:
Incomputability as a mathematical notion arose from work of Alan Turing and Alonzo Church in the 1930s. Like Turing himself, it attracted less attention than it deserved beyond the confines of mathematics. Today our experiences in computer science, physics, biology, artificial intelligence, economics and the humanities point to the importance of the notion for understanding the world around us. Th…
▽ More
Incomputability as a mathematical notion arose from work of Alan Turing and Alonzo Church in the 1930s. Like Turing himself, it attracted less attention than it deserved beyond the confines of mathematics. Today our experiences in computer science, physics, biology, artificial intelligence, economics and the humanities point to the importance of the notion for understanding the world around us. This article takes a Turing centenary look at how the interface between the computable world and the incomputable formed a central theme in Turing's work - from the early establishment of the standard model of the stored program computer, to the late work on emergence of form in nature, and the first approaches to understanding and simulating human intelligence. Turing's thinking was remarkably prescient, and his legacy still impacts on much of our work. The incomputable may turn out to be a specially important part of the legacy, with consequences that Alan Turing himself could not have envisaged.
△ Less
Submitted 22 April, 2013;
originally announced April 2013.
-
The Mathematician's Bias - and the Return to Embodied Computation
Authors:
S. Barry Cooper
Abstract:
There are growing uncertainties surrounding the classical model of computation established by Gödel, Church, Kleene, Turing and others in the 1930s onwards. The mismatch between the Turing machine conception, and the experiences of those more practically engaged in computing, has parallels with the wider one between science and those working creatively or intuitively out in the 'real' world. The s…
▽ More
There are growing uncertainties surrounding the classical model of computation established by Gödel, Church, Kleene, Turing and others in the 1930s onwards. The mismatch between the Turing machine conception, and the experiences of those more practically engaged in computing, has parallels with the wider one between science and those working creatively or intuitively out in the 'real' world. The scientific outlook is more flexible and basic than some understand or want to admit. The science is subject to limitations which threaten careers. We look at embodiment and disembodiment of computation as the key to the mismatch, and find Turing had the right idea all along - amongst a productive confusion of ideas about computation in the real and the abstract worlds.
△ Less
Submitted 19 April, 2013;
originally announced April 2013.
-
What Makes a Computation Unconventional?
Authors:
S. Barry Cooper
Abstract:
A coherent mathematical overview of computation and its generalisations is described. This conceptual framework is sufficient to comfortably host a wide range of contemporary thinking on embodied computation and its models.
A coherent mathematical overview of computation and its generalisations is described. This conceptual framework is sufficient to comfortably host a wide range of contemporary thinking on embodied computation and its models.
△ Less
Submitted 11 March, 2013;
originally announced March 2013.
-
Proceedings Sixth Workshop on Developments in Computational Models: Causality, Computation, and Physics
Authors:
S. Barry Cooper,
Prakash Panangaden,
Elham Kashefi
Abstract:
DCM 2010 provides a forum for ideas about new computing means and models, with a particular emphasis in 2010 on computational and causal models related to physics and biology. We believe that bringing together different approaches - in a community with the strong foundational background characteristic of FLoC - results in inspirational cross-boundary exchanges, and innovative further research. Day…
▽ More
DCM 2010 provides a forum for ideas about new computing means and models, with a particular emphasis in 2010 on computational and causal models related to physics and biology. We believe that bringing together different approaches - in a community with the strong foundational background characteristic of FLoC - results in inspirational cross-boundary exchanges, and innovative further research. Day two of this pre-FLoC 2010 workshop is given over to physics and quantum related computation. The content of day one is more typical of previous DCM workshops - covering a full spectrum of topics related to the development of new computational models or new features for traditional computational models. DCM 2010 was designed to foster interactions, and provide a forum for presenting new ideas and work in progress. It is also intended to enable newcomers to learn about current research in this area.
△ Less
Submitted 23 July, 2010; v1 submitted 9 June, 2010;
originally announced June 2010.
-
Proceedings Fifth Workshop on Developments in Computational Models--Computational Models From Nature
Authors:
S. Barry Cooper,
Vincent Danos
Abstract:
The special theme of DCM 2009, co-located with ICALP 2009, concerned Computational Models From Nature, with a particular emphasis on computational models derived from physics and biology. The intention was to bring together different approaches - in a community with a strong foundational background as proffered by the ICALP attendees - to create inspirational cross-boundary exchanges, and to lea…
▽ More
The special theme of DCM 2009, co-located with ICALP 2009, concerned Computational Models From Nature, with a particular emphasis on computational models derived from physics and biology. The intention was to bring together different approaches - in a community with a strong foundational background as proffered by the ICALP attendees - to create inspirational cross-boundary exchanges, and to lead to innovative further research. Specifically DCM 2009 sought contributions in quantum computation and information, probabilistic models, chemical, biological and bio-inspired ones, including spatial models, growth models and models of self-assembly. Contributions putting to the test logical or algorithmic aspects of computing (e.g., continuous computing with dynamical systems, or solid state computing models) were also very much welcomed.
△ Less
Submitted 14 November, 2009;
originally announced November 2009.
