-
Examination of Code generated by Large Language Models
Authors:
Robin Beer,
Alexander Feix,
Tim Guttzeit,
Tamara Muras,
Vincent Müller,
Maurice Rauscher,
Florian Schäffler,
Welf Löwe
Abstract:
Large language models (LLMs), such as ChatGPT and Copilot, are transforming software development by automating code generation and, arguably, enable rapid prototyping, support education, and boost productivity. Therefore, correctness and quality of the generated code should be on par with manually written code. To assess the current state of LLMs in generating correct code of high quality, we cond…
▽ More
Large language models (LLMs), such as ChatGPT and Copilot, are transforming software development by automating code generation and, arguably, enable rapid prototyping, support education, and boost productivity. Therefore, correctness and quality of the generated code should be on par with manually written code. To assess the current state of LLMs in generating correct code of high quality, we conducted controlled experiments with ChatGPT and Copilot: we let the LLMs generate simple algorithms in Java and Python along with the corresponding unit tests and assessed the correctness and the quality (coverage) of the generated (test) codes. We observed significant differences between the LLMs, between the languages, between algorithm and test codes, and over time. The present paper reports these results together with the experimental methods allowing repeated and comparable assessments for more algorithms, languages, and LLMs over time.
△ Less
Submitted 29 August, 2024;
originally announced August 2024.
-
Cultivating the Garden of Eden
Authors:
Randall D. Beer
Abstract:
Garden of Eden (GOE) states in cellular automata are grid configurations which have no precursors, that is, they can only occur as initial conditions. Finding individual configurations that minimize or maximize some criterion of interest (e.g., grid size, density, etc.) has been a popular sport in recreational mathematics, but systematic studies of the set of GOEs for a cellular automaton have bee…
▽ More
Garden of Eden (GOE) states in cellular automata are grid configurations which have no precursors, that is, they can only occur as initial conditions. Finding individual configurations that minimize or maximize some criterion of interest (e.g., grid size, density, etc.) has been a popular sport in recreational mathematics, but systematic studies of the set of GOEs for a cellular automaton have been rare. This paper presents the current results of an ongoing computational study of GOE configurations in Conway's Game of Life (GoL) cellular automaton. Specifically, we describe the current status of a map of the layout of GOEs and nonGOEs in density/size space, characterize how the density-dependent structure of the number of precursors varies with increasing grid size as we approach the point where GOEs begin to occur, provide a catalog of all known GOE configurations up to a grid size of 11x11, and initiate a study of the structure of the network of constraints that characterize GOE vs. nonGOE configurations.
△ Less
Submitted 14 October, 2022;
originally announced October 2022.
-
The Global Structure of Codimension-2 Local Bifurcations in Continuous-Time Recurrent Neural Networks
Authors:
Randall D. Beer
Abstract:
If we are ever to move beyond the study of isolated special cases in theoretical neuroscience, we need to develop more general theories of neural circuits over a given neural model. The present paper considers this challenge in the context of continuous-time recurrent neural networks (CTRNNs), a simple but dynamically-universal model that has been widely utilized in both computational neuroscience…
▽ More
If we are ever to move beyond the study of isolated special cases in theoretical neuroscience, we need to develop more general theories of neural circuits over a given neural model. The present paper considers this challenge in the context of continuous-time recurrent neural networks (CTRNNs), a simple but dynamically-universal model that has been widely utilized in both computational neuroscience and neural networks. Here we extend previous work on the parameter space structure of codimension-1 local bifurcations in CTRNNs to include codimension-2 local bifurcation manifolds. Specifically, we derive the necessary conditions for all generic local codimension-2 bifurcations for general CTRNNs, specialize these conditions to circuits containing from one to four neurons, illustrate in full detail the application of these conditions to example circuits, derive closed-form expressions for these bifurcation manifolds where possible, and demonstrate how this analysis allows us to find and trace several global codimension-1 bifurcation manifolds that originate from the codimension-2 bifurcations.
△ Less
Submitted 8 November, 2021;
originally announced November 2021.
-
A modeler's guide to handle complexity in energy systems optimization
Authors:
Leander Kotzur,
Lars Nolting,
Maximilian Hoffmann,
Theresa Groß,
Andreas Smolenko,
Jan Priesmann,
Henrik Büsing,
Robin Beer,
Felix Kullmann,
Bismark Singh,
Aaron Praktiknjo,
Detlef Stolten,
Martin Robinius
Abstract:
The determination of environmentally- and economically-optimal energy system designs and operations is complex. In particular, the integration of weather-dependent renewable energy technologies into energy system optimization models presents new challenges to computational tractability that cannot only be solved by advancements in computational resources. In consequence, energy system modelers mus…
▽ More
The determination of environmentally- and economically-optimal energy system designs and operations is complex. In particular, the integration of weather-dependent renewable energy technologies into energy system optimization models presents new challenges to computational tractability that cannot only be solved by advancements in computational resources. In consequence, energy system modelers must tackle the complexity of their models daily and introduce various methods to manipulate the underlying data and model structure, with the ultimate goal of finding optimal solutions. As which complexity reduction method is suitable for which research question is often unclear, herein we review some approaches to handling complexity. Thus, we first analyze the determinants of complexity and note that many drivers of complexity could be avoided a priori with a tailored model design. Second, we conduct a review of systematic complexity reduction methods for energy system optimization models, which can range from simple linearization performed by modelers to sophisticated multi-level approaches combining aggregation and decomposition methods. Based on this overview, we develop a guide for modelers who encounter computational limitations.
△ Less
Submitted 17 August, 2021; v1 submitted 15 September, 2020;
originally announced September 2020.
-
Computing Aggregate Properties of Preimages for 2D Cellular Automata
Authors:
Randall D. Beer
Abstract:
Computing properties of the set of precursors of a given configuration is a common problem underlying many important questions about cellular automata. Unfortunately, such computations quickly become intractable in dimension greater than one. This paper presents an algorithm --- incremental aggregation --- that can compute aggregate properties of the set of precursors exponentially faster than na{…
▽ More
Computing properties of the set of precursors of a given configuration is a common problem underlying many important questions about cellular automata. Unfortunately, such computations quickly become intractable in dimension greater than one. This paper presents an algorithm --- incremental aggregation --- that can compute aggregate properties of the set of precursors exponentially faster than na{ï}ve approaches. The incremental aggregation algorithm is demonstrated on two problems from the two-dimensional binary Game of Life cellular automaton: precursor count distributions and higher-order mean field theory coefficients. In both cases, incremental aggregation allows us to obtain new results that were previously beyond reach.
△ Less
Submitted 13 November, 2017;
originally announced November 2017.
-
Potential role of a ventral nerve cord central pattern generator in forward and backward locomotion in Caenorhabditis elegans
Authors:
Erick Olivares,
Eduardo J. Izquierdo,
Randall D. Beer
Abstract:
Despite the relative simplicity of C. elegans, its locomotion machinery is not yet well understood. We focus on the generation of dorsoventral body bends. While central pattern generators are commonly involved in animal locomotion, their presence in C. elegans has been questioned due to a lack of an evident neural circuit to support it. We developed a computational model grounded in the available…
▽ More
Despite the relative simplicity of C. elegans, its locomotion machinery is not yet well understood. We focus on the generation of dorsoventral body bends. While central pattern generators are commonly involved in animal locomotion, their presence in C. elegans has been questioned due to a lack of an evident neural circuit to support it. We developed a computational model grounded in the available neuroanatomy and neurophysiology and we used an evolutionary algorithm to explore the space of possible configurations of the circuit that matched the neural traces observed during forward and backward locomotion in the worm. Our results demonstrate that it is possible for the rhythmic contraction to be produced by a circuit present in the ventral nerve cord.
△ Less
Submitted 6 November, 2017; v1 submitted 5 May, 2017;
originally announced May 2017.
-
Information flow through a model of the C. elegans klinotaxis circuit
Authors:
Eduardo J. Izquierdo,
Paul L. Williams,
Randall D. Beer
Abstract:
Understanding how information about external stimuli is transformed into behavior is one of the central goals of neuroscience. Here we characterize the information flow through a complete sensorimotor circuit: from stimulus, to sensory neurons, to interneurons, to motor neurons, to muscles, to motion. Specifically, we apply a recently developed framework for quantifying information flow to a previ…
▽ More
Understanding how information about external stimuli is transformed into behavior is one of the central goals of neuroscience. Here we characterize the information flow through a complete sensorimotor circuit: from stimulus, to sensory neurons, to interneurons, to motor neurons, to muscles, to motion. Specifically, we apply a recently developed framework for quantifying information flow to a previously published ensemble of models of salt klinotaxis in the nematode worm C. elegans. The models are grounded in the neuroanatomy and currently known neurophysiology of the worm. The unknown model parameters were optimized to reproduce the worm's behavior. Information flow analysis reveals several key principles underlying how the models operate: (1) Interneuron class AIY is responsible for integrating information about positive and negative changes in concentration, and exhibits a strong left/right information asymmetry. (2) Gap junctions play a crucial role in the transfer of information responsible for the information symmetry observed in interneuron class AIZ. (3) Neck motor neuron class SMB implements an information gating mechanism that underlies the circuit's state-dependent response. (4) The neck carries non-uniform distribution about changes in concentration. Thus, not all directions of movement are equally informative. Each of these findings corresponds to an experimental prediction that could be tested in the worm to greatly refine our understanding of the neural circuit underlying klinotaxis. Information flow analysis also allows us to explore how information flow relates to underlying electrophysiology. Despite large variations in the neural parameters of individual circuits, the overall information flow architecture circuit is remarkably consistent across the ensemble, suggesting that information flow analysis captures general principles of operation for the klinotaxis circuit.
△ Less
Submitted 14 February, 2015;
originally announced February 2015.
-
Maximal Ergodic Inequalities for Banach Function Spaces
Authors:
Richard de Beer,
Louis Labuschagne
Abstract:
We analyse the Transfer Principle, which is used to generate weak type maximal inequalities for ergodic operators, and extend it to the general case of $σ$-compact locally compact Hausdorff groups acting measure-preservingly on $σ$-finite measure spaces. We show how the techniques developed here generate various weak type maximal inequalities on different Banach function spaces, and how the proper…
▽ More
We analyse the Transfer Principle, which is used to generate weak type maximal inequalities for ergodic operators, and extend it to the general case of $σ$-compact locally compact Hausdorff groups acting measure-preservingly on $σ$-finite measure spaces. We show how the techniques developed here generate various weak type maximal inequalities on different Banach function spaces, and how the properties of these function spaces influence the weak type inequalities that can be obtained. Finally, we demonstrate how the techniques developed imply almost sure pointwise convergence of a wide class of ergodic averages.
△ Less
Submitted 7 April, 2015; v1 submitted 31 August, 2013;
originally announced September 2013.
-
Generalized Measures of Information Transfer
Authors:
Paul L. Williams,
Randall D. Beer
Abstract:
Transfer entropy provides a general tool for analyzing the magnitudes and directions---but not the \emph{kinds}---of information transfer in a system. We extend transfer entropy in two complementary ways. First, we distinguish state-dependent from state-independent transfer, based on whether a source's influence depends on the state of the target. Second, for multiple sources, we distinguish betwe…
▽ More
Transfer entropy provides a general tool for analyzing the magnitudes and directions---but not the \emph{kinds}---of information transfer in a system. We extend transfer entropy in two complementary ways. First, we distinguish state-dependent from state-independent transfer, based on whether a source's influence depends on the state of the target. Second, for multiple sources, we distinguish between unique, redundant, and synergistic transfer. The new measures are demonstrated on several systems that extend examples from previous literature.
△ Less
Submitted 7 February, 2011;
originally announced February 2011.
-
Saturation Probabilities of Continuous-Time Sigmoidal Networks
Authors:
Randall D. Beer,
Bryan Daniels
Abstract:
From genetic regulatory networks to nervous systems, the interactions between elements in biological networks often take a sigmoidal or S-shaped form. This paper develops a probabilistic characterization of the parameter space of continuous-time sigmoidal networks (CTSNs), a simple but dynamically-universal model of such interactions. We describe an efficient and accurate method for calculating th…
▽ More
From genetic regulatory networks to nervous systems, the interactions between elements in biological networks often take a sigmoidal or S-shaped form. This paper develops a probabilistic characterization of the parameter space of continuous-time sigmoidal networks (CTSNs), a simple but dynamically-universal model of such interactions. We describe an efficient and accurate method for calculating the probability of observing effectively M-dimensional dynamics in an N-element CTSN, as well as a closed-form but approximate method. We then study the dependence of this probability on N, M, and the parameter ranges over which sampling occurs. This analysis provides insight into the overall structure of CTSN parameter space.
△ Less
Submitted 8 October, 2010;
originally announced October 2010.
-
Nonnegative Decomposition of Multivariate Information
Authors:
Paul L. Williams,
Randall D. Beer
Abstract:
Of the various attempts to generalize information theory to multiple variables, the most widely utilized, interaction information, suffers from the problem that it is sometimes negative. Here we reconsider from first principles the general structure of the information that a set of sources provides about a given variable. We begin with a new definition of redundancy as the minimum information that…
▽ More
Of the various attempts to generalize information theory to multiple variables, the most widely utilized, interaction information, suffers from the problem that it is sometimes negative. Here we reconsider from first principles the general structure of the information that a set of sources provides about a given variable. We begin with a new definition of redundancy as the minimum information that any source provides about each possible outcome of the variable, averaged over all possible outcomes. We then show how this measure of redundancy induces a lattice over sets of sources that clarifies the general structure of multivariate information. Finally, we use this redundancy lattice to propose a definition of partial information atoms that exhaustively decompose the Shannon information in a multivariate system in terms of the redundancy between synergies of subsets of the sources. Unlike interaction information, the atoms of our partial information decomposition are never negative and always support a clear interpretation as informational quantities. Our analysis also demonstrates how the negativity of interaction information can be explained by its confounding of redundancy and synergy.
△ Less
Submitted 14 April, 2010;
originally announced April 2010.
-
May We Have Your Attention: Analysis of a Selective Attention Task
Authors:
Eldan Goldenberg,
Jacob R. Garcowski,
Randall D. Beer
Abstract:
In this paper we present a deeper analysis than has previously been carried out of a selective attention problem, and the evolution of continuous-time recurrent neural networks to solve it. We show that the task has a rich structure, and agents must solve a variety of subproblems to perform well. We consider the relationship between the complexity of an agent and the ease with which it can evolv…
▽ More
In this paper we present a deeper analysis than has previously been carried out of a selective attention problem, and the evolution of continuous-time recurrent neural networks to solve it. We show that the task has a rich structure, and agents must solve a variety of subproblems to perform well. We consider the relationship between the complexity of an agent and the ease with which it can evolve behavior that generalizes well across subproblems, and demonstrate a shaping protocol that improves generalization.
△ Less
Submitted 29 June, 2006;
originally announced June 2006.
-
Noncommutative recurrence over locally compact Hausdorff groups
Authors:
Richard de Beer,
Rocco Duvenhage,
Anton Stroh
Abstract:
We extend previous results on noncommutative recurrence in unital *-algebras over the integers, to the case where one works over locally compact Hausdorff groups. We derive a generalization of Khintchine's recurrence theorem, as well as a form of multiple recurrence. This is done using the mean ergodic theorem in Hilbert space, via the GNS construction.
We extend previous results on noncommutative recurrence in unital *-algebras over the integers, to the case where one works over locally compact Hausdorff groups. We derive a generalization of Khintchine's recurrence theorem, as well as a form of multiple recurrence. This is done using the mean ergodic theorem in Hilbert space, via the GNS construction.
△ Less
Submitted 16 September, 2005;
originally announced September 2005.
