-
Comment on "Machine learning conservation laws from differential equations"
Authors:
Michael F. Zimmer
Abstract:
In lieu of abstract, first paragraph reads: Six months after the author derived a constant of motion for a 1D damped harmonic oscillator [1], a similar result appeared by Liu, Madhavan, and Tegmark [2, 3], without citing the author. However, their derivation contained six serious errors, causing both their method and result to be incorrect. In this Comment, those errors are reviewed.
In lieu of abstract, first paragraph reads: Six months after the author derived a constant of motion for a 1D damped harmonic oscillator [1], a similar result appeared by Liu, Madhavan, and Tegmark [2, 3], without citing the author. However, their derivation contained six serious errors, causing both their method and result to be incorrect. In this Comment, those errors are reviewed.
△ Less
Submitted 3 April, 2024;
originally announced April 2024.
-
Constants of Motion for Conserved and Non-conserved Dynamics
Authors:
Michael F. Zimmer
Abstract:
This paper begins with a dynamical model that was obtained by applying a machine learning technique (FJet) to time-series data; this dynamical model is then analyzed with Lie symmetry techniques to obtain constants of motion. This analysis is performed on both the conserved and non-conserved cases of the 1D and 2D harmonic oscillators. For the 1D oscillator, constants are found in the cases where…
▽ More
This paper begins with a dynamical model that was obtained by applying a machine learning technique (FJet) to time-series data; this dynamical model is then analyzed with Lie symmetry techniques to obtain constants of motion. This analysis is performed on both the conserved and non-conserved cases of the 1D and 2D harmonic oscillators. For the 1D oscillator, constants are found in the cases where the system is underdamped, overdamped, and critically damped. The novel existence of such a constant for a non-conserved model is interpreted as a manifestation of the conservation of energy of the {\em total} system (i.e., oscillator plus dissipative environment). For the 2D oscillator, constants are found for the isotropic and anisotropic cases, including when the frequencies are incommensurate; it is also generalized to arbitrary dimensions. In addition, a constant is identified which generalizes angular momentum for all ratios of the frequencies. The approach presented here can produce {\em multiple} constants of motion from a {\em single}, generic data set.
△ Less
Submitted 28 March, 2024;
originally announced March 2024.
-
Memory Gym: Towards Endless Tasks to Benchmark Memory Capabilities of Agents
Authors:
Marco Pleines,
Matthias Pallasch,
Frank Zimmer,
Mike Preuss
Abstract:
Memory Gym presents a suite of 2D partially observable environments, namely Mortar Mayhem, Mystery Path, and Searing Spotlights, designed to benchmark memory capabilities in decision-making agents. These environments, originally with finite tasks, are expanded into innovative, endless formats, mirroring the escalating challenges of cumulative memory games such as ``I packed my bag''. This progress…
▽ More
Memory Gym presents a suite of 2D partially observable environments, namely Mortar Mayhem, Mystery Path, and Searing Spotlights, designed to benchmark memory capabilities in decision-making agents. These environments, originally with finite tasks, are expanded into innovative, endless formats, mirroring the escalating challenges of cumulative memory games such as ``I packed my bag''. This progression in task design shifts the focus from merely assessing sample efficiency to also probing the levels of memory effectiveness in dynamic, prolonged scenarios. To address the gap in available memory-based Deep Reinforcement Learning baselines, we introduce an implementation that integrates Transformer-XL (TrXL) with Proximal Policy Optimization. This approach utilizes TrXL as a form of episodic memory, employing a sliding window technique. Our comparative study between the Gated Recurrent Unit (GRU) and TrXL reveals varied performances across different settings. TrXL, on the finite environments, demonstrates superior sample efficiency in Mystery Path and outperforms in Mortar Mayhem. However, GRU is more efficient on Searing Spotlights. Most notably, in all endless tasks, GRU makes a remarkable resurgence, consistently outperforming TrXL by significant margins. Website and Source Code: https://github.com/MarcoMeter/endless-memory-gym/
△ Less
Submitted 11 June, 2024; v1 submitted 29 September, 2023;
originally announced September 2023.
-
Sample Size in Natural Language Processing within Healthcare Research
Authors:
Jaya Chaturvedi,
Diana Shamsutdinova,
Felix Zimmer,
Sumithra Velupillai,
Daniel Stahl,
Robert Stewart,
Angus Roberts
Abstract:
Sample size calculation is an essential step in most data-based disciplines. Large enough samples ensure representativeness of the population and determine the precision of estimates. This is true for most quantitative studies, including those that employ machine learning methods, such as natural language processing, where free-text is used to generate predictions and classify instances of text. W…
▽ More
Sample size calculation is an essential step in most data-based disciplines. Large enough samples ensure representativeness of the population and determine the precision of estimates. This is true for most quantitative studies, including those that employ machine learning methods, such as natural language processing, where free-text is used to generate predictions and classify instances of text. Within the healthcare domain, the lack of sufficient corpora of previously collected data can be a limiting factor when determining sample sizes for new studies. This paper tries to address the issue by making recommendations on sample sizes for text classification tasks in the healthcare domain.
Models trained on the MIMIC-III database of critical care records from Beth Israel Deaconess Medical Center were used to classify documents as having or not having Unspecified Essential Hypertension, the most common diagnosis code in the database. Simulations were performed using various classifiers on different sample sizes and class proportions. This was repeated for a comparatively less common diagnosis code within the database of diabetes mellitus without mention of complication.
Smaller sample sizes resulted in better results when using a K-nearest neighbours classifier, whereas larger sample sizes provided better results with support vector machines and BERT models. Overall, a sample size larger than 1000 was sufficient to provide decent performance metrics.
The simulations conducted within this study provide guidelines that can be used as recommendations for selecting appropriate sample sizes and class proportions, and for predicting expected performance, when building classifiers for textual healthcare data. The methodology used here can be modified for sample size estimates calculations with other datasets.
△ Less
Submitted 5 September, 2023;
originally announced September 2023.
-
More Than a Wife and a Mom: A Study of Mom Vlogging Practices in China
Authors:
Kyrie Zhixuan Zhou,
Bohui Shen,
Franziska Zimmer,
Chuanli Xia,
Xin Tong
Abstract:
Mom vloggers are stay-at-home moms who record and share their daily life through short videos. In this exploratory study, we aspire to understand mom vloggers' motivations, practices, and challenges. Our mixed-methods inspection contained interviews with 4 mom vloggers in China and a content analysis of mom vlogs of 5 other mom vloggers. Mom vloggers' primary motivations are to make money, record…
▽ More
Mom vloggers are stay-at-home moms who record and share their daily life through short videos. In this exploratory study, we aspire to understand mom vloggers' motivations, practices, and challenges. Our mixed-methods inspection contained interviews with 4 mom vloggers in China and a content analysis of mom vlogs of 5 other mom vloggers. Mom vloggers' primary motivations are to make money, record daily life, and seek their individual identities and values, well meeting their financial and social needs after leaving their paid employment. When creating vlog content, mom bloggers encounter various challenges, such as a lack of video visibility, being stretched by both intensive motherhood and heavy digital work, privacy and self-presentation concerns, and so on. Based on the findings, we propose design implications toward resolving these challenges and benefiting mom vloggers' experiences.
△ Less
Submitted 27 September, 2023; v1 submitted 23 June, 2022;
originally announced June 2022.
-
Anonymous Expression in an Online Community for Women in China
Authors:
Zhixuan Zhou,
Zixin Wang,
Franziska Zimmer
Abstract:
Gender issues faced by women can range from workplace harassment to domestic violence. While publicly disclosing these issues on social media can be hard, some may incline to express themselves anonymously. We approached such an anonymous female community on Chinese social media where discussion on gender issues takes place with a qualitative content analysis. By observing anonymous experiences co…
▽ More
Gender issues faced by women can range from workplace harassment to domestic violence. While publicly disclosing these issues on social media can be hard, some may incline to express themselves anonymously. We approached such an anonymous female community on Chinese social media where discussion on gender issues takes place with a qualitative content analysis. By observing anonymous experiences contributed by female users and made publicly available by an influencer, we identified 20 issues commonly discussed, with cheating-partner, controlling parents and age anxiety taking the lead. The results are placed into context with Chinese culture and expectations about gender. By describing the results in context with the social challenges faced by women in China, and understanding how these issues are anonymously and openly discussed by them, we aim to motivate more policies and platform designs to accommodate the needs of the affected population.
△ Less
Submitted 18 August, 2022; v1 submitted 16 June, 2022;
originally announced June 2022.
-
Generalization, Mayhems and Limits in Recurrent Proximal Policy Optimization
Authors:
Marco Pleines,
Matthias Pallasch,
Frank Zimmer,
Mike Preuss
Abstract:
At first sight it may seem straightforward to use recurrent layers in Deep Reinforcement Learning algorithms to enable agents to make use of memory in the setting of partially observable environments. Starting from widely used Proximal Policy Optimization (PPO), we highlight vital details that one must get right when adding recurrence to achieve a correct and efficient implementation, namely: prop…
▽ More
At first sight it may seem straightforward to use recurrent layers in Deep Reinforcement Learning algorithms to enable agents to make use of memory in the setting of partially observable environments. Starting from widely used Proximal Policy Optimization (PPO), we highlight vital details that one must get right when adding recurrence to achieve a correct and efficient implementation, namely: properly shaping the neural net's forward pass, arranging the training data, correspondingly selecting hidden states for sequence beginnings and masking paddings for loss computation. We further explore the limitations of recurrent PPO by benchmarking the contributed novel environments Mortar Mayhem and Searing Spotlights that challenge the agent's memory beyond solely capacity and distraction tasks. Remarkably, we can demonstrate a transition to strong generalization in Mortar Mayhem when scaling the number of training seeds, while the agent does not succeed on Searing Spotlights, which seems to be a tough challenge for memory-based agents.
△ Less
Submitted 23 May, 2022;
originally announced May 2022.
-
On the Verge of Solving Rocket League using Deep Reinforcement Learning and Sim-to-sim Transfer
Authors:
Marco Pleines,
Konstantin Ramthun,
Yannik Wegener,
Hendrik Meyer,
Matthias Pallasch,
Sebastian Prior,
Jannik Drögemüller,
Leon Büttinghaus,
Thilo Röthemeyer,
Alexander Kaschwig,
Oliver Chmurzynski,
Frederik Rohkrähmer,
Roman Kalkreuth,
Frank Zimmer,
Mike Preuss
Abstract:
Autonomously trained agents that are supposed to play video games reasonably well rely either on fast simulation speeds or heavy parallelization across thousands of machines running concurrently. This work explores a third way that is established in robotics, namely sim-to-real transfer, or if the game is considered a simulation itself, sim-to-sim transfer. In the case of Rocket League, we demonst…
▽ More
Autonomously trained agents that are supposed to play video games reasonably well rely either on fast simulation speeds or heavy parallelization across thousands of machines running concurrently. This work explores a third way that is established in robotics, namely sim-to-real transfer, or if the game is considered a simulation itself, sim-to-sim transfer. In the case of Rocket League, we demonstrate that single behaviors of goalies and strikers can be successfully learned using Deep Reinforcement Learning in the simulation environment and transferred back to the original game. Although the implemented training simulation is to some extent inaccurate, the goalkeeping agent saves nearly 100% of its faced shots once transferred, while the striking agent scores in about 75% of cases. Therefore, the trained agent is robust enough and able to generalize to the target domain of Rocket League.
△ Less
Submitted 24 May, 2022; v1 submitted 10 May, 2022;
originally announced May 2022.
-
Extracting Dynamical Models from Data
Authors:
Michael F. Zimmer
Abstract:
The problem of determining the underlying dynamics of a system when only given data of its state over time has challenged scientists for decades. In this paper, the approach of using machine learning to model the updates of the phase space variables is introduced; this is done as a function of the phase space variables. (More generally, the modeling is done over functions of the jet space.) This a…
▽ More
The problem of determining the underlying dynamics of a system when only given data of its state over time has challenged scientists for decades. In this paper, the approach of using machine learning to model the updates of the phase space variables is introduced; this is done as a function of the phase space variables. (More generally, the modeling is done over functions of the jet space.) This approach (named FJet) allows one to accurately replicate the dynamics, and is demonstrated on the examples of the damped harmonic oscillator, the damped pendulum, and the Duffing oscillator; the underlying differential equation is also accurately recovered for each example. In addition, the results in no way depend on how the data is sampled over time (i.e., regularly or irregularly). It is demonstrated that a regression implementation of FJet is similar to the model resulting from a Taylor series expansion of the Runge-Kutta (RK) numerical integration scheme. This identification confers the advantage of explicitly revealing the function space to use in the modeling, as well as the associated uncertainty quantification for the updates. Finally, it is shown in the undamped harmonic oscillator example that the stability of the updates is stable $10^9$ times longer than with $4$th-order RK (with time step $0.1$).
△ Less
Submitted 31 January, 2024; v1 submitted 13 October, 2021;
originally announced October 2021.
-
2nd-order Updates with 1st-order Complexity
Authors:
Michael F. Zimmer
Abstract:
It has long been a goal to efficiently compute and use second order information on a function ($f$) to assist in numerical approximations. Here it is shown how, using only basic physics and a numerical approximation, such information can be accurately obtained at a cost of ${\cal O}(N)$ complexity, where $N$ is the dimensionality of the parameter space of $f$. In this paper, an algorithm ({\em VA-…
▽ More
It has long been a goal to efficiently compute and use second order information on a function ($f$) to assist in numerical approximations. Here it is shown how, using only basic physics and a numerical approximation, such information can be accurately obtained at a cost of ${\cal O}(N)$ complexity, where $N$ is the dimensionality of the parameter space of $f$. In this paper, an algorithm ({\em VA-Flow}) is developed to exploit this second order information, and pseudocode is presented. It is applied to two classes of problems, that of inverse kinematics (IK) and gradient descent (GD). In the IK application, the algorithm is fast and robust, and is shown to lead to smooth behavior even near singularities. For GD the algorithm also works very well, provided the cost function is locally well-described by a polynomial.
△ Less
Submitted 27 May, 2021; v1 submitted 24 May, 2021;
originally announced May 2021.
-
Neograd: Near-Ideal Gradient Descent
Authors:
Michael F. Zimmer
Abstract:
The purpose of this paper is to improve upon existing variants of gradient descent by solving two problems: (1) removing (or reducing) the plateau that occurs while minimizing the cost function, (2) continually adjusting the learning rate to an "ideal" value. The approach taken is to approximately solve for the learning rate as a function of a trust metric. When this technique is hybridized with m…
▽ More
The purpose of this paper is to improve upon existing variants of gradient descent by solving two problems: (1) removing (or reducing) the plateau that occurs while minimizing the cost function, (2) continually adjusting the learning rate to an "ideal" value. The approach taken is to approximately solve for the learning rate as a function of a trust metric. When this technique is hybridized with momentum, it creates an especially effective gradient descent variant, called NeogradM. It is shown to outperform Adam on several test problems, and can easily reach cost function values that are smaller by a factor of $10^8$, for example.
△ Less
Submitted 2 August, 2021; v1 submitted 15 October, 2020;
originally announced October 2020.
-
Obstacle Tower Without Human Demonstrations: How Far a Deep Feed-Forward Network Goes with Reinforcement Learning
Authors:
Marco Pleines,
Jenia Jitsev,
Mike Preuss,
Frank Zimmer
Abstract:
The Obstacle Tower Challenge is the task to master a procedurally generated chain of levels that subsequently get harder to complete. Whereas the most top performing entries of last year's competition used human demonstrations or reward shaping to learn how to cope with the challenge, we present an approach that performed competitively (placed 7th) but starts completely from scratch by means of De…
▽ More
The Obstacle Tower Challenge is the task to master a procedurally generated chain of levels that subsequently get harder to complete. Whereas the most top performing entries of last year's competition used human demonstrations or reward shaping to learn how to cope with the challenge, we present an approach that performed competitively (placed 7th) but starts completely from scratch by means of Deep Reinforcement Learning with a relatively simple feed-forward deep network structure. We especially look at the generalization performance of the taken approach concerning different seeds and various visual themes that have become available after the competition, and investigate where the agent fails and why. Note that our approach does not possess a short-term memory like employing recurrent hidden states. With this work, we hope to contribute to a better understanding of what is possible with a relatively simple, flexible solution that can be applied to learning in environments featuring complex 3D visual input where the abstract task structure itself is still fairly simple.
△ Less
Submitted 20 July, 2020; v1 submitted 1 April, 2020;
originally announced April 2020.
-
Dynamic Adaptation of Software-defined Networks for IoT Systems: A Search-based Approach
Authors:
Seung Yeob Shin,
Shiva Nejati,
Mehrdad Sabetzadeh,
Lionel C. Briand,
Chetan Arora,
Frank Zimmer
Abstract:
The concept of Internet of Things (IoT) has led to the development of many complex and critical systems such as smart emergency management systems. IoT-enabled applications typically depend on a communication network for transmitting large volumes of data in unpredictable and changing environments. These networks are prone to congestion when there is a burst in demand, e.g., as an emergency situat…
▽ More
The concept of Internet of Things (IoT) has led to the development of many complex and critical systems such as smart emergency management systems. IoT-enabled applications typically depend on a communication network for transmitting large volumes of data in unpredictable and changing environments. These networks are prone to congestion when there is a burst in demand, e.g., as an emergency situation is unfolding, and therefore rely on configurable software-defined networks (SDN). In this paper, we propose a dynamic adaptive SDN configuration approach for IoT systems. The approach enables resolving congestion in real time while minimizing network utilization, data transmission delays and adaptation costs. Our approach builds on existing work in dynamic adaptive search-based software engineering (SBSE) to reconfigure an SDN while simultaneously ensuring multiple quality of service criteria. We evaluate our approach on an industrial national emergency management system, which is aimed at detecting disasters and emergencies, and facilitating recovery and rescue operations by providing first responders with a reliable communication infrastructure. Our results indicate that (1) our approach is able to efficiently and effectively adapt an SDN to dynamically resolve congestion, and (2) compared to two baseline data forwarding algorithms that are static and non-adaptive, our approach increases data transmission rate by a factor of at least 3 and decreases data loss by at least 70%.
△ Less
Submitted 15 May, 2020; v1 submitted 29 May, 2019;
originally announced May 2019.
-
Speedup from a different parametrization within the Neural Network algorithm
Authors:
Michael F. Zimmer
Abstract:
A different parametrization of the hyperplanes is used in the neural network algorithm. As demonstrated on several autoencoder examples it significantly outperforms the usual parametrization, reaching lower training error values with only a fraction of the number of epochs. It's argued that it makes it easier to understand and initialize the parameters.
A different parametrization of the hyperplanes is used in the neural network algorithm. As demonstrated on several autoencoder examples it significantly outperforms the usual parametrization, reaching lower training error values with only a fraction of the number of epochs. It's argued that it makes it easier to understand and initialize the parameters.
△ Less
Submitted 2 June, 2017; v1 submitted 19 May, 2017;
originally announced May 2017.
