-
DST-GTN: Dynamic Spatio-Temporal Graph Transformer Network for Traffic Forecasting
Authors:
Songtao Huang,
Hongjin Song,
Tianqi Jiang,
Akbar Telikani,
Jun Shen,
Qingguo Zhou,
Binbin Yong,
Qiang Wu
Abstract:
Accurate traffic forecasting is essential for effective urban planning and congestion management. Deep learning (DL) approaches have gained colossal success in traffic forecasting but still face challenges in capturing the intricacies of traffic dynamics. In this paper, we identify and address this challenges by emphasizing that spatial features are inherently dynamic and change over time. A novel…
▽ More
Accurate traffic forecasting is essential for effective urban planning and congestion management. Deep learning (DL) approaches have gained colossal success in traffic forecasting but still face challenges in capturing the intricacies of traffic dynamics. In this paper, we identify and address this challenges by emphasizing that spatial features are inherently dynamic and change over time. A novel in-depth feature representation, called Dynamic Spatio-Temporal (Dyn-ST) features, is introduced, which encapsulates spatial characteristics across varying times. Moreover, a Dynamic Spatio-Temporal Graph Transformer Network (DST-GTN) is proposed by capturing Dyn-ST features and other dynamic adjacency relations between intersections. The DST-GTN can model dynamic ST relationships between nodes accurately and refine the representation of global and local ST characteristics by adopting adaptive weights in low-pass and all-pass filters, enabling the extraction of Dyn-ST features from traffic time-series data. Through numerical experiments on public datasets, the DST-GTN achieves state-of-the-art performance for a range of traffic forecasting tasks and demonstrates enhanced stability.
△ Less
Submitted 18 April, 2024;
originally announced April 2024.
-
Dismai-Bench: Benchmarking and designing generative models using disordered materials and interfaces
Authors:
Adrian Xiao Bin Yong,
Tianyu Su,
Elif Ertekin
Abstract:
Generative models have received significant attention in recent years for materials science applications, particularly in the area of inverse design for materials discovery. However, these models are usually assessed based on newly generated, unverified materials, which provide a narrow evaluation of a model's performance. Also, current efforts for inorganic materials have predominantly focused on…
▽ More
Generative models have received significant attention in recent years for materials science applications, particularly in the area of inverse design for materials discovery. However, these models are usually assessed based on newly generated, unverified materials, which provide a narrow evaluation of a model's performance. Also, current efforts for inorganic materials have predominantly focused on small crystals, even though the capability to generate large disordered structures would significantly expand the applicability of generative modeling. In this work, we present the Disordered Materials & Interfaces Benchmark (Dismai-Bench), a generative model benchmark that uses datasets of disordered alloys, interfaces, and amorphous silicon (256-264 atoms per structure). Models are trained on each dataset independently, and evaluated through direct structural comparisons between training and generated structures. Benchmarking was performed on two graph diffusion models and two (coordinate-based) U-Net diffusion models. The graph models were found to significantly outperform the U-Net models due to the higher expressive power of graphs. While noise in the less expressive models can assist in discovering materials by facilitating exploration beyond the training distribution, these models face significant challenges when confronted with more complex structures. To further demonstrate the benefits of this benchmarking in the development process of a generative model, we considered the case of developing a point-cloud-based generative adversarial network (GAN) to generate low-energy disordered interfaces. We show that the best performing architecture, CryinGAN, outperforms the U-Net models, and is competitive against the graph models despite its lack of invariances and weaker expressive power. This work provides a new framework and insights to guide the development of future generative models.
△ Less
Submitted 13 July, 2024; v1 submitted 10 April, 2024;
originally announced April 2024.
-
Combating New COVID-19 Variants in Indonesia: Is Accelerating Four-Dose Vaccinations Sufficient?
Authors:
Benny Yong,
Jonathan Hoseana,
Livia Owen
Abstract:
As new COVID-19 variants continue to emerge globally, we develop an SISI-type mathematical model to determine whether the transmission risks arising when such a new variant enters Indonesia can be mitigated by solely accelerating the country's ongoing four-dose vaccination programme. We begin by determining the model's basic reproduction number, as well as the model's equilibria and their stabilit…
▽ More
As new COVID-19 variants continue to emerge globally, we develop an SISI-type mathematical model to determine whether the transmission risks arising when such a new variant enters Indonesia can be mitigated by solely accelerating the country's ongoing four-dose vaccination programme. We begin by determining the model's basic reproduction number, as well as the model's equilibria and their stability. Subsequently, employing parameter values representing the country's situation as of March 20, 2023, we conduct a numerical sensitivity analysis in two simulated cases corresponding to two different levels of the new variant's transmission. The results show that, a satisfactory mitigation relying solely on vaccinations necessitates a drastic acceleration in the low-transmission case, and proves unachievable in the high-transmission case. Accordingly, we recommend that the acceleration of the ongoing four-dose vaccinations be carried out in conjunction with other intervention measures, such as improvements of the vaccine's efficacy and the disease's recovery rate.
△ Less
Submitted 31 August, 2023;
originally announced September 2023.
-
Codimension-Two Bifurcations of an SIR-Type Model for COVID-19 and Their Epidemiological Implications
Authors:
Livia Owen,
Jonathan Hoseana,
Benny Yong
Abstract:
We study the codimension-two bifurcations exhibited by a recently-developed SIR-type mathematical model for the spread of COVID-19, as its two main parameters -- the susceptible individuals' cautiousness level and the hospitals' bed-occupancy rate -- vary over their domains. We use AUTO to generate the model's bifurcation diagrams near the relevant bifurcation points: two Bogdanov-Takens points an…
▽ More
We study the codimension-two bifurcations exhibited by a recently-developed SIR-type mathematical model for the spread of COVID-19, as its two main parameters -- the susceptible individuals' cautiousness level and the hospitals' bed-occupancy rate -- vary over their domains. We use AUTO to generate the model's bifurcation diagrams near the relevant bifurcation points: two Bogdanov-Takens points and two generalised Hopf points, as well as a number of phase portraits describing the model's orbital behaviours for various pairs of parameter values near each bifurcation point. The analysis shows that, when a backward bifurcation occurs at the basic reproduction threshold, the transition of the model's asymptotic behaviour from endemic to disease-free takes place via an unexpectedly complex sequence of topological changes, involving the births and disappearances of not only equilibria but also limit cycles and homoclinic orbits. Epidemiologically, the analysis confirms the importance of a proper control of the values of the aforementioned parameters for a successful eradication of COVID-19. We recommend a number of strategies by which such a control may be achieved.
△ Less
Submitted 17 July, 2023;
originally announced July 2023.
-
From Pandemic to a New Normal: Strategies to Optimise Governmental Interventions in Indonesia Based on an SVEIQHR-Type Mathematical Model
Authors:
Benny Yong,
Jonathan Hoseana,
Livia Owen
Abstract:
There are five different forms of intervention presently realised by the Indonesian government in an effort to end the COVID-19 pandemic: vaccinations, social restrictions, tracings, testings, and treatments. In this paper, we construct an SVEIQHR-type mathematical model for the disease's spread in the country, which incorporates as parameters the rates of the above interventions, as well as the v…
▽ More
There are five different forms of intervention presently realised by the Indonesian government in an effort to end the COVID-19 pandemic: vaccinations, social restrictions, tracings, testings, and treatments. In this paper, we construct an SVEIQHR-type mathematical model for the disease's spread in the country, which incorporates as parameters the rates of the above interventions, as well as the vaccine's efficacy. We determine the model's equilibria and basic reproduction number. Using the model, we formulate strategies by which the interventions should be realised in order to optimise their impact. The results show that, in a disease-free state, when the number of new cases rises, the best strategy is to implement social restrictions, whereas in an endemic state, if a near-lockdown policy is undesirable, carrying out vaccinations is the best strategy; however, efforts should be aimed not primarily towards increasing the vaccination rate, but towards the use of high-efficacy vaccines.
△ Less
Submitted 30 March, 2022;
originally announced March 2022.
-
Bayesian autoencoders with uncertainty quantification: Towards trustworthy anomaly detection
Authors:
Bang Xiang Yong,
Alexandra Brintrup
Abstract:
Despite numerous studies of deep autoencoders (AEs) for unsupervised anomaly detection, AEs still lack a way to express uncertainty in their predictions, crucial for ensuring safe and trustworthy machine learning systems in high-stake applications. Therefore, in this work, the formulation of Bayesian autoencoders (BAEs) is adopted to quantify the total anomaly uncertainty, comprising epistemic and…
▽ More
Despite numerous studies of deep autoencoders (AEs) for unsupervised anomaly detection, AEs still lack a way to express uncertainty in their predictions, crucial for ensuring safe and trustworthy machine learning systems in high-stake applications. Therefore, in this work, the formulation of Bayesian autoencoders (BAEs) is adopted to quantify the total anomaly uncertainty, comprising epistemic and aleatoric uncertainties. To evaluate the quality of uncertainty, we consider the task of classifying anomalies with the additional option of rejecting predictions of high uncertainty. In addition, we use the accuracy-rejection curve and propose the weighted average accuracy as a performance metric. Our experiments demonstrate the effectiveness of the BAE and total anomaly uncertainty on a set of benchmark datasets and two real datasets for manufacturing: one for condition monitoring, the other for quality inspection.
△ Less
Submitted 25 February, 2022;
originally announced February 2022.
-
Do autoencoders need a bottleneck for anomaly detection?
Authors:
Bang Xiang Yong,
Alexandra Brintrup
Abstract:
A common belief in designing deep autoencoders (AEs), a type of unsupervised neural network, is that a bottleneck is required to prevent learning the identity function. Learning the identity function renders the AEs useless for anomaly detection. In this work, we challenge this limiting belief and investigate the value of non-bottlenecked AEs.
The bottleneck can be removed in two ways: (1) overp…
▽ More
A common belief in designing deep autoencoders (AEs), a type of unsupervised neural network, is that a bottleneck is required to prevent learning the identity function. Learning the identity function renders the AEs useless for anomaly detection. In this work, we challenge this limiting belief and investigate the value of non-bottlenecked AEs.
The bottleneck can be removed in two ways: (1) overparameterising the latent layer, and (2) introducing skip connections. However, limited works have reported on the use of one of the ways. For the first time, we carry out extensive experiments covering various combinations of bottleneck removal schemes, types of AEs and datasets. In addition, we propose the infinitely-wide AEs as an extreme example of non-bottlenecked AEs.
Their improvement over the baseline implies learning the identity function is not trivial as previously assumed. Moreover, we find that non-bottlenecked architectures (highest AUROC=0.857) can outperform their bottlenecked counterparts (highest AUROC=0.696) on the popular task of CIFAR (inliers) vs SVHN (anomalies), among other tasks, shedding light on the potential of developing non-bottlenecked AEs for improving anomaly detection.
△ Less
Submitted 25 February, 2022;
originally announced February 2022.
-
A Design of Governmental Policies for the Eradication of COVID-19 in Jakarta Using an SIR-Type Mathematical Model
Authors:
Benny Yong,
Jonathan Hoseana,
Livia Owen
Abstract:
Using a discretised version of our recently-developed SIR-type mathematical model for the spread of COVID-19, we construct a design of governmental policies for the eradication of the disease in the province of DKI Jakarta, Indonesia, taking as a basis the actual data of mid-2021. The design takes the form of a precise, quantitative method to determine the appropriate level(s) of restrictions on c…
▽ More
Using a discretised version of our recently-developed SIR-type mathematical model for the spread of COVID-19, we construct a design of governmental policies for the eradication of the disease in the province of DKI Jakarta, Indonesia, taking as a basis the actual data of mid-2021. The design takes the form of a precise, quantitative method to determine the appropriate level(s) of restrictions on community activities (PPKM) which should be enforced in the province on any given day, based on the current values of the disease's effective reproduction number and the hospitals' bed-occupancy rate.
△ Less
Submitted 24 October, 2021;
originally announced October 2021.
-
Coalitional Bayesian Autoencoders -- Towards explainable unsupervised deep learning
Authors:
Bang Xiang Yong,
Alexandra Brintrup
Abstract:
This paper aims to improve the explainability of Autoencoder's (AE) predictions by proposing two explanation methods based on the mean and epistemic uncertainty of log-likelihood estimate, which naturally arise from the probabilistic formulation of the AE called Bayesian Autoencoders (BAE). To quantitatively evaluate the performance of explanation methods, we test them in sensor network applicatio…
▽ More
This paper aims to improve the explainability of Autoencoder's (AE) predictions by proposing two explanation methods based on the mean and epistemic uncertainty of log-likelihood estimate, which naturally arise from the probabilistic formulation of the AE called Bayesian Autoencoders (BAE). To quantitatively evaluate the performance of explanation methods, we test them in sensor network applications, and propose three metrics based on covariate shift of sensors : (1) G-mean of Spearman drift coefficients, (2) G-mean of sensitivity-specificity of explanation ranking and (3) sensor explanation quality index (SEQI) which combines the two aforementioned metrics. Surprisingly, we find that explanations of BAE's predictions suffer from high correlation resulting in misleading explanations. To alleviate this, a "Coalitional BAE" is proposed, which is inspired by agent-based system theory. Our comprehensive experiments on publicly available condition monitoring datasets demonstrate the improved quality of explanations using the Coalitional BAE.
△ Less
Submitted 19 October, 2021;
originally announced October 2021.
-
Mathematical Analysis of an Epidemic Model for COVID-19: How Important Is the People's Cautiousness Level for Eradication?
Authors:
Benny Yong,
Livia Owen,
Jonathan Hoseana
Abstract:
We construct an SIR-type model for COVID-19, incorporating as a parameter the susceptible individuals' cautiousness level. We determine the model's basic reproduction number, study the stability of the equilibria analytically, and perform a sensitivity analysis to confirm the significance of the cautiousness level. Fixing specific values for all other parameters, we study numerically the model's d…
▽ More
We construct an SIR-type model for COVID-19, incorporating as a parameter the susceptible individuals' cautiousness level. We determine the model's basic reproduction number, study the stability of the equilibria analytically, and perform a sensitivity analysis to confirm the significance of the cautiousness level. Fixing specific values for all other parameters, we study numerically the model's dynamics as the cautiousness level varies, revealing backward transcritical, Hopf, and saddle-node bifurcations of equilibria, as well as homoclinic and fold bifurcations of limit cycles with the aid of AUTO. Considering some key events affecting the pandemic in Indonesia, we design a scenario in which the cautiousness level varies over time, and show that the model exhibits a hysteresis, whereby, a slight cautiousness decrease could bring a disease-free state to endemic, and this is reversible only by a drastic cautiousness increase, thereby mathematically justifying the importance of a high cautiousness level for resolving the pandemic.
△ Less
Submitted 4 April, 2022; v1 submitted 5 August, 2021;
originally announced August 2021.
-
Bayesian Autoencoders: Analysing and Fixing the Bernoulli likelihood for Out-of-Distribution Detection
Authors:
Bang Xiang Yong,
Tim Pearce,
Alexandra Brintrup
Abstract:
After an autoencoder (AE) has learnt to reconstruct one dataset, it might be expected that the likelihood on an out-of-distribution (OOD) input would be low. This has been studied as an approach to detect OOD inputs. Recent work showed this intuitive approach can fail for the dataset pairs FashionMNIST vs MNIST. This paper suggests this is due to the use of Bernoulli likelihood and analyses why th…
▽ More
After an autoencoder (AE) has learnt to reconstruct one dataset, it might be expected that the likelihood on an out-of-distribution (OOD) input would be low. This has been studied as an approach to detect OOD inputs. Recent work showed this intuitive approach can fail for the dataset pairs FashionMNIST vs MNIST. This paper suggests this is due to the use of Bernoulli likelihood and analyses why this is the case, proposing two fixes: 1) Compute the uncertainty of likelihood estimate by using a Bayesian version of the AE. 2) Use alternative distributions to model the likelihood.
△ Less
Submitted 28 July, 2021;
originally announced July 2021.
-
Multi Agent System for Machine Learning Under Uncertainty in Cyber Physical Manufacturing System
Authors:
Bang Xiang Yong,
Alexandra Brintrup
Abstract:
Recent advancements in predictive machine learning has led to its application in various use cases in manufacturing. Most research focused on maximising predictive accuracy without addressing the uncertainty associated with it. While accuracy is important, focusing primarily on it poses an overfitting danger, exposing manufacturers to risk, ultimately hindering the adoption of these techniques. In…
▽ More
Recent advancements in predictive machine learning has led to its application in various use cases in manufacturing. Most research focused on maximising predictive accuracy without addressing the uncertainty associated with it. While accuracy is important, focusing primarily on it poses an overfitting danger, exposing manufacturers to risk, ultimately hindering the adoption of these techniques. In this paper, we determine the sources of uncertainty in machine learning and establish the success criteria of a machine learning system to function well under uncertainty in a cyber-physical manufacturing system (CPMS) scenario. Then, we propose a multi-agent system architecture which leverages probabilistic machine learning as a means of achieving such criteria. We propose possible scenarios for which our proposed architecture is useful and discuss future work. Experimentally, we implement Bayesian Neural Networks for multi-tasks classification on a public dataset for the real-time condition monitoring of a hydraulic system and demonstrate the usefulness of the system by evaluating the probability of a prediction being accurate given its uncertainty. We deploy these models using our proposed agent-based framework and integrate web visualisation to demonstrate its real-time feasibility.
△ Less
Submitted 28 July, 2021;
originally announced July 2021.
-
Bayesian Autoencoders for Drift Detection in Industrial Environments
Authors:
Bang Xiang Yong,
Yasmin Fathy,
Alexandra Brintrup
Abstract:
Autoencoders are unsupervised models which have been used for detecting anomalies in multi-sensor environments. A typical use includes training a predictive model with data from sensors operating under normal conditions and using the model to detect anomalies. Anomalies can come either from real changes in the environment (real drift) or from faulty sensory devices (virtual drift); however, the us…
▽ More
Autoencoders are unsupervised models which have been used for detecting anomalies in multi-sensor environments. A typical use includes training a predictive model with data from sensors operating under normal conditions and using the model to detect anomalies. Anomalies can come either from real changes in the environment (real drift) or from faulty sensory devices (virtual drift); however, the use of Autoencoders to distinguish between different anomalies has not yet been considered. To this end, we first propose the development of Bayesian Autoencoders to quantify epistemic and aleatoric uncertainties. We then test the Bayesian Autoencoder using a real-world industrial dataset for hydraulic condition monitoring. The system is injected with noise and drifts, and we have found the epistemic uncertainty to be less sensitive to sensor perturbations as compared to the reconstruction loss. By observing the reconstructed signals with the uncertainties, we gain interpretable insights, and these uncertainties offer a potential avenue for distinguishing real and virtual drifts.
△ Less
Submitted 28 July, 2021;
originally announced July 2021.
-
Crop and weed classification based on AutoML
Authors:
Xuetao Jiang,
Binbin Yong,
Soheila Garshasbi,
Jun Shen,
Meiyu Jiang,
Qingguo Zhou
Abstract:
CNN models already play an important role in classification of crop and weed with high accuracy, more than 95% as reported in literature. However, to manually choose and fine-tune the deep learning models becomes laborious and indispensable in most traditional practices and research. Moreover, the classic objective functions are not thoroughly compatible with agricultural farming tasks as the corr…
▽ More
CNN models already play an important role in classification of crop and weed with high accuracy, more than 95% as reported in literature. However, to manually choose and fine-tune the deep learning models becomes laborious and indispensable in most traditional practices and research. Moreover, the classic objective functions are not thoroughly compatible with agricultural farming tasks as the corresponding models suffer from misclassifying crop to weed, often more likely than in other deep learning application domains. In this paper, we applied autonomous machine learning with a new objective function for crop and weed classification, achieving higher accuracy and lower crop killing rate (rate of identifying a crop as a weed). The experimental results show that our method outperforms state-of-the-art applications, for example, ResNet and VGG19.
△ Less
Submitted 28 March, 2022; v1 submitted 27 October, 2020;
originally announced October 2020.
-
Data taking strategy for the phase study in $ψ^{\prime} \to K^+K^-$
Authors:
Wang Bo-Qun,
Mo Xiao-Hu,
Yuan Chang-Zheng,
Ban Yong
Abstract:
The study of the relative phase between strong and electromagnetic amplitudes is of great importance for understanding the dynamics of charmonium decays. The information of the phase can be obtained model-independently by fitting the scan data of some special decay channels, one of which is $ψ^{\prime} \to K^{+}K^{-}$. To find out the optimal data taking strategy for a scan experiment in the measu…
▽ More
The study of the relative phase between strong and electromagnetic amplitudes is of great importance for understanding the dynamics of charmonium decays. The information of the phase can be obtained model-independently by fitting the scan data of some special decay channels, one of which is $ψ^{\prime} \to K^{+}K^{-}$. To find out the optimal data taking strategy for a scan experiment in the measurement of the phase in $ψ^{\prime} \to K^{+} K^{-}$, the minimization process is analyzed from a theoretical point of view. The result indicates that for one parameter fit, only one data taking point in the vicinity of a resonance peak is sufficient to acquire the optimal precision. Numerical results are obtained by fitting simulated scan data. Besides the results related to the relative phase between strong and electromagnetic amplitudes, the method is extended to analyze the fits of other resonant parameters, such as the mass and the total decay width of $ψ^{\prime}$.
△ Less
Submitted 24 May, 2012;
originally announced May 2012.
-
Fast computation of observed cross section for $ψ^{\prime} \to PP$ decays
Authors:
Wang Bo-Qun,
Mo Xiao-Hu,
Wang Ping,
Ban Yong
Abstract:
It has been conjectured that the relative phase between strong and electromagnetic amplitudes is universally $-90^{\circ}$ in charmonium decays. $ψ^{\prime}$ decaying into pseudoscalar pair provides a possibility to test this conjecture. However, the experimentally observed cross section for such a process is depicted by the two-fold integral which takes into account the initial state radiative (I…
▽ More
It has been conjectured that the relative phase between strong and electromagnetic amplitudes is universally $-90^{\circ}$ in charmonium decays. $ψ^{\prime}$ decaying into pseudoscalar pair provides a possibility to test this conjecture. However, the experimentally observed cross section for such a process is depicted by the two-fold integral which takes into account the initial state radiative (ISR) correction and energy spread effect. Using the generalized linear regression approach, a complex energy-dependent factor is approximated by a linear function of energy. Taking advantage of this simplification, the integration of ISR correction can be performed and an analytical expression with accuracy at the level of 1% is obtained. Then, the original two-fold integral is simplified into a one-fold integral, which reduces the total computing time by two orders of magnitude. Such a simplified expression for the observed cross section usually plays an indispensable role in the optimization of scan data taking, the determination of systematic uncertainty, and the analysis of data correlation.
△ Less
Submitted 19 August, 2010;
originally announced August 2010.
