-
Continual Dialogue State Tracking via Reason-of-Select Distillation
Authors:
Yujie Feng,
Bo Liu,
Xiaoyu Dong,
Zexin Lu,
Li-Ming Zhan,
Xiao-Ming Wu,
Albert Y. S. Lam
Abstract:
An ideal dialogue system requires continuous skill acquisition and adaptation to new tasks while retaining prior knowledge. Dialogue State Tracking (DST), vital in these systems, often involves learning new services and confronting catastrophic forgetting, along with a critical capability loss termed the "Value Selection Quandary." To address these challenges, we introduce the Reason-of-Select (Ro…
▽ More
An ideal dialogue system requires continuous skill acquisition and adaptation to new tasks while retaining prior knowledge. Dialogue State Tracking (DST), vital in these systems, often involves learning new services and confronting catastrophic forgetting, along with a critical capability loss termed the "Value Selection Quandary." To address these challenges, we introduce the Reason-of-Select (RoS) distillation method by enhancing smaller models with a novel 'meta-reasoning' capability. Meta-reasoning employs an enhanced multi-domain perspective, combining fragments of meta-knowledge from domain-specific dialogues during continual learning. This transcends traditional single-perspective reasoning. The domain bootstrapping process enhances the model's ability to dissect intricate dialogues from multiple possible values. Its domain-agnostic property aligns data distribution across different domains, effectively mitigating forgetting. Additionally, two novel improvements, "multi-value resolution" strategy and Semantic Contrastive Reasoning Selection method, significantly enhance RoS by generating DST-specific selection chains and mitigating hallucinations in teachers' reasoning, ensuring effective and reliable knowledge transfer. Extensive experiments validate the exceptional performance and robust generalization capabilities of our method. The source code is provided for reproducibility.
△ Less
Submitted 19 August, 2024;
originally announced August 2024.
-
Minimizing PLM-Based Few-Shot Intent Detectors
Authors:
Haode Zhang,
Xiao-Ming Wu,
Albert Y. S. Lam
Abstract:
Recent research has demonstrated the feasibility of training efficient intent detectors based on pre-trained language model~(PLM) with limited labeled data. However, deploying these detectors in resource-constrained environments such as mobile devices poses challenges due to their large sizes. In this work, we aim to address this issue by exploring techniques to minimize the size of PLM-based inte…
▽ More
Recent research has demonstrated the feasibility of training efficient intent detectors based on pre-trained language model~(PLM) with limited labeled data. However, deploying these detectors in resource-constrained environments such as mobile devices poses challenges due to their large sizes. In this work, we aim to address this issue by exploring techniques to minimize the size of PLM-based intent detectors trained with few-shot data. Specifically, we utilize large language models (LLMs) for data augmentation, employ a cutting-edge model compression method for knowledge distillation, and devise a vocabulary pruning mechanism called V-Prune. Through these approaches, we successfully achieve a compression ratio of 21 in model memory usage, including both Transformer and the vocabulary, while maintaining almost identical performance levels on four real-world benchmarks.
△ Less
Submitted 13 July, 2024;
originally announced July 2024.
-
Revisit Few-shot Intent Classification with PLMs: Direct Fine-tuning vs. Continual Pre-training
Authors:
Haode Zhang,
Haowen Liang,
Liming Zhan,
Xiao-Ming Wu,
Albert Y. S. Lam
Abstract:
We consider the task of few-shot intent detection, which involves training a deep learning model to classify utterances based on their underlying intents using only a small amount of labeled data. The current approach to address this problem is through continual pre-training, i.e., fine-tuning pre-trained language models (PLMs) on external resources (e.g., conversational corpora, public intent det…
▽ More
We consider the task of few-shot intent detection, which involves training a deep learning model to classify utterances based on their underlying intents using only a small amount of labeled data. The current approach to address this problem is through continual pre-training, i.e., fine-tuning pre-trained language models (PLMs) on external resources (e.g., conversational corpora, public intent detection datasets, or natural language understanding datasets) before using them as utterance encoders for training an intent classifier. In this paper, we show that continual pre-training may not be essential, since the overfitting problem of PLMs on this task may not be as serious as expected. Specifically, we find that directly fine-tuning PLMs on only a handful of labeled examples already yields decent results compared to methods that employ continual pre-training, and the performance gap diminishes rapidly as the number of labeled data increases. To maximize the utilization of the limited available data, we propose a context augmentation method and leverage sequential self-distillation to boost performance. Comprehensive experiments on real-world benchmarks show that given only two or more labeled samples per class, direct fine-tuning outperforms many strong baselines that utilize external data sources for continual pre-training. The code can be found at https://github.com/hdzhang-code/DFTPlus.
△ Less
Submitted 8 June, 2023;
originally announced June 2023.
-
New Intent Discovery with Pre-training and Contrastive Learning
Authors:
Yuwei Zhang,
Haode Zhang,
Li-Ming Zhan,
Xiao-Ming Wu,
Albert Y. S. Lam
Abstract:
New intent discovery aims to uncover novel intent categories from user utterances to expand the set of supported intent classes. It is a critical task for the development and service expansion of a practical dialogue system. Despite its importance, this problem remains under-explored in the literature. Existing approaches typically rely on a large amount of labeled utterances and employ pseudo-lab…
▽ More
New intent discovery aims to uncover novel intent categories from user utterances to expand the set of supported intent classes. It is a critical task for the development and service expansion of a practical dialogue system. Despite its importance, this problem remains under-explored in the literature. Existing approaches typically rely on a large amount of labeled utterances and employ pseudo-labeling methods for representation learning and clustering, which are label-intensive, inefficient, and inaccurate. In this paper, we provide new solutions to two important research questions for new intent discovery: (1) how to learn semantic utterance representations and (2) how to better cluster utterances. Particularly, we first propose a multi-task pre-training strategy to leverage rich unlabeled data along with external labeled data for representation learning. Then, we design a new contrastive loss to exploit self-supervisory signals in unlabeled data for clustering. Extensive experiments on three intent recognition benchmarks demonstrate the high effectiveness of our proposed method, which outperforms state-of-the-art methods by a large margin in both unsupervised and semi-supervised scenarios. The source code will be available at \url{https://github.com/zhang-yu-wei/MTP-CLNN}.
△ Less
Submitted 25 May, 2022;
originally announced May 2022.
-
Fine-tuning Pre-trained Language Models for Few-shot Intent Detection: Supervised Pre-training and Isotropization
Authors:
Haode Zhang,
Haowen Liang,
Yuwei Zhang,
Liming Zhan,
Xiao-Ming Wu,
Xiaolei Lu,
Albert Y. S. Lam
Abstract:
It is challenging to train a good intent classifier for a task-oriented dialogue system with only a few annotations. Recent studies have shown that fine-tuning pre-trained language models with a small amount of labeled utterances from public benchmarks in a supervised manner is extremely helpful. However, we find that supervised pre-training yields an anisotropic feature space, which may suppress…
▽ More
It is challenging to train a good intent classifier for a task-oriented dialogue system with only a few annotations. Recent studies have shown that fine-tuning pre-trained language models with a small amount of labeled utterances from public benchmarks in a supervised manner is extremely helpful. However, we find that supervised pre-training yields an anisotropic feature space, which may suppress the expressive power of the semantic representations. Inspired by recent research in isotropization, we propose to improve supervised pre-training by regularizing the feature space towards isotropy. We propose two regularizers based on contrastive learning and correlation matrix respectively, and demonstrate their effectiveness through extensive experiments. Our main finding is that it is promising to regularize supervised pre-training with isotropization to further improve the performance of few-shot intent detection. The source code can be found at https://github.com/fanolabs/isoIntentBert-main.
△ Less
Submitted 26 May, 2022; v1 submitted 15 May, 2022;
originally announced May 2022.
-
Two-Stage Auction Mechanism for Long-Term Participation in Crowdsourcing
Authors:
Timothy Shin Heng Mak,
Albert Y. S. Lam
Abstract:
Crowdsourcing has become an important tool to collect data for various artificial intelligence applications and auction can be an effective way to allocate work and determine reward in a crowdsourcing platform. In this paper, we focus on the crowdsourcing of small tasks such as image labelling and voice recording where we face a number of challenges. First, workers have different limits on the amo…
▽ More
Crowdsourcing has become an important tool to collect data for various artificial intelligence applications and auction can be an effective way to allocate work and determine reward in a crowdsourcing platform. In this paper, we focus on the crowdsourcing of small tasks such as image labelling and voice recording where we face a number of challenges. First, workers have different limits on the amount of work they would be willing to do, and they may also misreport these limits in their bid for work. Secondly, if the auction is repeated over time, unsuccessful workers may drop out of the system, reducing competition and diversity. To tackle these issues, we first extend the results of the celebrated Myerson's optimal auction mechanism for a single-parameter bid to the case where the bid consists of the unit cost of work, the maximum amount of work one is willing to do, and the actual work completed. We show that a simple payment mechanism is sufficient to ensure a dominant strategy from the workers, and that this dominant strategy is robust to the true utility function of the workers. Secondly, we propose a novel, flexible work allocation mechanism, which allows the requester to balance between cost efficiency and equality. While cost minimization is obviously important, encouraging equality in the allocation of work increases the diversity of the workforce as well as promotes long-term participation on the crowdsourcing platform. Our main results are proved analytically and validated through simulations.
△ Less
Submitted 21 February, 2022;
originally announced February 2022.
-
Effectiveness of Pre-training for Few-shot Intent Classification
Authors:
Haode Zhang,
Yuwei Zhang,
Li-Ming Zhan,
Jiaxin Chen,
Guangyuan Shi,
Xiao-Ming Wu,
Albert Y. S. Lam
Abstract:
This paper investigates the effectiveness of pre-training for few-shot intent classification. While existing paradigms commonly further pre-train language models such as BERT on a vast amount of unlabeled corpus, we find it highly effective and efficient to simply fine-tune BERT with a small set of labeled utterances from public datasets. Specifically, fine-tuning BERT with roughly 1,000 labeled d…
▽ More
This paper investigates the effectiveness of pre-training for few-shot intent classification. While existing paradigms commonly further pre-train language models such as BERT on a vast amount of unlabeled corpus, we find it highly effective and efficient to simply fine-tune BERT with a small set of labeled utterances from public datasets. Specifically, fine-tuning BERT with roughly 1,000 labeled data yields a pre-trained model -- IntentBERT, which can easily surpass the performance of existing pre-trained models for few-shot intent classification on novel domains with very different semantics. The high effectiveness of IntentBERT confirms the feasibility and practicality of few-shot intent detection, and its high generalization ability across different domains suggests that intent classification tasks may share a similar underlying structure, which can be efficiently learned from a small set of labeled data. The source code can be found at https://github.com/hdzhang-code/IntentBERT.
△ Less
Submitted 13 September, 2021;
originally announced September 2021.
-
Out-of-Scope Intent Detection with Self-Supervision and Discriminative Training
Authors:
Li-Ming Zhan,
Haowen Liang,
Bo Liu,
Lu Fan,
Xiao-Ming Wu,
Albert Y. S. Lam
Abstract:
Out-of-scope intent detection is of practical importance in task-oriented dialogue systems. Since the distribution of outlier utterances is arbitrary and unknown in the training stage, existing methods commonly rely on strong assumptions on data distribution such as mixture of Gaussians to make inference, resulting in either complex multi-step training procedures or hand-crafted rules such as conf…
▽ More
Out-of-scope intent detection is of practical importance in task-oriented dialogue systems. Since the distribution of outlier utterances is arbitrary and unknown in the training stage, existing methods commonly rely on strong assumptions on data distribution such as mixture of Gaussians to make inference, resulting in either complex multi-step training procedures or hand-crafted rules such as confidence threshold selection for outlier detection. In this paper, we propose a simple yet effective method to train an out-of-scope intent classifier in a fully end-to-end manner by simulating the test scenario in training, which requires no assumption on data distribution and no additional post-processing or threshold setting. Specifically, we construct a set of pseudo outliers in the training stage, by generating synthetic outliers using inliner features via self-supervision and sampling out-of-scope sentences from easily available open-domain datasets. The pseudo outliers are used to train a discriminative classifier that can be directly applied to and generalize well on the test task. We evaluate our method extensively on four benchmark dialogue datasets and observe significant improvements over state-of-the-art approaches. Our code has been released at https://github.com/liam0949/DCLOOS.
△ Less
Submitted 17 June, 2021; v1 submitted 16 June, 2021;
originally announced June 2021.
-
A Unified Framework for Wide Area Measurement System Planning
Authors:
James J. Q. Yu,
Albert Y. S. Lam,
David J. Hill,
Victor O. K. Li
Abstract:
Wide area measurement system (WAMS) is one of the essential components in the future power system. To make WAMS construction plans, practical models of the power network observability, reliability, and underlying communication infrastructures need to be considered. To address this challenging problem, in this paper we propose a unified framework for WAMS planning to cover most realistic concerns i…
▽ More
Wide area measurement system (WAMS) is one of the essential components in the future power system. To make WAMS construction plans, practical models of the power network observability, reliability, and underlying communication infrastructures need to be considered. To address this challenging problem, in this paper we propose a unified framework for WAMS planning to cover most realistic concerns in the construction process. The framework jointly optimizes the system construction cost, measurement reliability, and volume of synchrophasor data traffic resulting in a multi-objective optimization problem, which provides multiple Pareto optimal solutions to suit different requirements by the utilities. The framework is verified on two IEEE test systems. The simulation results demonstrate the trade-off relationships among the proposed objectives. Moreover, the proposed framework can develop optimal WAMS plans for full observability with minimal cost. This work develops a comprehensive framework for most practical WAMS construction designs.
△ Less
Submitted 21 November, 2017;
originally announced November 2017.
-
Delay Aware Intelligent Transient Stability Assessment System
Authors:
James J. Q. Yu,
Albert Y. S. Lam,
David J. Hill,
Victor O. K. Li
Abstract:
Transient stability assessment is a critical tool for power system design and operation. With the emerging advanced synchrophasor measurement techniques, machine learning methods are playing an increasingly important role in power system stability assessment. However, most existing research makes a strong assumption that the measurement data transmission delay is negligible. In this paper, we focu…
▽ More
Transient stability assessment is a critical tool for power system design and operation. With the emerging advanced synchrophasor measurement techniques, machine learning methods are playing an increasingly important role in power system stability assessment. However, most existing research makes a strong assumption that the measurement data transmission delay is negligible. In this paper, we focus on investigating the influence of communication delay on synchrophasor-based transient stability assessment. In particular, we develop a delay aware intelligent system to address this issue. By utilizing an ensemble of multiple long short-term memory networks, the proposed system can make early assessments to achieve a much shorter response time by utilizing incomplete system variable measurements. Compared with existing work, our system is able to make accurate assessments with a significantly improved efficiency. We perform numerous case studies to demonstrate the superiority of the proposed intelligent system, in which accurate assessments can be developed with time one third less than state-of-the-art methodologies. Moreover, the simulations indicate that noise in the measurements has trivial impact on the assessment performance, demonstrating the robustness of the proposed system.
△ Less
Submitted 21 November, 2017;
originally announced November 2017.
-
A multi-layer market for vehicle-to-grid energy trading in the smart grid
Authors:
Albert Y. S. Lam,
Longbo Huang,
Alonso Silva,
Walid Saad
Abstract:
In this work, we propose a multi-layer market for vehicle-to-grid energy trading. In the macro layer, we consider a double auction mechanism, under which the utility company act as an auctioneer and energy buyers and sellers interact. This double auction mechanism is strategy-proof and converges asymptotically. In the micro layer, the aggregators, which are the sellers in the macro layer, are paid…
▽ More
In this work, we propose a multi-layer market for vehicle-to-grid energy trading. In the macro layer, we consider a double auction mechanism, under which the utility company act as an auctioneer and energy buyers and sellers interact. This double auction mechanism is strategy-proof and converges asymptotically. In the micro layer, the aggregators, which are the sellers in the macro layer, are paid with commissions to sell the energy of plug-in hybrid electric vehicles (PHEVs) and to maximize their utilities. We analyze the interaction between the macro and micro layers and study some simplified cases. Depending on the elasticity of supply and demand, the utility is analyzed under different scenarios. Simulation results show that our approach can significantly increase the utility of PHEVs.
△ Less
Submitted 6 September, 2016;
originally announced September 2016.
-
Adaptive Chemical Reaction Optimization for Global Numerical Optimization
Authors:
James J. Q. Yu,
Albert Y. S. Lam,
Victor O. K. Li
Abstract:
A newly proposed chemical-reaction-inspired metaheurisic, Chemical Reaction Optimization (CRO), has been applied to many optimization problems in both discrete and continuous domains. To alleviate the effort in tuning parameters, this paper reduces the number of optimization parameters in canonical CRO and develops an adaptive scheme to evolve them. Our proposed Adaptive CRO (ACRO) adapts better t…
▽ More
A newly proposed chemical-reaction-inspired metaheurisic, Chemical Reaction Optimization (CRO), has been applied to many optimization problems in both discrete and continuous domains. To alleviate the effort in tuning parameters, this paper reduces the number of optimization parameters in canonical CRO and develops an adaptive scheme to evolve them. Our proposed Adaptive CRO (ACRO) adapts better to different optimization problems. We perform simulations with ACRO on a widely-used benchmark of continuous problems. The simulation results show that ACRO has superior performance over canonical CRO.
△ Less
Submitted 9 July, 2015;
originally announced July 2015.
-
Combinatorial Auction-Based Pricing for Multi-tenant Autonomous Vehicle Public Transportation System
Authors:
Albert Y. S. Lam
Abstract:
A smart city provides its people with high standard of living through advanced technologies and transport is one of the major foci. With the advent of autonomous vehicles (AVs), an AV-based public transportation system has been proposed recently, which is capable of providing new forms of transportation services with high efficiency, high flexibility, and low cost. For the benefit of passengers, m…
▽ More
A smart city provides its people with high standard of living through advanced technologies and transport is one of the major foci. With the advent of autonomous vehicles (AVs), an AV-based public transportation system has been proposed recently, which is capable of providing new forms of transportation services with high efficiency, high flexibility, and low cost. For the benefit of passengers, multitenancy can increase market competition leading to lower service charge and higher quality of service. In this paper, we study the pricing issue of the multi-tenant AV public transportation system and three types of services are defined. The pricing process for each service type is modeled as a combinatorial auction, in which the service providers, as bidders, compete for offering transportation services. The winners of the auction are determined through an integer linear program. To prevent the bidders from raising their bids for higher returns, we propose a strategy-proof Vickrey-Clarke-Groves-based charging mechanism, which can maximize the social welfare, to settle the final charges for the customers. We perform extensive simulations to verify the analytical results and evaluate the performance of the charging mechanism.
△ Less
Submitted 20 September, 2015; v1 submitted 3 March, 2015;
originally announced March 2015.
-
Chemical Reaction Optimization for the Set Covering Problem
Authors:
James J. Q. Yu,
Albert Y. S. Lam,
Victor O. K. Li
Abstract:
The set covering problem (SCP) is one of the representative combinatorial optimization problems, having many practical applications. This paper investigates the development of an algorithm to solve SCP by employing chemical reaction optimization (CRO), a general-purpose metaheuristic. It is tested on a wide range of benchmark instances of SCP. The simulation results indicate that this algorithm gi…
▽ More
The set covering problem (SCP) is one of the representative combinatorial optimization problems, having many practical applications. This paper investigates the development of an algorithm to solve SCP by employing chemical reaction optimization (CRO), a general-purpose metaheuristic. It is tested on a wide range of benchmark instances of SCP. The simulation results indicate that this algorithm gives outstanding performance compared with other heuristics and metaheuristics in solving SCP.
△ Less
Submitted 31 January, 2015;
originally announced February 2015.
-
An Inter-molecular Adaptive Collision Scheme for Chemical Reaction Optimization
Authors:
James J. Q. Yu,
Victor O. K. Li,
Albert Y. S. Lam
Abstract:
Optimization techniques are frequently applied in science and engineering research and development. Evolutionary algorithms, as a kind of general-purpose metaheuristic, have been shown to be very effective in solving a wide range of optimization problems. A recently proposed chemical-reaction-inspired metaheuristic, Chemical Reaction Optimization (CRO), has been applied to solve many global optimi…
▽ More
Optimization techniques are frequently applied in science and engineering research and development. Evolutionary algorithms, as a kind of general-purpose metaheuristic, have been shown to be very effective in solving a wide range of optimization problems. A recently proposed chemical-reaction-inspired metaheuristic, Chemical Reaction Optimization (CRO), has been applied to solve many global optimization problems. However, the functionality of the inter-molecular ineffective collision operator in the canonical CRO design overlaps that of the on-wall ineffective collision operator, which can potential impair the overall performance. In this paper we propose a new inter-molecular ineffective collision operator for CRO for global optimization. To fully utilize our newly proposed operator, we also design a scheme to adapt the algorithm to optimization problems with different search space characteristics. We analyze the performance of our proposed algorithm with a number of widely used benchmark functions. The simulation results indicate that the new algorithm has superior performance over the canonical CRO.
△ Less
Submitted 31 January, 2015;
originally announced February 2015.
-
Optimal V2G Scheduling of Electric Vehicles and Unit Commitment using Chemical Reaction Optimization
Authors:
James J. Q. Yu,
Victor O. K. Li,
Albert Y. S. Lam
Abstract:
An electric vehicle (EV) may be used as energy storage which allows the bi-directional electricity flow between the vehicle's battery and the electric power grid. In order to flatten the load profile of the electricity system, EV scheduling has become a hot research topic in recent years. In this paper, we propose a new formulation of the joint scheduling of EV and Unit Commitment (UC), called EVU…
▽ More
An electric vehicle (EV) may be used as energy storage which allows the bi-directional electricity flow between the vehicle's battery and the electric power grid. In order to flatten the load profile of the electricity system, EV scheduling has become a hot research topic in recent years. In this paper, we propose a new formulation of the joint scheduling of EV and Unit Commitment (UC), called EVUC. Our formulation considers the characteristics of EVs while optimizing the system total running cost. We employ Chemical Reaction Optimization (CRO), a general-purpose optimization algorithm to solve this problem and the simulation results on a widely used set of instances indicate that CRO can effectively optimize this problem.
△ Less
Submitted 31 January, 2015;
originally announced February 2015.
-
Sensor Deployment for Air Pollution Monitoring Using Public Transportation System
Authors:
James J. Q. Yu,
Victor O. K. Li,
Albert Y. S. Lam
Abstract:
Air pollution monitoring is a very popular research topic and many monitoring systems have been developed. In this paper, we formulate the Bus Sensor Deployment Problem (BSDP) to select the bus routes on which sensors are deployed, and we use Chemical Reaction Optimization (CRO) to solve BSDP. CRO is a recently proposed metaheuristic designed to solve a wide range of optimization problems. Using t…
▽ More
Air pollution monitoring is a very popular research topic and many monitoring systems have been developed. In this paper, we formulate the Bus Sensor Deployment Problem (BSDP) to select the bus routes on which sensors are deployed, and we use Chemical Reaction Optimization (CRO) to solve BSDP. CRO is a recently proposed metaheuristic designed to solve a wide range of optimization problems. Using the real world data, namely Hong Kong Island bus route data, we perform a series of simulations and the results show that CRO is capable of solving this optimization problem efficiently.
△ Less
Submitted 31 January, 2015;
originally announced February 2015.
-
Real-Coded Chemical Reaction Optimization with Different Perturbation Functions
Authors:
James J. Q. Yu,
Albert Y. S. Lam,
Victor O. K. Li
Abstract:
Chemical Reaction Optimization (CRO) is a powerful metaheuristic which mimics the interactions of molecules in chemical reactions to search for the global optimum. The perturbation function greatly influences the performance of CRO on solving different continuous problems. In this paper, we study four different probability distributions, namely, the Gaussian distribution, the Cauchy distribution,…
▽ More
Chemical Reaction Optimization (CRO) is a powerful metaheuristic which mimics the interactions of molecules in chemical reactions to search for the global optimum. The perturbation function greatly influences the performance of CRO on solving different continuous problems. In this paper, we study four different probability distributions, namely, the Gaussian distribution, the Cauchy distribution, the exponential distribution, and a modified Rayleigh distribution, for the perturbation function of CRO. Different distributions have different impacts on the solutions. The distributions are tested by a set of well-known benchmark functions and simulation results show that problems with different characteristics have different preference on the distribution function. Our study gives guidelines to design CRO for different types of optimization problems.
△ Less
Submitted 31 January, 2015;
originally announced February 2015.
-
Evolutionary Artificial Neural Network Based on Chemical Reaction Optimization
Authors:
James J. Q. Yu,
Albert Y. S. Lam,
Victor O. K. Li
Abstract:
Evolutionary algorithms (EAs) are very popular tools to design and evolve artificial neural networks (ANNs), especially to train them. These methods have advantages over the conventional backpropagation (BP) method because of their low computational requirement when searching in a large solution space. In this paper, we employ Chemical Reaction Optimization (CRO), a newly developed global optimiza…
▽ More
Evolutionary algorithms (EAs) are very popular tools to design and evolve artificial neural networks (ANNs), especially to train them. These methods have advantages over the conventional backpropagation (BP) method because of their low computational requirement when searching in a large solution space. In this paper, we employ Chemical Reaction Optimization (CRO), a newly developed global optimization method, to replace BP in training neural networks. CRO is a population-based metaheuristics mimicking the transition of molecules and their interactions in a chemical reaction. Simulation results show that CRO outperforms many EA strategies commonly used to train neural networks.
△ Less
Submitted 31 January, 2015;
originally announced February 2015.
-
Information-Theoretic Bounds for Performance of Resource-Constrained Communication Systems
Authors:
Albert Y. S. Lam,
Yanhui Geng,
Victor O. K. Li
Abstract:
Resource-constrained systems are prevalent in communications. Such a system is composed of many components but only some of them can be allocated with resources such as time slots. According to the amount of information about the system, algorithms are employed to allocate resources and the overall system performance depends on the result of resource allocation. We do not always have complete info…
▽ More
Resource-constrained systems are prevalent in communications. Such a system is composed of many components but only some of them can be allocated with resources such as time slots. According to the amount of information about the system, algorithms are employed to allocate resources and the overall system performance depends on the result of resource allocation. We do not always have complete information, and thus, the system performance may not be satisfactory. In this work, we propose a general model for the resource-constrained communication systems. We draw the relationship between system information and performance and derive the performance bounds for the optimal algorithm for the system. This gives the expected performance corresponding to the available information, and we can determine if we should put more efforts to collect more accurate information before actually constructing an algorithm for the system. Several examples of applications in communications to the model are also given.
△ Less
Submitted 1 April, 2014;
originally announced April 2014.
-
An Optimal and Distributed Method for Voltage Regulation in Power Distribution Systems
Authors:
Baosen Zhang,
Albert Y. S. Lam,
Alejandro Dominguez-Garcia,
David Tse
Abstract:
This paper addresses the problem of voltage regulation in power distribution networks with deep-penetration of distributed energy resources, e.g., renewable-based generation, and storage-capable loads such as plug-in hybrid electric vehicles. We cast the problem as an optimization program, where the objective is to minimize the losses in the network subject to constraints on bus voltage magnitudes…
▽ More
This paper addresses the problem of voltage regulation in power distribution networks with deep-penetration of distributed energy resources, e.g., renewable-based generation, and storage-capable loads such as plug-in hybrid electric vehicles. We cast the problem as an optimization program, where the objective is to minimize the losses in the network subject to constraints on bus voltage magnitudes, limits on active and reactive power injections, transmission line thermal limits and losses. We provide sufficient conditions under which the optimization problem can be solved via its convex relaxation. Using data from existing networks, we show that these sufficient conditions are expected to be satisfied by most networks. We also provide an efficient distributed algorithm to solve the problem. The algorithm adheres to a communication topology described by a graph that is the same as the graph that describes the electrical network topology. We illustrate the operation of the algorithm, including its robustness against communication link failures, through several case studies involving 5-, 34-, and 123-bus power distribution systems.
△ Less
Submitted 7 February, 2015; v1 submitted 23 April, 2012;
originally announced April 2012.