-
The Feasibility of Implementing Large-Scale Transformers on Multi-FPGA Platforms
Authors:
Yu Gao,
Juan Camilo Vega,
Paul Chow
Abstract:
FPGAs are rarely mentioned when discussing the implementation of large machine learning applications, such as Large Language Models (LLMs), in the data center. There has been much evidence showing that single FPGAs can be competitive with GPUs in performance for some computations, especially for low latency, and often much more efficient when power is considered. This suggests that there is merit…
▽ More
FPGAs are rarely mentioned when discussing the implementation of large machine learning applications, such as Large Language Models (LLMs), in the data center. There has been much evidence showing that single FPGAs can be competitive with GPUs in performance for some computations, especially for low latency, and often much more efficient when power is considered. This suggests that there is merit to exploring the use of multiple FPGAs for large machine learning applications. The challenge with using multiple FPGAs is that there is no commonly-accepted flow for developing and deploying multi-FPGA applications, i.e., there are no tools to describe a large application, map it to multiple FPGAs and then deploy the application on a multi-FPGA platform. In this paper, we explore the feasibility of implementing large transformers using multiple FPGAs by developing a scalable multi-FPGA platform and some tools to map large applications to the platform. We validate our approach by designing an efficient multi-FPGA version of the I-BERT transformer and implement one encoder using six FPGAs as a working proof-of-concept to show that our platform and tools work. Based on our proof-of-concept prototype and the estimations of performance using the latest FPGAs compared to GPUs, we conclude that there can be a place for FPGAs in the world of large machine learning applications. We demonstrate a promising first step that shows that with the right infrastructure and tools it is reasonable to continue to explore the possible benefits of using FPGAs for applications such as LLMs.
△ Less
Submitted 24 April, 2024;
originally announced April 2024.
-
High-resolution canopy height map in the Landes forest (France) based on GEDI, Sentinel-1, and Sentinel-2 data with a deep learning approach
Authors:
Martin Schwartz,
Philippe Ciais,
Catherine Ottlé,
Aurelien De Truchis,
Cedric Vega,
Ibrahim Fayad,
Martin Brandt,
Rasmus Fensholt,
Nicolas Baghdadi,
François Morneau,
David Morin,
Dominique Guyon,
Sylvia Dayau,
Jean-Pierre Wigneron
Abstract:
In intensively managed forests in Europe, where forests are divided into stands of small size and may show heterogeneity within stands, a high spatial resolution (10 - 20 meters) is arguably needed to capture the differences in canopy height. In this work, we developed a deep learning model based on multi-stream remote sensing measurements to create a high-resolution canopy height map over the "La…
▽ More
In intensively managed forests in Europe, where forests are divided into stands of small size and may show heterogeneity within stands, a high spatial resolution (10 - 20 meters) is arguably needed to capture the differences in canopy height. In this work, we developed a deep learning model based on multi-stream remote sensing measurements to create a high-resolution canopy height map over the "Landes de Gascogne" forest in France, a large maritime pine plantation of 13,000 km$^2$ with flat terrain and intensive management. This area is characterized by even-aged and mono-specific stands, of a typical length of a few hundred meters, harvested every 35 to 50 years. Our deep learning U-Net model uses multi-band images from Sentinel-1 and Sentinel-2 with composite time averages as input to predict tree height derived from GEDI waveforms. The evaluation is performed with external validation data from forest inventory plots and a stereo 3D reconstruction model based on Skysat imagery available at specific locations. We trained seven different U-net models based on a combination of Sentinel-1 and Sentinel-2 bands to evaluate the importance of each instrument in the dominant height retrieval. The model outputs allow us to generate a 10 m resolution canopy height map of the whole "Landes de Gascogne" forest area for 2020 with a mean absolute error of 2.02 m on the Test dataset. The best predictions were obtained using all available satellite layers from Sentinel-1 and Sentinel-2 but using only one satellite source also provided good predictions. For all validation datasets in coniferous forests, our model showed better metrics than previous canopy height models available in the same region.
△ Less
Submitted 20 December, 2022;
originally announced December 2022.
-
Fuzzy temporal convolutional neural networks in P300-based Brain-computer interface for smart home interaction
Authors:
Christian Flores Vega,
Jonathan Quevedo,
Elmer Escandón,
Mehrin Kiani,
Weiping Ding,
Javier Andreu-Perez
Abstract:
The processing and classification of electroencephalographic signals (EEG) are increasingly performed using deep learning frameworks, such as convolutional neural networks (CNNs), to generate abstract features from brain data, automatically paving the way for remarkable classification prowess. However, EEG patterns exhibit high variability across time and uncertainty due to noise. It is a signific…
▽ More
The processing and classification of electroencephalographic signals (EEG) are increasingly performed using deep learning frameworks, such as convolutional neural networks (CNNs), to generate abstract features from brain data, automatically paving the way for remarkable classification prowess. However, EEG patterns exhibit high variability across time and uncertainty due to noise. It is a significant problem to be addressed in P300-based Brain Computer Interface (BCI) for smart home interaction. It operates in a non-optimal natural environment where added noise is often present. In this work, we propose a sequential unification of temporal convolutional networks (TCNs) modified to EEG signals, LSTM cells, with a fuzzy neural block (FNB), which we called EEG-TCFNet. Fuzzy components may enable a higher tolerance to noisy conditions. We applied three different architectures comparing the effect of using block FNB to classify a P300 wave to build a BCI for smart home interaction with healthy and post-stroke individuals. Our results reported a maximum classification accuracy of 98.6% and 74.3% using the proposed method of EEG-TCFNet in subject-dependent strategy and subject-independent strategy, respectively. Overall, FNB usage in all three CNN topologies outperformed those without FNB. In addition, we compared the addition of FNB to other state-of-the-art methods and obtained higher classification accuracies on account of the integration with FNB. The remarkable performance of the proposed model, EEG-TCFNet, and the general integration of fuzzy units to other classifiers would pave the way for enhanced P300-based BCIs for smart home interaction within natural settings.
△ Less
Submitted 8 April, 2022;
originally announced April 2022.
-
MeLPUF: Memory-in-Logic PUF Structures for Low-Overhead IC Authentication
Authors:
Christopher Vega,
Shubhra Deb Paul,
Patanjali SLPSK,
Swarup Bhunia
Abstract:
Physically Unclonable Functions (PUFs) are used for securing electronic devices across the implementation spectrum ranging from Field Programmable Gate Array (FPGA) to system on chips (SoCs). However, existing PUF implementations often suffer from one or more significant deficiencies: (1) significant design overhead; (2) difficulty to configure and integrate based on application-specific requireme…
▽ More
Physically Unclonable Functions (PUFs) are used for securing electronic devices across the implementation spectrum ranging from Field Programmable Gate Array (FPGA) to system on chips (SoCs). However, existing PUF implementations often suffer from one or more significant deficiencies: (1) significant design overhead; (2) difficulty to configure and integrate based on application-specific requirements; (3) vulnerability to model-building attacks; and (4) spatial locality to a specific region of a chip. These factors limit their application in the authentication of designs used in diverse applications. In this work, we propose MeLPUF: Memory-in-Logic PUF; a low-overhead, distributed PUF that leverages the existing logic gates in a design to create cross-coupled inverters (i.e., memory cells) in a logic circuit as an entropy source. It exploits these memory cells' power-up states as the entropy source to generate device-specific unique fingerprints. A dedicated control signal governs these on-demand memory cells. They can be dispersed across the combinational logic of a design to achieve distributed authentication. They can also be synthesized with a standard logic synthesis tool to meet the target area, power, and performance constraints. We evaluate the quality of MeLPUF signatures with circuit-level simulations and experimental measurements using FPGA silicon (TSMC 55nm process). Our analysis shows the high quality of the PUF in terms of uniqueness, randomness, and robustness while incurring modest overhead. We further demonstrate the scalability of MeLPUF by aggregating power-up states from multiple memory cells, thus creating PUF signatures or digital identifiers of varying lengths. Additionally, we suggest optimization techniques that can be leveraged to boost the performance of MeLPUF further.
△ Less
Submitted 29 March, 2023; v1 submitted 5 December, 2020;
originally announced December 2020.
-
KISS methodologies for network management and anomaly detection
Authors:
Carlos Vega,
Javier Aracil,
Eduardo Magaña
Abstract:
Current networks are increasingly growing in size and complexity and so is the amount of monitoring data that they produce. As a result, network monitoring systems have to scale accordingly. As a possible approach, horizontal scalability can be achieved by large data centralization systems based on clusters, which are expensive and difficult to deploy in a real production scenario. In this paper w…
▽ More
Current networks are increasingly growing in size and complexity and so is the amount of monitoring data that they produce. As a result, network monitoring systems have to scale accordingly. As a possible approach, horizontal scalability can be achieved by large data centralization systems based on clusters, which are expensive and difficult to deploy in a real production scenario. In this paper we propose and evaluate a series of methodologies, deployed in real industrial production environments, for network monitoring and management, from the architecture design to the visualization system as well as for the anomaly detection methodologies, that intend to squeeze the vertical resources and overcome the difficulties of data collection and centralization.
△ Less
Submitted 6 February, 2019;
originally announced February 2019.
-
Diluting the Scalability Boundaries: Exploring the Use of Disaggregated Architectures for High-Level Network Data Analysis
Authors:
Carlos Vega,
Jose Fernando Zazo,
Hugo Meyer,
Ferad Zyulkyarov,
Sergio Lopez Buedo,
Javier Aracil
Abstract:
Traditional data centers are designed with a rigid architecture of fit-for-purpose servers that provision resources beyond the average workload in order to deal with occasional peaks of data. Heterogeneous data centers are pushing towards more cost-efficient architectures with better resource provisioning. In this paper we study the feasibility of using disaggregated architectures for intensive da…
▽ More
Traditional data centers are designed with a rigid architecture of fit-for-purpose servers that provision resources beyond the average workload in order to deal with occasional peaks of data. Heterogeneous data centers are pushing towards more cost-efficient architectures with better resource provisioning. In this paper we study the feasibility of using disaggregated architectures for intensive data applications, in contrast to the monolithic approach of server-oriented architectures. Particularly, we have tested a proactive network analysis system in which the workload demands are highly variable. In the context of the dReDBox disaggregated architecture, the results show that the overhead caused by using remote memory resources is significant, between 66\% and 80\%, but we have also observed that the memory usage is one order of magnitude higher for the stress case with respect to average workloads. Therefore, dimensioning memory for the worst case in conventional systems will result in a notable waste of resources. Finally, we found that, for the selected use case, parallelism is limited by memory. Therefore, using a disaggregated architecture will allow for increased parallelism, which, at the same time, will mitigate the overhead caused by remote memory.
△ Less
Submitted 18 September, 2017;
originally announced September 2017.
-
Loginson: a transform and load system for very large scale log analysis in large IT infrastructures
Authors:
Carlos Vega,
Paula Roquero,
Rafael Leira,
Ivan Gonzalez,
Javier Aracil
Abstract:
Nowadays, most systems and applications produce log records that are useful for security and monitoring purposes such as debugging programming errors, checking system status, and detecting configuration problems or even attacks. To this end, a log repository becomes necessary whereby logs can be accessed and visualized in a timely manner. This paper presents Loginson, a high-performance log centra…
▽ More
Nowadays, most systems and applications produce log records that are useful for security and monitoring purposes such as debugging programming errors, checking system status, and detecting configuration problems or even attacks. To this end, a log repository becomes necessary whereby logs can be accessed and visualized in a timely manner. This paper presents Loginson, a high-performance log centralization system for large-scale log collection and processing in large IT infrastructures. Besides log collection, Loginson provides high-level analytics through a visual interface for the purpose of troubleshooting critical incidents. We note that Loginson outperforms all of the other log centralization solutions by taking full advantage of the vertical scalability, and therefore decreasing Capital Expenditure (CAPEX) and Operating Expense (OPEX) costs for deployment scenarios with a huge volume of log data.
△ Less
Submitted 7 March, 2017;
originally announced March 2017.
-
Multi-Gbps HTTP Traffic Analysis in Commodity Hardware Based on Local Knowledge of TCP Streams
Authors:
Carlos Vega,
Paula Roquero,
Javier Aracil
Abstract:
In this paper we propose and implement novel techniques for performance evaluation of web traffic (response time, response code, etc.), with no reassembly of the underlying TCP connection, which severely restricts the traffic analysis throughput. Furthermore, our proposed software for HTTP traffic analysis runs in standard hardware, which is very cost-effective. Besides, we present sub-TCP connect…
▽ More
In this paper we propose and implement novel techniques for performance evaluation of web traffic (response time, response code, etc.), with no reassembly of the underlying TCP connection, which severely restricts the traffic analysis throughput. Furthermore, our proposed software for HTTP traffic analysis runs in standard hardware, which is very cost-effective. Besides, we present sub-TCP connection load balancing techniques that significantly increase throughput at the expense of losing very few HTTP transactions. Such techniques provide performance evaluation statistics which are indistinguishable from the single-threaded alternative with full TCP connection reassembly.
△ Less
Submitted 17 January, 2017;
originally announced January 2017.
