-
Attend-Fusion: Efficient Audio-Visual Fusion for Video Classification
Authors:
Mahrukh Awan,
Asmar Nadeem,
Muhammad Junaid Awan,
Armin Mustafa,
Syed Sameed Husain
Abstract:
Exploiting both audio and visual modalities for video classification is a challenging task, as the existing methods require large model architectures, leading to high computational complexity and resource requirements. Smaller architectures, on the other hand, struggle to achieve optimal performance. In this paper, we propose Attend-Fusion, an audio-visual (AV) fusion approach that introduces a co…
▽ More
Exploiting both audio and visual modalities for video classification is a challenging task, as the existing methods require large model architectures, leading to high computational complexity and resource requirements. Smaller architectures, on the other hand, struggle to achieve optimal performance. In this paper, we propose Attend-Fusion, an audio-visual (AV) fusion approach that introduces a compact model architecture specifically designed to capture intricate audio-visual relationships in video data. Through extensive experiments on the challenging YouTube-8M dataset, we demonstrate that Attend-Fusion achieves an F1 score of 75.64\% with only 72M parameters, which is comparable to the performance of larger baseline models such as Fully-Connected Late Fusion (75.96\% F1 score, 341M parameters). Attend-Fusion achieves similar performance to the larger baseline model while reducing the model size by nearly 80\%, highlighting its efficiency in terms of model complexity. Our work demonstrates that the Attend-Fusion model effectively combines audio and visual information for video classification, achieving competitive performance with significantly reduced model size. This approach opens new possibilities for deploying high-performance video understanding systems in resource-constrained environments across various applications.
△ Less
Submitted 26 August, 2024;
originally announced August 2024.
-
Unsupervised Landmark Discovery Using Consistency Guided Bottleneck
Authors:
Mamona Awan,
Muhammad Haris Khan,
Sanoojan Baliah,
Muhammad Ahmad Waseem,
Salman Khan,
Fahad Shahbaz Khan,
Arif Mahmood
Abstract:
We study a challenging problem of unsupervised discovery of object landmarks. Many recent methods rely on bottlenecks to generate 2D Gaussian heatmaps however, these are limited in generating informed heatmaps while training, presumably due to the lack of effective structural cues. Also, it is assumed that all predicted landmarks are semantically relevant despite having no ground truth supervision…
▽ More
We study a challenging problem of unsupervised discovery of object landmarks. Many recent methods rely on bottlenecks to generate 2D Gaussian heatmaps however, these are limited in generating informed heatmaps while training, presumably due to the lack of effective structural cues. Also, it is assumed that all predicted landmarks are semantically relevant despite having no ground truth supervision. In the current work, we introduce a consistency-guided bottleneck in an image reconstruction-based pipeline that leverages landmark consistency, a measure of compatibility score with the pseudo-ground truth to generate adaptive heatmaps. We propose obtaining pseudo-supervision via forming landmark correspondence across images. The consistency then modulates the uncertainty of the discovered landmarks in the generation of adaptive heatmaps which rank consistent landmarks above their noisy counterparts, providing effective structural information for improved robustness. Evaluations on five diverse datasets including MAFL, AFLW, LS3D, Cats, and Shoes demonstrate excellent performance of the proposed approach compared to the existing state-of-the-art methods. Our code is publicly available at https://github.com/MamonaAwan/CGB_ULD.
△ Less
Submitted 19 September, 2023;
originally announced September 2023.
-
Evaluating the Potential of Disaggregated Memory Systems for HPC applications
Authors:
Nan Ding,
Pieter Maris,
Hai Ah Nam,
Taylor Groves,
Muaaz Gul Awan,
LeAnn Lindsey,
Christopher Daley,
Oguz Selvitopi,
Leonid Oliker,
Nicholas Wright,
Samuel Williams
Abstract:
Disaggregated memory is a promising approach that addresses the limitations of traditional memory architectures by enabling memory to be decoupled from compute nodes and shared across a data center. Cloud platforms have deployed such systems to improve overall system memory utilization, but performance can vary across workloads. High-performance computing (HPC) is crucial in scientific and enginee…
▽ More
Disaggregated memory is a promising approach that addresses the limitations of traditional memory architectures by enabling memory to be decoupled from compute nodes and shared across a data center. Cloud platforms have deployed such systems to improve overall system memory utilization, but performance can vary across workloads. High-performance computing (HPC) is crucial in scientific and engineering applications, where HPC machines also face the issue of underutilized memory. As a result, improving system memory utilization while understanding workload performance is essential for HPC operators. Therefore, learning the potential of a disaggregated memory system before deployment is a critical step. This paper proposes a methodology for exploring the design space of a disaggregated memory system. It incorporates key metrics that affect performance on disaggregated memory systems: memory capacity, local and remote memory access ratio, injection bandwidth, and bisection bandwidth, providing an intuitive approach to guide machine configurations based on technology trends and workload characteristics. We apply our methodology to analyze thirteen diverse workloads, including AI training, data analysis, genomics, protein, fusion, atomic nuclei, and traditional HPC bookends. Our methodology demonstrates the ability to comprehend the potential and pitfalls of a disaggregated memory system and provides motivation for machine configurations. Our results show that eleven of our thirteen applications can leverage injection bandwidth disaggregated memory without affecting performance, while one pays a rack bisection bandwidth penalty and two pay the system-wide bisection bandwidth penalty. In addition, we also show that intra-rack memory disaggregation would meet the application's memory requirement and provide enough remote memory bandwidth.
△ Less
Submitted 16 June, 2023; v1 submitted 6 June, 2023;
originally announced June 2023.
-
Extreme-scale many-against-many protein similarity search
Authors:
Oguz Selvitopi,
Saliya Ekanayake,
Giulia Guidi,
Muaaz G. Awan,
Georgios A. Pavlopoulos,
Ariful Azad,
Nikos Kyrpides,
Leonid Oliker,
Katherine Yelick,
Aydın Buluç
Abstract:
Similarity search is one of the most fundamental computations that are regularly performed on ever-increasing protein datasets. Scalability is of paramount importance for uncovering novel phenomena that occur at very large scales. We unleash the power of over 20,000 GPUs on the Summit system to perform all-vs-all protein similarity search on one of the largest publicly available datasets with 405…
▽ More
Similarity search is one of the most fundamental computations that are regularly performed on ever-increasing protein datasets. Scalability is of paramount importance for uncovering novel phenomena that occur at very large scales. We unleash the power of over 20,000 GPUs on the Summit system to perform all-vs-all protein similarity search on one of the largest publicly available datasets with 405 million proteins, in less than 3.5 hours, cutting the time-to-solution for many use cases from weeks. The variability of protein sequence lengths, as well as the sparsity of the space of pairwise comparisons, make this a challenging problem in distributed memory. Due to the need to construct and maintain a data structure holding indices to all other sequences, this application has a huge memory footprint that makes it hard to scale the problem sizes. We overcome this memory limitation by innovative matrix-based blocking techniques, without introducing additional load imbalance.
△ Less
Submitted 3 March, 2023;
originally announced March 2023.
-
Understanding the Power of Evolutionary Computation for GPU Code Optimization
Authors:
Jhe-Yu Liou,
Muaaz Awan,
Steven Hofmeyr,
Stephanie Forrest,
Carole-Jean Wu
Abstract:
Achieving high performance for GPU codes requires developers to have significant knowledge in parallel programming and GPU architectures, and in-depth understanding of the application. This combination makes it challenging to find performance optimizations for GPU-based applications, especially in scientific computing. This paper shows that significant speedups can be achieved on two quite differe…
▽ More
Achieving high performance for GPU codes requires developers to have significant knowledge in parallel programming and GPU architectures, and in-depth understanding of the application. This combination makes it challenging to find performance optimizations for GPU-based applications, especially in scientific computing. This paper shows that significant speedups can be achieved on two quite different scientific workloads using the tool, GEVO, to improve performance over human-optimized GPU code. GEVO uses evolutionary computation to find code edits that improve the runtime of a multiple sequence alignment kernel and a SARS-CoV-2 simulation by 28.9% and 29% respectively. Further, when GEVO begins with an early, unoptimized version of the sequence alignment program, it finds an impressive 30 times speedup -- a performance improvement similar to that of the hand-tuned version. This work presents an in-depth analysis of the discovered optimizations, revealing that the primary sources of improvement vary across applications; that most of the optimizations generalize across GPU architectures; and that several of the most important optimizations involve significant code interdependencies. The results showcase the potential of automated program optimization tools to help reduce the optimization burden for scientific computing developers and enhance performance portability for domain-specific accelerators.
△ Less
Submitted 25 August, 2022;
originally announced August 2022.
-
Almost-Matching-Exactly for Treatment Effect Estimation under Network Interference
Authors:
M. Usaid Awan,
Marco Morucci,
Vittorio Orlandi,
Sudeepa Roy,
Cynthia Rudin,
Alexander Volfovsky
Abstract:
We propose a matching method that recovers direct treatment effects from randomized experiments where units are connected in an observed network, and units that share edges can potentially influence each others' outcomes. Traditional treatment effect estimators for randomized experiments are biased and error prone in this setting. Our method matches units almost exactly on counts of unique subgrap…
▽ More
We propose a matching method that recovers direct treatment effects from randomized experiments where units are connected in an observed network, and units that share edges can potentially influence each others' outcomes. Traditional treatment effect estimators for randomized experiments are biased and error prone in this setting. Our method matches units almost exactly on counts of unique subgraphs within their neighborhood graphs. The matches that we construct are interpretable and high-quality. Our method can be extended easily to accommodate additional unit-level covariate information. We show empirically that our method performs better than other existing methodologies for this problem, while producing meaningful, interpretable results.
△ Less
Submitted 2 March, 2020;
originally announced March 2020.
-
The Parallelism Motifs of Genomic Data Analysis
Authors:
Katherine Yelick,
Aydin Buluc,
Muaaz Awan,
Ariful Azad,
Benjamin Brock,
Rob Egan,
Saliya Ekanayake,
Marquita Ellis,
Evangelos Georganas,
Giulia Guidi,
Steven Hofmeyr,
Oguz Selvitopi,
Cristina Teodoropol,
Leonid Oliker
Abstract:
Genomic data sets are growing dramatically as the cost of sequencing continues to decline and small sequencing devices become available. Enormous community databases store and share this data with the research community, but some of these genomic data analysis problems require large scale computational platforms to meet both the memory and computational requirements. These applications differ from…
▽ More
Genomic data sets are growing dramatically as the cost of sequencing continues to decline and small sequencing devices become available. Enormous community databases store and share this data with the research community, but some of these genomic data analysis problems require large scale computational platforms to meet both the memory and computational requirements. These applications differ from scientific simulations that dominate the workload on high end parallel systems today and place different requirements on programming support, software libraries, and parallel architectural design. For example, they involve irregular communication patterns such as asynchronous updates to shared data structures. We consider several problems in high performance genomics analysis, including alignment, profiling, clustering, and assembly for both single genomes and metagenomes. We identify some of the common computational patterns or motifs that help inform parallelization strategies and compare our motifs to some of the established lists, arguing that at least two key patterns, sorting and hashing, are missing.
△ Less
Submitted 20 January, 2020;
originally announced January 2020.
-
Interpretable Almost-Matching-Exactly With Instrumental Variables
Authors:
M. Usaid Awan,
Yameng Liu,
Marco Morucci,
Sudeepa Roy,
Cynthia Rudin,
Alexander Volfovsky
Abstract:
Uncertainty in the estimation of the causal effect in observational studies is often due to unmeasured confounding, i.e., the presence of unobserved covariates linking treatments and outcomes. Instrumental Variables (IV) are commonly used to reduce the effects of unmeasured confounding. Existing methods for IV estimation either require strong parametric assumptions, use arbitrary distance metrics,…
▽ More
Uncertainty in the estimation of the causal effect in observational studies is often due to unmeasured confounding, i.e., the presence of unobserved covariates linking treatments and outcomes. Instrumental Variables (IV) are commonly used to reduce the effects of unmeasured confounding. Existing methods for IV estimation either require strong parametric assumptions, use arbitrary distance metrics, or do not scale well to large datasets. We propose a matching framework for IV in the presence of observed categorical confounders that addresses these weaknesses. Our method first matches units exactly, and then consecutively drops variables to approximately match the remaining units on as many variables as possible. We show that our algorithm constructs better matches than other existing methods on simulated datasets, and we produce interesting results in an application to political canvassing.
△ Less
Submitted 28 July, 2019; v1 submitted 27 June, 2019;
originally announced June 2019.
-
FLAME: A Fast Large-scale Almost Matching Exactly Approach to Causal Inference
Authors:
Tianyu Wang,
Marco Morucci,
M. Usaid Awan,
Yameng Liu,
Sudeepa Roy,
Cynthia Rudin,
Alexander Volfovsky
Abstract:
A classical problem in causal inference is that of matching, where treatment units need to be matched to control units based on covariate information. In this work, we propose a method that computes high quality almost-exact matches for high-dimensional categorical datasets. This method, called FLAME (Fast Large-scale Almost Matching Exactly), learns a distance metric for matching using a hold-out…
▽ More
A classical problem in causal inference is that of matching, where treatment units need to be matched to control units based on covariate information. In this work, we propose a method that computes high quality almost-exact matches for high-dimensional categorical datasets. This method, called FLAME (Fast Large-scale Almost Matching Exactly), learns a distance metric for matching using a hold-out training data set. In order to perform matching efficiently for large datasets, FLAME leverages techniques that are natural for query processing in the area of database management, and two implementations of FLAME are provided: the first uses SQL queries and the second uses bit-vector techniques. The algorithm starts by constructing matches of the highest quality (exact matches on all covariates), and successively eliminates variables in order to match exactly on as many variables as possible, while still maintaining interpretable high-quality matches and balance between treatment and control groups. We leverage these high quality matches to estimate conditional average treatment effects (CATEs). Our experiments show that FLAME scales to huge datasets with millions of observations where existing state-of-the-art methods fail, and that it achieves significantly better performance than other matching methods.
△ Less
Submitted 4 February, 2021; v1 submitted 19 July, 2017;
originally announced July 2017.
-
Mixed-criticality Scheduling with Dynamic Redistribution of Shared Cache
Authors:
Muhammad Ali Awan,
Konstantinos Bletsas,
Pedro F. Souto,
Benny Akesson,
Eduardo Tovar
Abstract:
The design of mixed-criticality systems often involvespainful tradeoffs between safety guarantees and performance.However, the use of more detailed architectural modelsin the design and analysis of scheduling arrangements for mixedcriticalitysystems can provide greater confidence in the analysis,but also opportunities for better performance. Motivated by thisview, we propose an extension of Vestal…
▽ More
The design of mixed-criticality systems often involvespainful tradeoffs between safety guarantees and performance.However, the use of more detailed architectural modelsin the design and analysis of scheduling arrangements for mixedcriticalitysystems can provide greater confidence in the analysis,but also opportunities for better performance. Motivated by thisview, we propose an extension of Vestal 19s model for mixedcriticalitymulticore systems that (i) accounts for the per-taskpartitioning of the last-level cache and (ii) supports the dynamicreassignment, for better schedulability, of cache portions initiallyreserved for lower-criticality tasks to the higher-criticalitytasks, when the system switches to high-criticality mode. Tothis model, we apply partitioned EDF scheduling with Ekbergand Yi 19s deadline-scaling technique. Our schedulability analysisand scalefactor calculation is cognisant of the cache resourcesassigned to each task, by using WCET estimates that take intoaccount these resources. It is hence able to leverage the dynamicreconfiguration of the cache partitioning, at mode change, forbetter performance, in terms of provable schedulability. We alsopropose heuristics for partitioning the cache in low- and highcriticalitymode, that promote schedulability. Our experimentswith synthetic task sets, indicate tangible improvements inschedulability compared to a baseline cache-aware arrangementwhere there is no redistribution of cache resources from low- tohigh-criticality tasks in the event of a mode change.
△ Less
Submitted 28 April, 2017;
originally announced April 2017.
