-
Taming 3DGS: High-Quality Radiance Fields with Limited Resources
Authors:
Saswat Subhajyoti Mallick,
Rahul Goel,
Bernhard Kerbl,
Francisco Vicente Carrasco,
Markus Steinberger,
Fernando De La Torre
Abstract:
3D Gaussian Splatting (3DGS) has transformed novel-view synthesis with its fast, interpretable, and high-fidelity rendering. However, its resource requirements limit its usability. Especially on constrained devices, training performance degrades quickly and often cannot complete due to excessive memory consumption of the model. The method converges with an indefinite number of Gaussians -- many of…
▽ More
3D Gaussian Splatting (3DGS) has transformed novel-view synthesis with its fast, interpretable, and high-fidelity rendering. However, its resource requirements limit its usability. Especially on constrained devices, training performance degrades quickly and often cannot complete due to excessive memory consumption of the model. The method converges with an indefinite number of Gaussians -- many of them redundant -- making rendering unnecessarily slow and preventing its usage in downstream tasks that expect fixed-size inputs. To address these issues, we tackle the challenges of training and rendering 3DGS models on a budget. We use a guided, purely constructive densification process that steers densification toward Gaussians that raise the reconstruction quality. Model size continuously increases in a controlled manner towards an exact budget, using score-based densification of Gaussians with training-time priors that measure their contribution. We further address training speed obstacles: following a careful analysis of 3DGS' original pipeline, we derive faster, numerically equivalent solutions for gradient computation and attribute updates, including an alternative parallelization for efficient backpropagation. We also propose quality-preserving approximations where suitable to reduce training time even further. Taken together, these enhancements yield a robust, scalable solution with reduced training times, lower compute and memory requirements, and high quality. Our evaluation shows that in a budgeted setting, we obtain competitive quality metrics with 3DGS while achieving a 4--5x reduction in both model size and training time. With more generous budgets, our measured quality surpasses theirs. These advances open the door for novel-view synthesis in constrained environments, e.g., mobile devices.
△ Less
Submitted 21 June, 2024;
originally announced June 2024.
-
A Novel Approach to Breast Cancer Histopathological Image Classification Using Cross-Colour Space Feature Fusion and Quantum-Classical Stack Ensemble Method
Authors:
Sambit Mallick,
Snigdha Paul,
Anindya Sen
Abstract:
Breast cancer classification stands as a pivotal pillar in ensuring timely diagnosis and effective treatment. This study with histopathological images underscores the profound significance of harnessing the synergistic capabilities of colour space ensembling and quantum-classical stacking to elevate the precision of breast cancer classification. By delving into the distinct colour spaces of RGB, H…
▽ More
Breast cancer classification stands as a pivotal pillar in ensuring timely diagnosis and effective treatment. This study with histopathological images underscores the profound significance of harnessing the synergistic capabilities of colour space ensembling and quantum-classical stacking to elevate the precision of breast cancer classification. By delving into the distinct colour spaces of RGB, HSV and CIE L*u*v, the authors initiated a comprehensive investigation guided by advanced methodologies. Employing the DenseNet121 architecture for feature extraction the authors have capitalized on the robustness of Random Forest, SVM, QSVC, and VQC classifiers. This research encompasses a unique feature fusion technique within the colour space ensemble. This approach not only deepens our comprehension of breast cancer classification but also marks a milestone in personalized medical assessment. The amalgamation of quantum and classical classifiers through stacking emerges as a potent catalyst, effectively mitigating the inherent constraints of individual classifiers, paving a robust path towards more dependable and refined breast cancer identification. Through rigorous experimentation and meticulous analysis, fusion of colour spaces like RGB with HSV and RGB with CIE L*u*v, presents an classification accuracy, nearing the value of unity. This underscores the transformative potential of our approach, where the fusion of diverse colour spaces and the synergy of quantum and classical realms converge to establish a new horizon in medical diagnostics. Thus the implications of this research extend across medical disciplines, offering promising avenues for advancing diagnostic accuracy and treatment efficacy.
△ Less
Submitted 3 April, 2024;
originally announced April 2024.
-
Keeping Up With the Winner! Targeted Advertisement to Communities in Social Networks
Authors:
Shailaja Mallick,
Vishwaraj Doshi,
Do Young Eun
Abstract:
When a new product enters a market already dominated by an existing product, will it survive along with this dominant product? Most of the existing works have shown the coexistence of two competing products spreading/being adopted on overlaid graphs with same set of users. However, when it comes to the survival of a weaker product on the same graph, it has been established that the stronger one do…
▽ More
When a new product enters a market already dominated by an existing product, will it survive along with this dominant product? Most of the existing works have shown the coexistence of two competing products spreading/being adopted on overlaid graphs with same set of users. However, when it comes to the survival of a weaker product on the same graph, it has been established that the stronger one dominates the market and wipes out the other. This paper makes a step towards narrowing this gap so that a new/weaker product can also survive along with its competitor with a positive market share. Specifically, we identify a locally optimal set of users to induce a community that is targeted with advertisement by the product launching company under a given budget constraint. To this end, we model the system as competing Susceptible-Infected-Susceptible (SIS) epidemics and employ perturbation techniques to quantify and attain a positive market share in a cost-efficient manner. Our extensive simulation results with real-world graph dataset show that with our choice of target users, a new product can establish itself with positive market share, which otherwise would be dominated and eventually wiped out of the competitive market under the same budget constraint.
△ Less
Submitted 28 March, 2024;
originally announced March 2024.
-
Balancing Act: Distribution-Guided Debiasing in Diffusion Models
Authors:
Rishubh Parihar,
Abhijnya Bhat,
Abhipsa Basu,
Saswat Mallick,
Jogendra Nath Kundu,
R. Venkatesh Babu
Abstract:
Diffusion Models (DMs) have emerged as powerful generative models with unprecedented image generation capability. These models are widely used for data augmentation and creative applications. However, DMs reflect the biases present in the training datasets. This is especially concerning in the context of faces, where the DM prefers one demographic subgroup vs others (eg. female vs male). In this w…
▽ More
Diffusion Models (DMs) have emerged as powerful generative models with unprecedented image generation capability. These models are widely used for data augmentation and creative applications. However, DMs reflect the biases present in the training datasets. This is especially concerning in the context of faces, where the DM prefers one demographic subgroup vs others (eg. female vs male). In this work, we present a method for debiasing DMs without relying on additional data or model retraining. Specifically, we propose Distribution Guidance, which enforces the generated images to follow the prescribed attribute distribution. To realize this, we build on the key insight that the latent features of denoising UNet hold rich demographic semantics, and the same can be leveraged to guide debiased generation. We train Attribute Distribution Predictor (ADP) - a small mlp that maps the latent features to the distribution of attributes. ADP is trained with pseudo labels generated from existing attribute classifiers. The proposed Distribution Guidance with ADP enables us to do fair generation. Our method reduces bias across single/multiple attributes and outperforms the baseline by a significant margin for unconditional and text-conditional diffusion models. Further, we present a downstream task of training a fair attribute classifier by rebalancing the training set with our generated data.
△ Less
Submitted 29 May, 2024; v1 submitted 28 February, 2024;
originally announced February 2024.
-
Introducing SSBD+ Dataset with a Convolutional Pipeline for detecting Self-Stimulatory Behaviours in Children using raw videos
Authors:
Vaibhavi Lokegaonkar,
Vijay Jaisankar,
Pon Deepika,
Madhav Rao,
T K Srikanth,
Sarbani Mallick,
Manjit Sodhi
Abstract:
Conventionally, evaluation for the diagnosis of Autism spectrum disorder is done by a trained specialist through questionnaire-based formal assessments and by observation of behavioral cues under various settings to capture the early warning signs of autism. These evaluation techniques are highly subjective and their accuracy relies on the experience of the specialist. In this regard, machine lear…
▽ More
Conventionally, evaluation for the diagnosis of Autism spectrum disorder is done by a trained specialist through questionnaire-based formal assessments and by observation of behavioral cues under various settings to capture the early warning signs of autism. These evaluation techniques are highly subjective and their accuracy relies on the experience of the specialist. In this regard, machine learning-based methods for automated capturing of early signs of autism from the recorded videos of the children is a promising alternative. In this paper, the authors propose a novel pipelined deep learning architecture to detect certain self-stimulatory behaviors that help in the diagnosis of autism spectrum disorder (ASD). The authors also supplement their tool with an augmented version of the Self Stimulatory Behavior Dataset (SSBD) and also propose a new label in SSBD Action detection: no-class. The deep learning model with the new dataset is made freely available for easy adoption to the researchers and developers community. An overall accuracy of around 81% was achieved from the proposed pipeline model that is targeted for real-time and hands-free automated diagnosis. All of the source code, data, licenses of use, and other relevant material is made freely available in https://github.com/sarl-iiitb/
△ Less
Submitted 25 November, 2023;
originally announced November 2023.
-
CHIP: Contrastive Hierarchical Image Pretraining
Authors:
Arpit Mittal,
Harshil Jhaveri,
Swapnil Mallick,
Abhishek Ajmera
Abstract:
Few-shot object classification is the task of classifying objects in an image with limited number of examples as supervision. We propose a one-shot/few-shot classification model that can classify an object of any unseen class into a relatively general category in an hierarchically based classification. Our model uses a three-level hierarchical contrastive loss based ResNet152 classifier for classi…
▽ More
Few-shot object classification is the task of classifying objects in an image with limited number of examples as supervision. We propose a one-shot/few-shot classification model that can classify an object of any unseen class into a relatively general category in an hierarchically based classification. Our model uses a three-level hierarchical contrastive loss based ResNet152 classifier for classifying an object based on its features extracted from Image embedding, not used during the training phase. For our experimentation, we have used a subset of the ImageNet (ILSVRC-12) dataset that contains only the animal classes for training our model and created our own dataset of unseen classes for evaluating our trained model. Our model provides satisfactory results in classifying the unknown objects into a generic category which has been later discussed in greater detail.
△ Less
Submitted 12 October, 2023;
originally announced October 2023.
-
The Influence of the Other-Race Effect on Susceptibility to Face Morphing Attacks
Authors:
Snipta Mallick,
Geraldine Jeckeln,
Connor J. Parde,
Carlos D. Castillo,
Alice J. O'Toole
Abstract:
Facial morphs created between two identities resemble both of the faces used to create the morph. Consequently, humans and machines are prone to mistake morphs made from two identities for either of the faces used to create the morph. This vulnerability has been exploited in "morph attacks" in security scenarios. Here, we asked whether the "other-race effect" (ORE) -- the human advantage for ident…
▽ More
Facial morphs created between two identities resemble both of the faces used to create the morph. Consequently, humans and machines are prone to mistake morphs made from two identities for either of the faces used to create the morph. This vulnerability has been exploited in "morph attacks" in security scenarios. Here, we asked whether the "other-race effect" (ORE) -- the human advantage for identifying own- vs. other-race faces -- exacerbates morph attack susceptibility for humans. We also asked whether face-identification performance in a deep convolutional neural network (DCNN) is affected by the race of morphed faces. Caucasian (CA) and East-Asian (EA) participants performed a face-identity matching task on pairs of CA and EA face images in two conditions. In the morph condition, different-identity pairs consisted of an image of identity "A" and a 50/50 morph between images of identity "A" and "B". In the baseline condition, morphs of different identities never appeared. As expected, morphs were identified mistakenly more often than original face images. Moreover, CA participants showed an advantage for CA faces in comparison to EA faces (a partial ORE). Of primary interest, morph identification was substantially worse for cross-race faces than for own-race faces. Similar to humans, the DCNN performed more accurately for original face images than for morphed image pairs. Notably, the deep network proved substantially more accurate than humans in both cases. The results point to the possibility that DCNNs might be useful for improving face identification accuracy when morphed faces are presented. They also indicate the significance of the ORE in morph attack susceptibility in applied settings.
△ Less
Submitted 26 April, 2022;
originally announced April 2022.
-
Large Scale Distributed Linear Algebra With Tensor Processing Units
Authors:
Adam G. M. Lewis,
Jackson Beall,
Martin Ganahl,
Markus Hauru,
Shrestha Basu Mallick,
Guifre Vidal
Abstract:
We have repurposed Google Tensor Processing Units (TPUs), application-specific chips developed for machine learning, into large-scale dense linear algebra supercomputers. The TPUs' fast inter-core interconnects (ICI)s, physically two-dimensional network topology, and high-bandwidth memory (HBM) permit distributed matrix multiplication algorithms to rapidly become computationally bound. In this reg…
▽ More
We have repurposed Google Tensor Processing Units (TPUs), application-specific chips developed for machine learning, into large-scale dense linear algebra supercomputers. The TPUs' fast inter-core interconnects (ICI)s, physically two-dimensional network topology, and high-bandwidth memory (HBM) permit distributed matrix multiplication algorithms to rapidly become computationally bound. In this regime, the matrix-multiply units (MXU)s dominate the runtime, yielding impressive scaling, performance, and raw size: operating in float32 precision, a full 2048-core pod of third generation TPUs can multiply two matrices with linear size $N= 220= 1 048 576$ in about 2 minutes. Via curated algorithms emphasizing large, single-core matrix multiplications, other tasks in dense linear algebra can similarly scale. As examples, we present (i) QR decomposition; (ii) resolution of linear systems; and (iii) the computation of matrix functions by polynomial iteration, demonstrated by the matrix polar factorization.
△ Less
Submitted 16 December, 2021;
originally announced December 2021.
-
Competing Epidemics on Graphs -- Global Convergence and Coexistence
Authors:
Vishwaraj Doshi,
Shailaja Mallick,
Do Young Eun
Abstract:
The dynamics of the spread of contagions such as viruses, infectious diseases or even rumors/opinions over contact networks (graphs) have effectively been captured by the well known \textit{Susceptible-Infected-Susceptible} ($SIS$) epidemic model in recent years. When it comes to competition between two such contagions spreading on overlaid graphs, their propagation is captured by so-called \texti…
▽ More
The dynamics of the spread of contagions such as viruses, infectious diseases or even rumors/opinions over contact networks (graphs) have effectively been captured by the well known \textit{Susceptible-Infected-Susceptible} ($SIS$) epidemic model in recent years. When it comes to competition between two such contagions spreading on overlaid graphs, their propagation is captured by so-called \textit{bi-virus} epidemic models. Analysis of such dynamical systems involve the identification of equilibrium points and its convergence properties, which determine whether either of the viruses dies out, or both survive together. We demonstrate how the existing works are unsuccessful in characterizing a large subset of the model parameter space, including all parameters for which the competitiveness of the bi-virus system is significant enough to attain coexistence of the epidemics. In this paper, we fill in this void and obtain convergence results for the entirety of the model parameter space; giving precise conditions (necessary and sufficient) under which the system \textit{globally converges} to a \textit{trichotomy} of possible outcomes: a virus-free state, a single-virus state, and to a coexistence state -- the first such result.
△ Less
Submitted 22 April, 2021;
originally announced April 2021.
-
Application Behavior Enforcement Based On Network Characteristics
Authors:
Pinaki Mitra,
Girish Sundaram,
Swarup Kumar Mallick
Abstract:
Every device defines the behavior of various applications running on it with the help of user profiles or settings. What if a user wants different different applications to run in different different networks. Then he cannot do this because in todays existing operating systems the application profile, user specific tool settings and any such system usage settings do not consider the current networ…
▽ More
Every device defines the behavior of various applications running on it with the help of user profiles or settings. What if a user wants different different applications to run in different different networks. Then he cannot do this because in todays existing operating systems the application profile, user specific tool settings and any such system usage settings do not consider the current network characteristics of the user or system and are therefore static in nature.Therefore there is a need for an intelligent system which will dynamically change or apply changes to the settings of various applications running on the system considering the current network characteristics based on user specifications. This paper presents an idea such that a user can set different different applications in different different networks.This paper presents how the user will get pop up messages when he visits to safe web sites. And according to the users current network status he will get a pop up message when he will go for download or stream audio or video files.
△ Less
Submitted 15 August, 2013;
originally announced August 2013.
