-
CSUM: A Novel Mechanism for Updating CubeSat while Preserving Authenticity and Integrity
Authors:
Ankit Gangwal,
Aashish Paliwal
Abstract:
The recent rise of CubeSat has revolutionized global space explorations, as it offers cost-effective solutions for low-orbit space applications (including climate monitoring, weather measurements, communications, and earth observation). A salient feature of CubeSat is that applications currently on-boarded can either be updated or entirely replaced by new applications via software updates, which a…
▽ More
The recent rise of CubeSat has revolutionized global space explorations, as it offers cost-effective solutions for low-orbit space applications (including climate monitoring, weather measurements, communications, and earth observation). A salient feature of CubeSat is that applications currently on-boarded can either be updated or entirely replaced by new applications via software updates, which allows reusing in-orbit hardware, reduces space debris, and saves cost as well as time. Securing software updates employing traditional methods (e.g., encryption) remains impractical mainly due to the low-resource capabilities of CubeSat. Therefore, the security of software updates for CubeSats remains a critical issue.
In this paper, we propose CubeSat Update Mechanism (CSUM), a lightweight scheme to provide integrity, authentication, and data freshness guarantees for software update broadcasts to CubeSats using a hash chain. We empirically evaluate our proof of concept implementation to demonstrate the feasibility and effectiveness of our approach. CSUM can validate 50,000 consecutive updates successfully in less than a second. We also perform a comparative analysis of different cryptographic primitives. Our empirical evaluations show that the hash-based approach is at least 61$\times$ faster than the conventional mechanisms, even in resource-constrained environments. Finally, we discuss the limitations, challenges, and potential future research directions for CubeSat software update procedures.
△ Less
Submitted 30 June, 2024;
originally announced July 2024.
-
CoherentGS: Sparse Novel View Synthesis with Coherent 3D Gaussians
Authors:
Avinash Paliwal,
Wei Ye,
Jinhui Xiong,
Dmytro Kotovenko,
Rakesh Ranjan,
Vikas Chandra,
Nima Khademi Kalantari
Abstract:
The field of 3D reconstruction from images has rapidly evolved in the past few years, first with the introduction of Neural Radiance Field (NeRF) and more recently with 3D Gaussian Splatting (3DGS). The latter provides a significant edge over NeRF in terms of the training and inference speed, as well as the reconstruction quality. Although 3DGS works well for dense input images, the unstructured p…
▽ More
The field of 3D reconstruction from images has rapidly evolved in the past few years, first with the introduction of Neural Radiance Field (NeRF) and more recently with 3D Gaussian Splatting (3DGS). The latter provides a significant edge over NeRF in terms of the training and inference speed, as well as the reconstruction quality. Although 3DGS works well for dense input images, the unstructured point-cloud like representation quickly overfits to the more challenging setup of extremely sparse input images (e.g., 3 images), creating a representation that appears as a jumble of needles from novel views. To address this issue, we propose regularized optimization and depth-based initialization. Our key idea is to introduce a structured Gaussian representation that can be controlled in 2D image space. We then constraint the Gaussians, in particular their position, and prevent them from moving independently during optimization. Specifically, we introduce single and multiview constraints through an implicit convolutional decoder and a total variation loss, respectively. With the coherency introduced to the Gaussians, we further constrain the optimization through a flow-based loss function. To support our regularized optimization, we propose an approach to initialize the Gaussians using monocular depth estimates at each input view. We demonstrate significant improvements compared to the state-of-the-art sparse-view NeRF-based approaches on a variety of scenes.
△ Less
Submitted 28 March, 2024;
originally announced March 2024.
-
Jailbreaker in Jail: Moving Target Defense for Large Language Models
Authors:
Bocheng Chen,
Advait Paliwal,
Qiben Yan
Abstract:
Large language models (LLMs), known for their capability in understanding and following instructions, are vulnerable to adversarial attacks. Researchers have found that current commercial LLMs either fail to be "harmless" by presenting unethical answers, or fail to be "helpful" by refusing to offer meaningful answers when faced with adversarial queries. To strike a balance between being helpful an…
▽ More
Large language models (LLMs), known for their capability in understanding and following instructions, are vulnerable to adversarial attacks. Researchers have found that current commercial LLMs either fail to be "harmless" by presenting unethical answers, or fail to be "helpful" by refusing to offer meaningful answers when faced with adversarial queries. To strike a balance between being helpful and harmless, we design a moving target defense (MTD) enhanced LLM system. The system aims to deliver non-toxic answers that align with outputs from multiple model candidates, making them more robust against adversarial attacks. We design a query and output analysis model to filter out unsafe or non-responsive answers. %to achieve the two objectives of randomly selecting outputs from different LLMs. We evaluate over 8 most recent chatbot models with state-of-the-art adversarial queries. Our MTD-enhanced LLM system reduces the attack success rate from 37.5\% to 0\%. Meanwhile, it decreases the response refusal rate from 50\% to 0\%.
△ Less
Submitted 3 October, 2023;
originally announced October 2023.
-
De-authentication using Ambient Light Sensor
Authors:
Ankit Gangwal,
Aashish Paliwal,
Mauro Conti
Abstract:
While user authentication happens before initiating or resuming a login session, de-authentication detects the absence of a previously-authenticated user to revoke her currently active login session. The absence of proper de-authentication can lead to well-known lunchtime attacks, where a nearby adversary takes over a carelessly departed user's running login session. The existing solutions for aut…
▽ More
While user authentication happens before initiating or resuming a login session, de-authentication detects the absence of a previously-authenticated user to revoke her currently active login session. The absence of proper de-authentication can lead to well-known lunchtime attacks, where a nearby adversary takes over a carelessly departed user's running login session. The existing solutions for automatic de-authentication have distinct practical limitations, e.g., extraordinary deployment requirements or high initial cost of external equipment.
In this paper, we propose "DE-authentication using Ambient Light sensor" (DEAL), a novel, inexpensive, fast, and user-friendly de-authentication approach. DEAL utilizes the built-in ambient light sensor of a modern computer to determine if the user is leaving her work-desk. DEAL, by design, is resilient to natural shifts in lighting conditions and can be configured to handle abrupt changes in ambient illumination (e.g., due to toggling of room lights). We collected data samples from 4800 sessions with 120 volunteers in 4 typical workplace settings and conducted a series of experiments to evaluate the quality of our proposed approach thoroughly. Our results show that DEAL can de-authenticate a departing user within 4 seconds with a hit rate of 89.15% and a fall-out of 7.35%. Finally, bypassing DEAL to launch a lunchtime attack is practically infeasible as it requires the attacker to either take the user's position within a few seconds or manipulate the sensor readings sophisticatedly in real-time.
△ Less
Submitted 23 February, 2024; v1 submitted 21 September, 2023;
originally announced September 2023.
-
ReShader: View-Dependent Highlights for Single Image View-Synthesis
Authors:
Avinash Paliwal,
Brandon Nguyen,
Andrii Tsarov,
Nima Khademi Kalantari
Abstract:
In recent years, novel view synthesis from a single image has seen significant progress thanks to the rapid advancements in 3D scene representation and image inpainting techniques. While the current approaches are able to synthesize geometrically consistent novel views, they often do not handle the view-dependent effects properly. Specifically, the highlights in their synthesized images usually ap…
▽ More
In recent years, novel view synthesis from a single image has seen significant progress thanks to the rapid advancements in 3D scene representation and image inpainting techniques. While the current approaches are able to synthesize geometrically consistent novel views, they often do not handle the view-dependent effects properly. Specifically, the highlights in their synthesized images usually appear to be glued to the surfaces, making the novel views unrealistic. To address this major problem, we make a key observation that the process of synthesizing novel views requires changing the shading of the pixels based on the novel camera, and moving them to appropriate locations. Therefore, we propose to split the view synthesis process into two independent tasks of pixel reshading and relocation. During the reshading process, we take the single image as the input and adjust its shading based on the novel camera. This reshaded image is then used as the input to an existing view synthesis method to relocate the pixels and produce the final novel view image. We propose to use a neural network to perform reshading and generate a large set of synthetic input-reshaded pairs to train our network. We demonstrate that our approach produces plausible novel view images with realistic moving highlights on a variety of real world scenes.
△ Less
Submitted 3 June, 2024; v1 submitted 19 September, 2023;
originally announced September 2023.
-
Implicit View-Time Interpolation of Stereo Videos using Multi-Plane Disparities and Non-Uniform Coordinates
Authors:
Avinash Paliwal,
Andrii Tsarov,
Nima Khademi Kalantari
Abstract:
In this paper, we propose an approach for view-time interpolation of stereo videos. Specifically, we build upon X-Fields that approximates an interpolatable mapping between the input coordinates and 2D RGB images using a convolutional decoder. Our main contribution is to analyze and identify the sources of the problems with using X-Fields in our application and propose novel techniques to overcome…
▽ More
In this paper, we propose an approach for view-time interpolation of stereo videos. Specifically, we build upon X-Fields that approximates an interpolatable mapping between the input coordinates and 2D RGB images using a convolutional decoder. Our main contribution is to analyze and identify the sources of the problems with using X-Fields in our application and propose novel techniques to overcome these challenges. Specifically, we observe that X-Fields struggles to implicitly interpolate the disparities for large baseline cameras. Therefore, we propose multi-plane disparities to reduce the spatial distance of the objects in the stereo views. Moreover, we propose non-uniform time coordinates to handle the non-linear and sudden motion spikes in videos. We additionally introduce several simple, but important, improvements over X-Fields. We demonstrate that our approach is able to produce better results than the state of the art, while running in near real-time rates and having low memory and storage costs.
△ Less
Submitted 30 March, 2023;
originally announced March 2023.
-
Biomedical Multi-hop Question Answering Using Knowledge Graph Embeddings and Language Models
Authors:
Dattaraj J. Rao,
Shraddha S. Mane,
Mukta A. Paliwal
Abstract:
Biomedical knowledge graphs (KG) are heterogenous networks consisting of biological entities as nodes and relations between them as edges. These entities and relations are extracted from millions of research papers and unified in a single resource. The goal of biomedical multi-hop question-answering over knowledge graph (KGQA) is to help biologist and scientist to get valuable insights by asking q…
▽ More
Biomedical knowledge graphs (KG) are heterogenous networks consisting of biological entities as nodes and relations between them as edges. These entities and relations are extracted from millions of research papers and unified in a single resource. The goal of biomedical multi-hop question-answering over knowledge graph (KGQA) is to help biologist and scientist to get valuable insights by asking questions in natural language. Relevant answers can be found by first understanding the question and then querying the KG for right set of nodes and relationships to arrive at an answer. To model the question, language models such as RoBERTa and BioBERT are used to understand context from natural language question. One of the challenges in KGQA is missing links in the KG. Knowledge graph embeddings (KGE) help to overcome this problem by encoding nodes and edges in a dense and more efficient way. In this paper, we use a publicly available KG called Hetionet which is an integrative network of biomedical knowledge assembled from 29 different databases of genes, compounds, diseases, and more. We have enriched this KG dataset by creating a multi-hop biomedical question-answering dataset in natural language for testing the biomedical multi-hop question-answering system and this dataset will be made available to the research community. The major contribution of this research is an integrated system that combines language models with KG embeddings to give highly relevant answers to free-form questions asked by biologists in an intuitive interface. Biomedical multi-hop question-answering system is tested on this data and results are highly encouraging.
△ Less
Submitted 10 November, 2022;
originally announced November 2022.
-
Frame Interpolation for Dynamic Scenes with Implicit Flow Encoding
Authors:
Pedro Figueirêdo,
Avinash Paliwal,
Nima Khademi Kalantari
Abstract:
In this paper, we propose an algorithm to interpolate between a pair of images of a dynamic scene. While in the past years significant progress in frame interpolation has been made, current approaches are not able to handle images with brightness and illumination changes, which are common even when the images are captured shortly apart. We propose to address this problem by taking advantage of the…
▽ More
In this paper, we propose an algorithm to interpolate between a pair of images of a dynamic scene. While in the past years significant progress in frame interpolation has been made, current approaches are not able to handle images with brightness and illumination changes, which are common even when the images are captured shortly apart. We propose to address this problem by taking advantage of the existing optical flow methods that are highly robust to the variations in the illumination. Specifically, using the bidirectional flows estimated using an existing pre-trained flow network, we predict the flows from an intermediate frame to the two input images. To do this, we propose to encode the bidirectional flows into a coordinate-based network, powered by a hypernetwork, to obtain a continuous representation of the flow across time. Once we obtain the estimated flows, we use them within an existing blending network to obtain the final intermediate frame. Through extensive experiments, we demonstrate that our approach is able to produce significantly better results than state-of-the-art frame interpolation algorithms.
△ Less
Submitted 16 November, 2022; v1 submitted 27 September, 2022;
originally announced September 2022.
-
Multi-Stage Raw Video Denoising with Adversarial Loss and Gradient Mask
Authors:
Avinash Paliwal,
Libing Zeng,
Nima Khademi Kalantari
Abstract:
In this paper, we propose a learning-based approach for denoising raw videos captured under low lighting conditions. We propose to do this by first explicitly aligning the neighboring frames to the current frame using a convolutional neural network (CNN). We then fuse the registered frames using another CNN to obtain the final denoised frame. To avoid directly aligning the temporally distant frame…
▽ More
In this paper, we propose a learning-based approach for denoising raw videos captured under low lighting conditions. We propose to do this by first explicitly aligning the neighboring frames to the current frame using a convolutional neural network (CNN). We then fuse the registered frames using another CNN to obtain the final denoised frame. To avoid directly aligning the temporally distant frames, we perform the two processes of alignment and fusion in multiple stages. Specifically, at each stage, we perform the denoising process on three consecutive input frames to generate the intermediate denoised frames which are then passed as the input to the next stage. By performing the process in multiple stages, we can effectively utilize the information of neighboring frames without directly aligning the temporally distant frames. We train our multi-stage system using an adversarial loss with a conditional discriminator. Specifically, we condition the discriminator on a soft gradient mask to prevent introducing high-frequency artifacts in smooth regions. We show that our system is able to produce temporally coherent videos with realistic details. Furthermore, we demonstrate through extensive experiments that our approach outperforms state-of-the-art image and video denoising methods both numerically and visually.
△ Less
Submitted 16 April, 2021; v1 submitted 4 March, 2021;
originally announced March 2021.
-
Deep Slow Motion Video Reconstruction with Hybrid Imaging System
Authors:
Avinash Paliwal,
Nima Khademi Kalantari
Abstract:
Slow motion videos are becoming increasingly popular, but capturing high-resolution videos at extremely high frame rates requires professional high-speed cameras. To mitigate this problem, current techniques increase the frame rate of standard videos through frame interpolation by assuming linear object motion which is not valid in challenging cases. In this paper, we address this problem using tw…
▽ More
Slow motion videos are becoming increasingly popular, but capturing high-resolution videos at extremely high frame rates requires professional high-speed cameras. To mitigate this problem, current techniques increase the frame rate of standard videos through frame interpolation by assuming linear object motion which is not valid in challenging cases. In this paper, we address this problem using two video streams as input; an auxiliary video with high frame rate and low spatial resolution, providing temporal information, in addition to the standard main video with low frame rate and high spatial resolution. We propose a two-stage deep learning system consisting of alignment and appearance estimation that reconstructs high resolution slow motion video from the hybrid video input. For alignment, we propose to compute flows between the missing frame and two existing frames of the main video by utilizing the content of the auxiliary video frames. For appearance estimation, we propose to combine the warped and auxiliary frames using a context and occlusion aware network. We train our model on synthetically generated hybrid videos and show high-quality results on a variety of test scenes. To demonstrate practicality, we show the performance of our system on two real dual camera setups with small baseline.
△ Less
Submitted 21 April, 2020; v1 submitted 27 February, 2020;
originally announced February 2020.
-
Graph Representations for Higher-Order Logic and Theorem Proving
Authors:
Aditya Paliwal,
Sarah Loos,
Markus Rabe,
Kshitij Bansal,
Christian Szegedy
Abstract:
This paper presents the first use of graph neural networks (GNNs) for higher-order proof search and demonstrates that GNNs can improve upon state-of-the-art results in this domain. Interactive, higher-order theorem provers allow for the formalization of most mathematical theories and have been shown to pose a significant challenge for deep learning. Higher-order logic is highly expressive and, eve…
▽ More
This paper presents the first use of graph neural networks (GNNs) for higher-order proof search and demonstrates that GNNs can improve upon state-of-the-art results in this domain. Interactive, higher-order theorem provers allow for the formalization of most mathematical theories and have been shown to pose a significant challenge for deep learning. Higher-order logic is highly expressive and, even though it is well-structured with a clearly defined grammar and semantics, there still remains no well-established method to convert formulas into graph-based representations. In this paper, we consider several graphical representations of higher-order logic and evaluate them against the HOList benchmark for higher-order theorem proving.
△ Less
Submitted 12 September, 2019; v1 submitted 23 May, 2019;
originally announced May 2019.
-
Reinforced Genetic Algorithm Learning for Optimizing Computation Graphs
Authors:
Aditya Paliwal,
Felix Gimeno,
Vinod Nair,
Yujia Li,
Miles Lubin,
Pushmeet Kohli,
Oriol Vinyals
Abstract:
We present a deep reinforcement learning approach to minimizing the execution cost of neural network computation graphs in an optimizing compiler. Unlike earlier learning-based works that require training the optimizer on the same graph to be optimized, we propose a learning approach that trains an optimizer offline and then generalizes to previously unseen graphs without further training. This al…
▽ More
We present a deep reinforcement learning approach to minimizing the execution cost of neural network computation graphs in an optimizing compiler. Unlike earlier learning-based works that require training the optimizer on the same graph to be optimized, we propose a learning approach that trains an optimizer offline and then generalizes to previously unseen graphs without further training. This allows our approach to produce high-quality execution decisions on real-world TensorFlow graphs in seconds instead of hours. We consider two optimization tasks for computation graphs: minimizing running time and peak memory usage. In comparison to an extensive set of baselines, our approach achieves significant improvements over classical and other learning-based methods on these two tasks.
△ Less
Submitted 10 February, 2020; v1 submitted 7 May, 2019;
originally announced May 2019.
-
Distributed Approximation Algorithms for the Combinatorial Motion Planning Problem
Authors:
Simran Dokania,
Aditya Paliwal,
Shrisha Rao
Abstract:
We present a new $4$-approximation algorithm for the Combinatorial Motion Planning problem which runs in $\mathcal{O}(n^2α(n^2,n))$ time, where $α$ is the functional inverse of the Ackermann function, and a fully distributed version for the same in asynchronous message passing systems, which runs in $\mathcal{O}(n\log_2n)$ time with a message complexity of $\mathcal{O}(n^2)$. This also includes th…
▽ More
We present a new $4$-approximation algorithm for the Combinatorial Motion Planning problem which runs in $\mathcal{O}(n^2α(n^2,n))$ time, where $α$ is the functional inverse of the Ackermann function, and a fully distributed version for the same in asynchronous message passing systems, which runs in $\mathcal{O}(n\log_2n)$ time with a message complexity of $\mathcal{O}(n^2)$. This also includes the first fully distributed algorithm in asynchronous message passing systems to perform "shortcut" operations on paths, a procedure which is important in approximation algorithms for the vehicle routing problem and its variants. We also show that our algorithm gives feasible solutions to the $k$-TSP problem with an approximation factor of $2$ in both centralized and distributed environments. The broad idea of the algorithm is to distribute the set of vertices into two subsets and construct paths for each salesman over each of the two subsets. Finally we combine these pairwise disjoint paths for each salesman to obtain a set of paths that span the entire graph. This is similar to the algorithm by Yadlapalli et. al. \cite{3.66} but differs in respect to the fact that it does not require us to use minimum cost matching as a subroutine, and hence can be easily distributed.
△ Less
Submitted 23 May, 2018;
originally announced May 2018.
