-
VLM-Social-Nav: Socially Aware Robot Navigation through Scoring using Vision-Language Models
Authors:
Daeun Song,
Jing Liang,
Amirreza Payandeh,
Xuesu Xiao,
Dinesh Manocha
Abstract:
We propose VLM-Social-Nav, a novel Vision-Language Model (VLM) based navigation approach to compute a robot's motion in human-centered environments. Our goal is to make real-time decisions on robot actions that are socially compliant with human expectations. We utilize a perception model to detect important social entities and prompt a VLM to generate guidance for socially compliant robot behavior…
▽ More
We propose VLM-Social-Nav, a novel Vision-Language Model (VLM) based navigation approach to compute a robot's motion in human-centered environments. Our goal is to make real-time decisions on robot actions that are socially compliant with human expectations. We utilize a perception model to detect important social entities and prompt a VLM to generate guidance for socially compliant robot behavior. VLM-Social-Nav uses a VLM-based scoring module that computes a cost term that ensures socially appropriate and effective robot actions generated by the underlying planner. Our overall approach reduces reliance on large training datasets and enhances adaptability in decision-making. In practice, it results in improved socially compliant navigation in human-shared environments. We demonstrate and evaluate our system in four different real-world social navigation scenarios with a Turtlebot robot. We observe at least 27.38% improvement in the average success rate and 19.05% improvement in the average collision rate in the four social navigation scenarios. Our user study score shows that VLM-Social-Nav generates the most socially compliant navigation behavior.
△ Less
Submitted 7 July, 2024; v1 submitted 29 March, 2024;
originally announced April 2024.
-
DTG : Diffusion-based Trajectory Generation for Mapless Global Navigation
Authors:
Jing Liang,
Amirreza Payandeh,
Daeun Song,
Xuesu Xiao,
Dinesh Manocha
Abstract:
We present a novel end-to-end diffusion-based trajectory generation method, DTG, for mapless global navigation in challenging outdoor scenarios with occlusions and unstructured off-road features like grass, buildings, bushes, etc. Given a distant goal, our approach computes a trajectory that satisfies the following goals: (1) minimize the travel distance to the goal; (2) maximize the traversabilit…
▽ More
We present a novel end-to-end diffusion-based trajectory generation method, DTG, for mapless global navigation in challenging outdoor scenarios with occlusions and unstructured off-road features like grass, buildings, bushes, etc. Given a distant goal, our approach computes a trajectory that satisfies the following goals: (1) minimize the travel distance to the goal; (2) maximize the traversability by choosing paths that do not lie in undesirable areas. Specifically, we present a novel Conditional RNN(CRNN) for diffusion models to efficiently generate trajectories. Furthermore, we propose an adaptive training method that ensures that the diffusion model generates more traversable trajectories. We evaluate our methods in various outdoor scenes and compare the performance with other global navigation algorithms on a Husky robot. In practice, we observe at least a 15% improvement in traveling distance and around a 7% improvement in traversability.
△ Less
Submitted 24 March, 2024; v1 submitted 14 March, 2024;
originally announced March 2024.
-
VANP: Learning Where to See for Navigation with Self-Supervised Vision-Action Pre-Training
Authors:
Mohammad Nazeri,
Junzhe Wang,
Amirreza Payandeh,
Xuesu Xiao
Abstract:
Humans excel at efficiently navigating through crowds without collision by focusing on specific visual regions relevant to navigation. However, most robotic visual navigation methods rely on deep learning models pre-trained on vision tasks, which prioritize salient objects -- not necessarily relevant to navigation and potentially misleading. Alternative approaches train specialized navigation mode…
▽ More
Humans excel at efficiently navigating through crowds without collision by focusing on specific visual regions relevant to navigation. However, most robotic visual navigation methods rely on deep learning models pre-trained on vision tasks, which prioritize salient objects -- not necessarily relevant to navigation and potentially misleading. Alternative approaches train specialized navigation models from scratch, requiring significant computation. On the other hand, self-supervised learning has revolutionized computer vision and natural language processing, but its application to robotic navigation remains underexplored due to the difficulty of defining effective self-supervision signals. Motivated by these observations, in this work, we propose a Self-Supervised Vision-Action Model for Visual Navigation Pre-Training (VANP). Instead of detecting salient objects that are beneficial for tasks such as classification or detection, VANP learns to focus only on specific visual regions that are relevant to the navigation task. To achieve this, VANP uses a history of visual observations, future actions, and a goal image for self-supervision, and embeds them using two small Transformer Encoders. Then, VANP maximizes the information between the embeddings by using a mutual information maximization objective function. We demonstrate that most VANP-extracted features match with human navigation intuition. VANP achieves comparable performance as models learned end-to-end with half the training time and models trained on a large-scale, fully supervised dataset, i.e., ImageNet, with only 0.08% data.
△ Less
Submitted 29 July, 2024; v1 submitted 12 March, 2024;
originally announced March 2024.
-
Rethinking Social Robot Navigation: Leveraging the Best of Two Worlds
Authors:
Amir Hossain Raj,
Zichao Hu,
Haresh Karnan,
Rohan Chandra,
Amirreza Payandeh,
Luisa Mao,
Peter Stone,
Joydeep Biswas,
Xuesu Xiao
Abstract:
Empowering robots to navigate in a socially compliant manner is essential for the acceptance of robots moving in human-inhabited environments. Previously, roboticists have developed geometric navigation systems with decades of empirical validation to achieve safety and efficiency. However, the many complex factors of social compliance make geometric navigation systems hard to adapt to social situa…
▽ More
Empowering robots to navigate in a socially compliant manner is essential for the acceptance of robots moving in human-inhabited environments. Previously, roboticists have developed geometric navigation systems with decades of empirical validation to achieve safety and efficiency. However, the many complex factors of social compliance make geometric navigation systems hard to adapt to social situations, where no amount of tuning enables them to be both safe (people are too unpredictable) and efficient (the frozen robot problem). With recent advances in deep learning approaches, the common reaction has been to entirely discard these classical navigation systems and start from scratch, building a completely new learning-based social navigation planner. In this work, we find that this reaction is unnecessarily extreme: using a large-scale real-world social navigation dataset, SCAND, we find that geometric systems can produce trajectory plans that align with the human demonstrations in a large number of social situations. We, therefore, ask if we can rethink the social robot navigation problem by leveraging the advantages of both geometric and learning-based methods. We validate this hybrid paradigm through a proof-of-concept experiment, in which we develop a hybrid planner that switches between geometric and learning-based planning. Our experiments on both SCAND and two physical robots show that the hybrid planner can achieve better social compliance compared to using either the geometric or learning-based approach alone.
△ Less
Submitted 9 March, 2024; v1 submitted 23 September, 2023;
originally announced September 2023.
-
How susceptible are LLMs to Logical Fallacies?
Authors:
Amirreza Payandeh,
Dan Pluth,
Jordan Hosier,
Xuesu Xiao,
Vijay K. Gurbani
Abstract:
This paper investigates the rational thinking capability of Large Language Models (LLMs) in multi-round argumentative debates by exploring the impact of fallacious arguments on their logical reasoning performance. More specifically, we present Logic Competence Measurement Benchmark (LOGICOM), a diagnostic benchmark to assess the robustness of LLMs against logical fallacies. LOGICOM involves two ag…
▽ More
This paper investigates the rational thinking capability of Large Language Models (LLMs) in multi-round argumentative debates by exploring the impact of fallacious arguments on their logical reasoning performance. More specifically, we present Logic Competence Measurement Benchmark (LOGICOM), a diagnostic benchmark to assess the robustness of LLMs against logical fallacies. LOGICOM involves two agents: a persuader and a debater engaging in a multi-round debate on a controversial topic, where the persuader tries to convince the debater of the correctness of its claim. First, LOGICOM assesses the potential of LLMs to change their opinions through reasoning. Then, it evaluates the debater's performance in logical reasoning by contrasting the scenario where the persuader employs logical fallacies against one where logical reasoning is used. We use this benchmark to evaluate the performance of GPT-3.5 and GPT-4 using a dataset containing controversial topics, claims, and reasons supporting them. Our findings indicate that both GPT-3.5 and GPT-4 can adjust their opinion through reasoning. However, when presented with logical fallacies, GPT-3.5 and GPT-4 are erroneously convinced 41% and 69% more often, respectively, compared to when logical reasoning is used. Finally, we introduce a new dataset containing over 5k pairs of logical vs. fallacious arguments. The source code and dataset of this work are made publicly available.
△ Less
Submitted 18 August, 2023;
originally announced August 2023.
-
Toward Human-Like Social Robot Navigation: A Large-Scale, Multi-Modal, Social Human Navigation Dataset
Authors:
Duc M. Nguyen,
Mohammad Nazeri,
Amirreza Payandeh,
Aniket Datar,
Xuesu Xiao
Abstract:
Humans are well-adept at navigating public spaces shared with others, where current autonomous mobile robots still struggle: while safely and efficiently reaching their goals, humans communicate their intentions and conform to unwritten social norms on a daily basis; conversely, robots become clumsy in those daily social scenarios, getting stuck in dense crowds, surprising nearby pedestrians, or e…
▽ More
Humans are well-adept at navigating public spaces shared with others, where current autonomous mobile robots still struggle: while safely and efficiently reaching their goals, humans communicate their intentions and conform to unwritten social norms on a daily basis; conversely, robots become clumsy in those daily social scenarios, getting stuck in dense crowds, surprising nearby pedestrians, or even causing collisions. While recent research on robot learning has shown promises in data-driven social robot navigation, good-quality training data is still difficult to acquire through either trial and error or expert demonstrations. In this work, we propose to utilize the body of rich, widely available, social human navigation data in many natural human-inhabited public spaces for robots to learn similar, human-like, socially compliant navigation behaviors. To be specific, we design an open-source egocentric data collection sensor suite wearable by walking humans to provide multi-modal robot perception data; we collect a large-scale (~100 km, 20 hours, 300 trials, 13 humans) dataset in a variety of public spaces which contain numerous natural social navigation interactions; we analyze our dataset, demonstrate its usability, and point out future research directions and use cases.
△ Less
Submitted 9 August, 2023; v1 submitted 26 March, 2023;
originally announced March 2023.
-
Deep representation learning: Fundamentals, Perspectives, Applications, and Open Challenges
Authors:
Kourosh T. Baghaei,
Amirreza Payandeh,
Pooya Fayyazsanavi,
Shahram Rahimi,
Zhiqian Chen,
Somayeh Bakhtiari Ramezani
Abstract:
Machine Learning algorithms have had a profound impact on the field of computer science over the past few decades. These algorithms performance is greatly influenced by the representations that are derived from the data in the learning process. The representations learned in a successful learning process should be concise, discrete, meaningful, and able to be applied across a variety of tasks. A r…
▽ More
Machine Learning algorithms have had a profound impact on the field of computer science over the past few decades. These algorithms performance is greatly influenced by the representations that are derived from the data in the learning process. The representations learned in a successful learning process should be concise, discrete, meaningful, and able to be applied across a variety of tasks. A recent effort has been directed toward developing Deep Learning models, which have proven to be particularly effective at capturing high-dimensional, non-linear, and multi-modal characteristics. In this work, we discuss the principles and developments that have been made in the process of learning representations, and converting them into desirable applications. In addition, for each framework or model, the key issues and open challenges, as well as the advantages, are examined.
△ Less
Submitted 27 November, 2022;
originally announced November 2022.
-
Key management system for WSNs based on hash functions and elliptic curve cryptography
Authors:
Hamzeh Ghasemzadeh,
Ali Payandeh,
Mohammad Reza Aref
Abstract:
Due to hostile environment and wireless communication channel, security mechanisms are essential for wireless sensor networks (WSNs). Existence of a pair of shared key is a prerequisite for many of these security mechanisms; a task that key management system addresses. Recently, an energy efficient method based on public key cryptography (PKC) was proposed. We analyze this protocol and show that i…
▽ More
Due to hostile environment and wireless communication channel, security mechanisms are essential for wireless sensor networks (WSNs). Existence of a pair of shared key is a prerequisite for many of these security mechanisms; a task that key management system addresses. Recently, an energy efficient method based on public key cryptography (PKC) was proposed. We analyze this protocol and show that it is vulnerable to denial of service (DOS) attacks and adversary can exhaust memory and battery of nodes. Then, we analyze this protocol and show that using a more knowledgeable BS this vulnerability can be solved very efficiently. Based on this observation we propose a modified version of the protocol that achieves immediate authentication and can prevent DOS attacks. We show that the improved protocol achieves immediate authentication at the expense of 1.82 mj extra energy consumption while retaining other desirable characteristics of the basic method.
△ Less
Submitted 22 November, 2017;
originally announced November 2017.
-
A Hybrid DOS-Tolerant PKC-Based Key Management System for WSNs
Authors:
Hamzeh Ghasemzadeh,
Ali Payandeh,
Mohammad Reza Aref
Abstract:
Security is a critical and vital task in wireless sensor networks, therefore different key management systems have been proposed, many of which are based on symmetric cryptography. Such systems are very energy efficient, but they lack some other desirable characteristics. On the other hand, systems based on public key cryptography have those desirable characteristics, but they consume more energy.…
▽ More
Security is a critical and vital task in wireless sensor networks, therefore different key management systems have been proposed, many of which are based on symmetric cryptography. Such systems are very energy efficient, but they lack some other desirable characteristics. On the other hand, systems based on public key cryptography have those desirable characteristics, but they consume more energy. Recently based on authenticated messages from base station a new PKC based key agreement protocol was proposed. We show this method is susceptible to a form of denial of service attack where resources of the network can be exhausted with bogus messages. Then, we propose two different improvements to solve this vulnerability. Simulation results show that these new protocols retain desirable characteristics of the basic method and solve its deficiencies.
△ Less
Submitted 19 January, 2017;
originally announced January 2017.
-
Secure FSM- based arithmetic codes
Authors:
Hashem Moradmand Ziyabar,
Mahnaz Sinaie,
Ali Payandeh,
Vahid Tabataba Vakili
Abstract:
Recently, arithmetic coding has attracted the attention of many scholars because of its high compression capability. Accordingly, in this paper a method which adds secrecy to this well-known source code is proposed. Finite state arithmetic code (FSAC) is used as source code to add security. Its finite state machine (FSM) characteristic is exploited to insert some random jumps during source coding…
▽ More
Recently, arithmetic coding has attracted the attention of many scholars because of its high compression capability. Accordingly, in this paper a method which adds secrecy to this well-known source code is proposed. Finite state arithmetic code (FSAC) is used as source code to add security. Its finite state machine (FSM) characteristic is exploited to insert some random jumps during source coding process. In addition, a Huffman code is designed for each state to make decoding possible even in jumps. Being Prefix free, Huffman codes are useful in tracking correct states for an authorized user when s/he decodes with correct symmetric pseudo random key. The robustness of our proposed scheme is further reinforced by adding another extra uncertainty by swapping outputs of Huffman codes in each state. Several test images are used for inspecting the validity of the proposed Huffman Finite State Arithmetic Coding (HFSAC). The results of several experimental, key space analyses, statistical analysis, key sensitivity and plaintext sensitivity tests show that HFSAC with a little effect on compression efficiency for image cryptosystem provides an efficient and secure way for real-time image encryption and transmission.
△ Less
Submitted 5 June, 2012;
originally announced June 2012.
