-
Context-Aware Entity Grounding with Open-Vocabulary 3D Scene Graphs
Authors:
Haonan Chang,
Kowndinya Boyalakuntla,
Shiyang Lu,
Siwei Cai,
Eric Jing,
Shreesh Keskar,
Shijie Geng,
Adeeb Abbas,
Lifeng Zhou,
Kostas Bekris,
Abdeslam Boularias
Abstract:
We present an Open-Vocabulary 3D Scene Graph (OVSG), a formal framework for grounding a variety of entities, such as object instances, agents, and regions, with free-form text-based queries. Unlike conventional semantic-based object localization approaches, our system facilitates context-aware entity localization, allowing for queries such as ``pick up a cup on a kitchen table" or ``navigate to a…
▽ More
We present an Open-Vocabulary 3D Scene Graph (OVSG), a formal framework for grounding a variety of entities, such as object instances, agents, and regions, with free-form text-based queries. Unlike conventional semantic-based object localization approaches, our system facilitates context-aware entity localization, allowing for queries such as ``pick up a cup on a kitchen table" or ``navigate to a sofa on which someone is sitting". In contrast to existing research on 3D scene graphs, OVSG supports free-form text input and open-vocabulary querying. Through a series of comparative experiments using the ScanNet dataset and a self-collected dataset, we demonstrate that our proposed approach significantly surpasses the performance of previous semantic-based localization techniques. Moreover, we highlight the practical application of OVSG in real-world robot navigation and manipulation experiments.
△ Less
Submitted 27 September, 2023;
originally announced September 2023.
-
LGMCTS: Language-Guided Monte-Carlo Tree Search for Executable Semantic Object Rearrangement
Authors:
Haonan Chang,
Kai Gao,
Kowndinya Boyalakuntla,
Alex Lee,
Baichuan Huang,
Harish Udhaya Kumar,
Jinjin Yu,
Abdeslam Boularias
Abstract:
We introduce a novel approach to the executable semantic object rearrangement problem. In this challenge, a robot seeks to create an actionable plan that rearranges objects within a scene according to a pattern dictated by a natural language description. Unlike existing methods such as StructFormer and StructDiffusion, which tackle the issue in two steps by first generating poses and then leveragi…
▽ More
We introduce a novel approach to the executable semantic object rearrangement problem. In this challenge, a robot seeks to create an actionable plan that rearranges objects within a scene according to a pattern dictated by a natural language description. Unlike existing methods such as StructFormer and StructDiffusion, which tackle the issue in two steps by first generating poses and then leveraging a task planner for action plan formulation, our method concurrently addresses pose generation and action planning. We achieve this integration using a Language-Guided Monte-Carlo Tree Search (LGMCTS). Quantitative evaluations are provided on two simulation datasets, and complemented by qualitative tests with a real robot.
△ Less
Submitted 20 March, 2024; v1 submitted 27 September, 2023;
originally announced September 2023.
-
eGEN: An Energy-saving Modeling Language and Code Generator for Location-sensing of Mobile Apps
Authors:
Kowndinya Boyalakuntla,
Marimuthu C,
Sridhar Chimalakonda,
Chandrasekaran K
Abstract:
The demand for reducing the energy consumption of location-based applications has increased in recent years. The abnormal battery-draining behavior of GPS makes it difficult for the developers to decide on battery optimization during the development phase directly. It will reduce the burden on developers if battery-saving strategies are considered early, and relevant battery-aware code is generate…
▽ More
The demand for reducing the energy consumption of location-based applications has increased in recent years. The abnormal battery-draining behavior of GPS makes it difficult for the developers to decide on battery optimization during the development phase directly. It will reduce the burden on developers if battery-saving strategies are considered early, and relevant battery-aware code is generated from the design phase artifacts. Therefore, we aim to develop tool support, eGEN, to specify and create native location-based mobile apps. eGEN consists of Domain-specific Modeling Language (DSML) and a code generator for location-sensing. It is developed using Xtext and Xtend as an Eclipse plug-in, and currently, it supports native Android apps. eGEN is evaluated through controlled experiments by instrumenting the generated code in five location-based open-source Android applications. The experimental results show 4.35 minutes of average GPS reduction per hour and 188 mA of average reduction in battery consumption while showing only 97 meters degrade in location accuracy over 3 kilometers of a cycling path. Hence, we believe that code generated by eGEN would help developers to balance between energy and accuracy requirements of location-based applications. The source code, documentation, tool demo video, and tool installation video are available at https://github.com/Kowndinya2000/egen.
△ Less
Submitted 8 April, 2022;
originally announced April 2022.
-
WAccess -- A Web Accessibility Tool based on WCAG 2.2, 2.1 and 2.0 Guidelines
Authors:
Kowndinya Boyalakuntla,
Akhila Sri Manasa Venigalla,
Sridhar Chimalakonda
Abstract:
The vision of providing access to all web content equally for all users makes web accessibility a fundamental goal of today's internet. Web accessibility is the practice of removing barriers from websites that could hinder functionality for users with various disabilities. Web accessibility is measured against the accessibility guidelines such as WCAG, GIGW, and so on. WCAG 2.2 is the latest set o…
▽ More
The vision of providing access to all web content equally for all users makes web accessibility a fundamental goal of today's internet. Web accessibility is the practice of removing barriers from websites that could hinder functionality for users with various disabilities. Web accessibility is measured against the accessibility guidelines such as WCAG, GIGW, and so on. WCAG 2.2 is the latest set of guidelines for web accessibility that helps in making websites accessible. The web accessibility tools available in the World Wide Web Consortium (W3C), only conform up to WCAG 2.1 guidelines, while no tools exist for the latest set of guidelines. Despite the availability of several tools to check the conformity of websites with WCAG 2.1 guidelines, there is a scarcity of tools that are both open source and scalable. To support automated accessibility evaluation of numerous websites against WCAG 2.2, 2.1, and 2.0 we present a tool, WAccess. WAccess highlights violations of 13 guidelines from WCAG 2.0, 9 guidelines from WCAG 2.1, and 7 guidelines from WCAG 2.2 of a specific web page on the web console and suggests the fix for violations while specifying violating code snippet simultaneously. We evaluated WAccess against 2227 government websites of India and observed a total of about 6.1 million violations.
△ Less
Submitted 20 September, 2021; v1 submitted 14 July, 2021;
originally announced July 2021.
