-
Vintern-1B: An Efficient Multimodal Large Language Model for Vietnamese
Authors:
Khang T. Doan,
Bao G. Huynh,
Dung T. Hoang,
Thuc D. Pham,
Nhat H. Pham,
Quan T. M. Nguyen,
Bang Q. Vo,
Suong N. Hoang
Abstract:
In this report, we introduce Vintern-1B, a reliable 1-billion-parameters multimodal large language model (MLLM) for Vietnamese language tasks. By integrating the Qwen2-0.5B-Instruct language model with the InternViT-300M-448px visual model, Vintern-1B is optimized for a range of applications, including optical character recognition (OCR), document extraction, and general question-answering in Viet…
▽ More
In this report, we introduce Vintern-1B, a reliable 1-billion-parameters multimodal large language model (MLLM) for Vietnamese language tasks. By integrating the Qwen2-0.5B-Instruct language model with the InternViT-300M-448px visual model, Vintern-1B is optimized for a range of applications, including optical character recognition (OCR), document extraction, and general question-answering in Vietnamese context. The model is fine-tuned on an extensive dataset of over 3 million image-question-answer pairs, achieving robust performance and reliable results across multiple Vietnamese language benchmarks like OpenViVQA and ViTextVQA. Vintern-1B is small enough to fit into various on-device applications easily. Additionally, we have open-sourced several Vietnamese vision question answering (VQA) datasets for text and diagrams, created with Gemini 1.5 Flash. Our models are available at: https://huggingface.co/5CD-AI/Vintern-1B-v2.
△ Less
Submitted 23 August, 2024; v1 submitted 22 August, 2024;
originally announced August 2024.
-
Exploiting Hatred by Targets for Hate Speech Detection on Vietnamese Social Media Texts
Authors:
Cuong Nhat Vo,
Khanh Bao Huynh,
Son T. Luu,
Trong-Hop Do
Abstract:
The growth of social networks makes toxic content spread rapidly. Hate speech detection is a task to help decrease the number of harmful comments. With the diversity in the hate speech created by users, it is necessary to interpret the hate speech besides detecting it. Hence, we propose a methodology to construct a system for targeted hate speech detection from online streaming texts from social m…
▽ More
The growth of social networks makes toxic content spread rapidly. Hate speech detection is a task to help decrease the number of harmful comments. With the diversity in the hate speech created by users, it is necessary to interpret the hate speech besides detecting it. Hence, we propose a methodology to construct a system for targeted hate speech detection from online streaming texts from social media. We first introduce the ViTHSD - a targeted hate speech detection dataset for Vietnamese Social Media Texts. The dataset contains 10K comments, each comment is labeled to specific targets with three levels: clean, offensive, and hate. There are 5 targets in the dataset, and each target is labeled with the corresponding level manually by humans with strict annotation guidelines. The inter-annotator agreement obtained from the dataset is 0.45 by Cohen's Kappa index, which is indicated as a moderate level. Then, we construct a baseline for this task by combining the Bi-GRU-LSTM-CNN with the pre-trained language model to leverage the power of text representation of BERTology. Finally, we suggest a methodology to integrate the baseline model for targeted hate speech detection into the online streaming system for practical application in preventing hateful and offensive content on social media.
△ Less
Submitted 30 April, 2024;
originally announced April 2024.
-
Grasp-Anything: Large-scale Grasp Dataset from Foundation Models
Authors:
An Dinh Vuong,
Minh Nhat Vu,
Hieu Le,
Baoru Huang,
Binh Huynh,
Thieu Vo,
Andreas Kugi,
Anh Nguyen
Abstract:
Foundation models such as ChatGPT have made significant strides in robotic tasks due to their universal representation of real-world domains. In this paper, we leverage foundation models to tackle grasp detection, a persistent challenge in robotics with broad industrial applications. Despite numerous grasp datasets, their object diversity remains limited compared to real-world figures. Fortunately…
▽ More
Foundation models such as ChatGPT have made significant strides in robotic tasks due to their universal representation of real-world domains. In this paper, we leverage foundation models to tackle grasp detection, a persistent challenge in robotics with broad industrial applications. Despite numerous grasp datasets, their object diversity remains limited compared to real-world figures. Fortunately, foundation models possess an extensive repository of real-world knowledge, including objects we encounter in our daily lives. As a consequence, a promising solution to the limited representation in previous grasp datasets is to harness the universal knowledge embedded in these foundation models. We present Grasp-Anything, a new large-scale grasp dataset synthesized from foundation models to implement this solution. Grasp-Anything excels in diversity and magnitude, boasting 1M samples with text descriptions and more than 3M objects, surpassing prior datasets. Empirically, we show that Grasp-Anything successfully facilitates zero-shot grasp detection on vision-based tasks and real-world robotic experiments. Our dataset and code are available at https://grasp-anything-2023.github.io.
△ Less
Submitted 18 September, 2023;
originally announced September 2023.
-
AI for Anticipatory Action: Moving Beyond Climate Forecasting
Authors:
Benjamin Q. Huynh,
Mathew V. Kiang
Abstract:
Disaster response agencies have been shifting from a paradigm of climate forecasting towards one of anticipatory action: assessing not just what the climate will be, but how it will impact specific populations, thereby enabling proactive response and resource allocation. Machine learning models are becoming exceptionally powerful at climate forecasting, but methodological gaps remain in terms of f…
▽ More
Disaster response agencies have been shifting from a paradigm of climate forecasting towards one of anticipatory action: assessing not just what the climate will be, but how it will impact specific populations, thereby enabling proactive response and resource allocation. Machine learning models are becoming exceptionally powerful at climate forecasting, but methodological gaps remain in terms of facilitating anticipatory action. Here we provide an overview of anticipatory action, review relevant applications of machine learning, identify common challenges, and highlight areas where machine learning can uniquely contribute to advancing disaster response for populations most vulnerable to climate change.
△ Less
Submitted 28 July, 2023;
originally announced July 2023.
-
Potential for allocative harm in an environmental justice data tool
Authors:
Benjamin Q. Huynh,
Elizabeth T. Chin,
Allison Koenecke,
Derek Ouyang,
Daniel E. Ho,
Mathew V. Kiang,
David H. Rehkopf
Abstract:
Neighborhood-level screening algorithms are increasingly being deployed to inform policy decisions. We evaluate one such algorithm, CalEnviroScreen - designed to promote environmental justice and used to guide hundreds of millions of dollars in public funding annually - assessing its potential for allocative harm. We observe the model to be sensitive to subjective model decisions, with 16% of trac…
▽ More
Neighborhood-level screening algorithms are increasingly being deployed to inform policy decisions. We evaluate one such algorithm, CalEnviroScreen - designed to promote environmental justice and used to guide hundreds of millions of dollars in public funding annually - assessing its potential for allocative harm. We observe the model to be sensitive to subjective model decisions, with 16% of tracts potentially changing designation, as well as financially consequential, estimating the effect of its positive designations as a 104% (62-145%) increase in funding, equivalent to \$2.08 billion (\$1.56-2.41 billion) over four years. We also observe allocative tradeoffs and susceptibility to manipulation, raising ethical concerns. We recommend incorporating sensitivity analyses to mitigate allocative harm and accountability mechanisms to prevent misuse.
△ Less
Submitted 12 April, 2023; v1 submitted 12 April, 2023;
originally announced April 2023.
-
Using Artificial Intelligence and IoT for Constructing a Smart Trash Bin
Authors:
Khang Nhut Lam,
Nguyen Hoang Huynh,
Nguyen Bao Ngoc,
To Thi Huynh Nhu,
Nguyen Thanh Thao,
Pham Hoang Hao,
Vo Van Kiet,
Bui Xuan Huynh,
Jugal Kalita
Abstract:
The research reported in this paper transforms a normal trash bin into a smarter one by applying computer vision technology. With the support of sensors and actuator devices, the trash bin can automatically classify garbage. In particular, a camera on the trash bin takes pictures of trash, then the central processing unit analyzes and makes decisions regarding which bin to drop trash into. The acc…
▽ More
The research reported in this paper transforms a normal trash bin into a smarter one by applying computer vision technology. With the support of sensors and actuator devices, the trash bin can automatically classify garbage. In particular, a camera on the trash bin takes pictures of trash, then the central processing unit analyzes and makes decisions regarding which bin to drop trash into. The accuracy of our trash bin system achieves 90%. Besides, our model is connected to the Internet to update the bin status for further management. A mobile application is developed for managing the bin.
△ Less
Submitted 12 August, 2022;
originally announced August 2022.
-
Impact Intensity Estimation of a Quadruped Robot without Using a Force Sensor
Authors:
Ba-Phuc Huynh,
Joonbum Bae
Abstract:
Estimating the impact intensity is one of the significant tasks of the legged robot. Accurate feedback of the impact may support the robot to plan a suitable and efficient trajectory to adapt to unknown complex terrains. Ordinarily, this task is performed by a force sensor in the robot's foot. In this letter, an impact intensity estimation without using a force sensor is proposed. An artificial ne…
▽ More
Estimating the impact intensity is one of the significant tasks of the legged robot. Accurate feedback of the impact may support the robot to plan a suitable and efficient trajectory to adapt to unknown complex terrains. Ordinarily, this task is performed by a force sensor in the robot's foot. In this letter, an impact intensity estimation without using a force sensor is proposed. An artificial neural network model is designed to predict the motor torques of the legs in an instantaneous position in the trajectory without utilizing the complex kinematic and dynamic models of motion. An unscented Kalman filter is used during the trajectory to smooth and stabilize the measurement. Based on the difference between the predicted information and the filtered value, the state and intensity of the robot foot's impact with the obstacles are estimated. The simulation and experiment on a quadruped robot are carried out to verify the effectiveness of the proposed method.
△ Less
Submitted 3 April, 2022;
originally announced April 2022.
-
Ranking Feature-Block Importance in Artificial Multiblock Neural Networks
Authors:
Anna Jenul,
Stefan Schrunner,
Bao Ngoc Huynh,
Runar Helin,
Cecilia Marie Futsæther,
Kristian Hovde Liland,
Oliver Tomic
Abstract:
In artificial neural networks, understanding the contributions of input features on the prediction fosters model explainability and delivers relevant information about the dataset. While typical setups for feature importance ranking assess input features individually, in this study, we go one step further and rank the importance of groups of features, denoted as feature-blocks. A feature-block can…
▽ More
In artificial neural networks, understanding the contributions of input features on the prediction fosters model explainability and delivers relevant information about the dataset. While typical setups for feature importance ranking assess input features individually, in this study, we go one step further and rank the importance of groups of features, denoted as feature-blocks. A feature-block can contain features of a specific type or features derived from a particular source, which are presented to the neural network in separate input branches (multiblock ANNs). This work presents three methods pursuing distinct strategies to rank features in multiblock ANNs by their importance: (1) a composite strategy building on individual feature importance rankings, (2) a knock-in, and (3) a knock-out strategy. While the composite strategy builds on state-of-the-art feature importance rankings, knock-in and knock-out strategies evaluate the block as a whole via a mutual information criterion. Our experiments consist of a simulation study validating all three approaches, followed by a case study on two distinct real-world datasets to compare the strategies. We conclude that each strategy has its merits for specific application scenarios.
△ Less
Submitted 14 April, 2022; v1 submitted 21 September, 2021;
originally announced September 2021.
-
Impact of Community Structure on Consensus Machine Learning
Authors:
Bao Huynh,
Haimonti Dutta,
Dane Taylor
Abstract:
Consensus dynamics support decentralized machine learning for data that is distributed across a cloud compute cluster or across the internet of things. In these and other settings, one seeks to minimize the time $τ_ε$ required to obtain consensus within some $ε>0$ margin of error. $τ_ε$ typically depends on the topology of the underlying communication network, and for many algorithms $τ_ε$ depends…
▽ More
Consensus dynamics support decentralized machine learning for data that is distributed across a cloud compute cluster or across the internet of things. In these and other settings, one seeks to minimize the time $τ_ε$ required to obtain consensus within some $ε>0$ margin of error. $τ_ε$ typically depends on the topology of the underlying communication network, and for many algorithms $τ_ε$ depends on the second-smallest eigenvalue $λ_2\in[0,1]$ of the network's normalized Laplacian matrix: $τ_ε\sim\mathcal{O}(λ_2^{-1})$. Here, we analyze the effect on $τ_ε$ of network community structure, which can arise when compute nodes/sensors are spatially clustered, for example. We study consensus machine learning over networks drawn from stochastic block models, which yield random networks that can contain heterogeneous communities with different sizes and densities. Using random matrix theory, we analyze the effects of communities on $λ_2$ and consensus, finding that $λ_2$ generally increases (i.e., $τ_ε$ decreases) as one decreases the extent of community structure. We further observe that there exists a critical level of community structure at which $τ_ε$ reaches a lower bound and is no longer limited by the presence of communities. We support our findings with empirical experiments for decentralized support vector machines.
△ Less
Submitted 2 November, 2020;
originally announced November 2020.
-
Estimation and Feature Selection in Mixtures of Generalized Linear Experts Models
Authors:
Bao Tuyen Huynh,
Faicel Chamroukhi
Abstract:
Mixtures-of-Experts (MoE) are conditional mixture models that have shown their performance in modeling heterogeneity in data in many statistical learning approaches for prediction, including regression and classification, as well as for clustering. Their estimation in high-dimensional problems is still however challenging. We consider the problem of parameter estimation and feature selection in Mo…
▽ More
Mixtures-of-Experts (MoE) are conditional mixture models that have shown their performance in modeling heterogeneity in data in many statistical learning approaches for prediction, including regression and classification, as well as for clustering. Their estimation in high-dimensional problems is still however challenging. We consider the problem of parameter estimation and feature selection in MoE models with different generalized linear experts models, and propose a regularized maximum likelihood estimation that efficiently encourages sparse solutions for heterogeneous data with high-dimensional predictors. The developed proximal-Newton EM algorithm includes proximal Newton-type procedures to update the model parameter by monotonically maximizing the objective function and allows to perform efficient estimation and feature selection. An experimental study shows the good performance of the algorithms in terms of recovering the actual sparse solutions, parameter estimation, and clustering of heterogeneous regression data, compared to the main state-of-the art competitors.
△ Less
Submitted 14 July, 2019;
originally announced July 2019.
-
Scaling limits for random triangulations on the torus
Authors:
Vincent Beffara,
Cong Bang Huynh,
Benjamin Lévêque
Abstract:
We study the scaling limit of essentially simple triangulations on the torus. We consider, for every $n\geq 1$, a uniformly random triangulation $G_n$ over the set of (appropriately rooted) essentially simple triangulations on the torus with $n$ vertices. We view $G_n$ as a metric space by endowing its set of vertices with the graph distance denoted by $d_{G_n}$ and show that the random metric spa…
▽ More
We study the scaling limit of essentially simple triangulations on the torus. We consider, for every $n\geq 1$, a uniformly random triangulation $G_n$ over the set of (appropriately rooted) essentially simple triangulations on the torus with $n$ vertices. We view $G_n$ as a metric space by endowing its set of vertices with the graph distance denoted by $d_{G_n}$ and show that the random metric space $(V(G_n),n^{-1/4}d_{G_n})$ converges in distribution in the Gromov-Hausdorff sense when $n$ goes to infinity, at least along subsequences, toward a random metric space. One of the crucial steps in the argument is to construct a simple labeling on the map and show its convergence to an explicit scaling limit. We moreover show that this labeling approximates the distance to the root up to a uniform correction of order $o(n^{1/4})$.
△ Less
Submitted 6 May, 2019;
originally announced May 2019.
-
Hypergraph based semi-supervised learning algorithms applied to speech recognition problem: a novel approach
Authors:
Loc Hoang Tran,
Trang Hoang,
Bui Hoang Nam Huynh
Abstract:
Most network-based speech recognition methods are based on the assumption that the labels of two adjacent speech samples in the network are likely to be the same. However, assuming the pairwise relationship between speech samples is not complete. The information a group of speech samples that show very similar patterns and tend to have similar labels is missed. The natural way overcoming the infor…
▽ More
Most network-based speech recognition methods are based on the assumption that the labels of two adjacent speech samples in the network are likely to be the same. However, assuming the pairwise relationship between speech samples is not complete. The information a group of speech samples that show very similar patterns and tend to have similar labels is missed. The natural way overcoming the information loss of the above assumption is to represent the feature data of speech samples as the hypergraph. Thus, in this paper, the three un-normalized, random walk, and symmetric normalized hypergraph Laplacian based semi-supervised learning methods applied to hypergraph constructed from the feature data of speech samples in order to predict the labels of speech samples are introduced. Experiment results show that the sensitivity performance measures of these three hypergraph Laplacian based semi-supervised learning methods are greater than the sensitivity performance measures of the Hidden Markov Model method (the current state of the art method applied to speech recognition problem) and graph based semi-supervised learning methods (i.e. the current state of the art network-based method for classification problems) applied to network created from the feature data of speech samples.
△ Less
Submitted 28 October, 2018;
originally announced October 2018.
-
Regularized Maximum Likelihood Estimation and Feature Selection in Mixtures-of-Experts Models
Authors:
Faicel Chamroukhi,
Bao-Tuyen Huynh
Abstract:
Mixture of Experts (MoE) are successful models for modeling heterogeneous data in many statistical learning problems including regression, clustering and classification. Generally fitted by maximum likelihood estimation via the well-known EM algorithm, their application to high-dimensional problems is still therefore challenging. We consider the problem of fitting and feature selection in MoE mode…
▽ More
Mixture of Experts (MoE) are successful models for modeling heterogeneous data in many statistical learning problems including regression, clustering and classification. Generally fitted by maximum likelihood estimation via the well-known EM algorithm, their application to high-dimensional problems is still therefore challenging. We consider the problem of fitting and feature selection in MoE models, and propose a regularized maximum likelihood estimation approach that encourages sparse solutions for heterogeneous regression data models with potentially high-dimensional predictors. Unlike state-of-the art regularized MLE for MoE, the proposed modelings do not require an approximate of the penalty function. We develop two hybrid EM algorithms: an Expectation-Majorization-Maximization (EM/MM) algorithm, and an EM algorithm with coordinate ascent algorithm. The proposed algorithms allow to automatically obtaining sparse solutions without thresholding, and avoid matrix inversion by allowing univariate parameter updates. An experimental study shows the good performance of the algorithms in terms of recovering the actual sparse solutions, parameter estimation, and clustering of heterogeneous regression data.
△ Less
Submitted 29 October, 2018;
originally announced October 2018.
-
ARbis Pictus: A Study of Language Learning with Augmented Reality
Authors:
Adam Ibrahim,
Brandon Huynh,
Jonathan Downey,
Tobias Höllerer,
Dorothy Chun,
John O'Donovan
Abstract:
This paper describes "ARbis Pictus" --a novel system for immersive language learning through dynamic labeling of real-world objects in augmented reality. We describe a within-subjects lab-based study (N=52) that explores the effect of our system on participants learning nouns in an unfamiliar foreign language, compared to a traditional flashcard-based approach. Our results show that the immersive…
▽ More
This paper describes "ARbis Pictus" --a novel system for immersive language learning through dynamic labeling of real-world objects in augmented reality. We describe a within-subjects lab-based study (N=52) that explores the effect of our system on participants learning nouns in an unfamiliar foreign language, compared to a traditional flashcard-based approach. Our results show that the immersive experience of learning with virtual labels on real-world objects is both more effective and more enjoyable for the majority of participants, compared to flashcards. Specifically, when participants learned through augmented reality, they scored significantly better by 7% (p=0.011) on productive recall tests performed same-day, and significantly better by 21% (p=0.001) on 4-day delayed productive recall post tests than when they learned using the flashcard method. We believe this result is an indication of the strong potential for language learning in augmented reality, particularly because of the improvement shown in sustained recall compared to the traditional approach.
△ Less
Submitted 17 June, 2019; v1 submitted 30 November, 2017;
originally announced November 2017.
