-
LLM-jp: A Cross-organizational Project for the Research and Development of Fully Open Japanese LLMs
Authors:
LLM-jp,
:,
Akiko Aizawa,
Eiji Aramaki,
Bowen Chen,
Fei Cheng,
Hiroyuki Deguchi,
Rintaro Enomoto,
Kazuki Fujii,
Kensuke Fukumoto,
Takuya Fukushima,
Namgi Han,
Yuto Harada,
Chikara Hashimoto,
Tatsuya Hiraoka,
Shohei Hisada,
Sosuke Hosokawa,
Lu Jie,
Keisuke Kamata,
Teruhito Kanazawa,
Hiroki Kanezashi,
Hiroshi Kataoka,
Satoru Katsumata,
Daisuke Kawahara,
Seiya Kawano
, et al. (57 additional authors not shown)
Abstract:
This paper introduces LLM-jp, a cross-organizational project for the research and development of Japanese large language models (LLMs). LLM-jp aims to develop open-source and strong Japanese LLMs, and as of this writing, more than 1,500 participants from academia and industry are working together for this purpose. This paper presents the background of the establishment of LLM-jp, summaries of its…
▽ More
This paper introduces LLM-jp, a cross-organizational project for the research and development of Japanese large language models (LLMs). LLM-jp aims to develop open-source and strong Japanese LLMs, and as of this writing, more than 1,500 participants from academia and industry are working together for this purpose. This paper presents the background of the establishment of LLM-jp, summaries of its activities, and technical reports on the LLMs developed by LLM-jp. For the latest activities, visit https://llm-jp.nii.ac.jp/en/.
△ Less
Submitted 4 July, 2024;
originally announced July 2024.
-
Bipartite-play Dialogue Collection for Practical Automatic Evaluation of Dialogue Systems
Authors:
Shiki Sato,
Yosuke Kishinami,
Hiroaki Sugiyama,
Reina Akama,
Ryoko Tokuhisa,
Jun Suzuki
Abstract:
Automation of dialogue system evaluation is a driving force for the efficient development of dialogue systems. This paper introduces the bipartite-play method, a dialogue collection method for automating dialogue system evaluation. It addresses the limitations of existing dialogue collection methods: (i) inability to compare with systems that are not publicly available, and (ii) vulnerability to c…
▽ More
Automation of dialogue system evaluation is a driving force for the efficient development of dialogue systems. This paper introduces the bipartite-play method, a dialogue collection method for automating dialogue system evaluation. It addresses the limitations of existing dialogue collection methods: (i) inability to compare with systems that are not publicly available, and (ii) vulnerability to cheating by intentionally selecting systems to be compared. Experimental results show that the automatic evaluation using the bipartite-play method mitigates these two drawbacks and correlates as strongly with human subjectivity as existing methods.
△ Less
Submitted 19 November, 2022;
originally announced November 2022.
-
Spoken Dialogue Strategy Focusing on Asymmetric Communication with Android Robots
Authors:
Daisuke Kawakubo,
Hitoshi Ishii,
Riku Okazawa,
Shunta Nishizawa,
Haruki Hatakeyama,
Hiroaki Sugiyama,
Masaki Shuzo,
Eisaku Maeda
Abstract:
Humans are easily conscious of small differences in an android robot's (AR's) behaviors and utterances, resulting in treating the AR as not-human, while ARs treat us as humans. Thus, there exists asymmetric communication between ARs and humans. In our system at Dialogue Robot Competition 2022, this asymmetry was a considerable research target in our dialogue strategy. For example, tricky phrases s…
▽ More
Humans are easily conscious of small differences in an android robot's (AR's) behaviors and utterances, resulting in treating the AR as not-human, while ARs treat us as humans. Thus, there exists asymmetric communication between ARs and humans. In our system at Dialogue Robot Competition 2022, this asymmetry was a considerable research target in our dialogue strategy. For example, tricky phrases such as questions related to personal matters and forceful requests for agreement were experimentally used in AR's utterances. We assumed that these AR phrases would have a reasonable chance of success, although humans would likely hesitate to use the phrases. Additionally, during a five-minute dialogue, our AR's character, such as its voice tones and sentence expressions, changed from mechanical to human-like type in order to pretend to tailor to customers. The characteristics of the AR developed by our team, DSML-TDU, are introduced in this paper.
△ Less
Submitted 18 October, 2022;
originally announced October 2022.
-
Empirical Analysis of Training Strategies of Transformer-based Japanese Chit-chat Systems
Authors:
Hiroaki Sugiyama,
Masahiro Mizukami,
Tsunehiro Arimoto,
Hiromi Narimatsu,
Yuya Chiba,
Hideharu Nakajima,
Toyomi Meguro
Abstract:
In recent years, several high-performance conversational systems have been proposed based on the Transformer encoder-decoder model. Although previous studies analyzed the effects of the model parameters and the decoding method on subjective dialogue evaluations with overall metrics, they did not analyze how the differences of fine-tuning datasets affect on user's detailed impression. In addition,…
▽ More
In recent years, several high-performance conversational systems have been proposed based on the Transformer encoder-decoder model. Although previous studies analyzed the effects of the model parameters and the decoding method on subjective dialogue evaluations with overall metrics, they did not analyze how the differences of fine-tuning datasets affect on user's detailed impression. In addition, the Transformer-based approach has only been verified for English, not for such languages with large inter-language distances as Japanese. In this study, we develop large-scale Transformer-based Japanese dialogue models and Japanese chit-chat datasets to examine the effectiveness of the Transformer-based approach for building chit-chat dialogue systems. We evaluated and analyzed the impressions of human dialogues in different fine-tuning datasets, model parameters, and the use of additional information.
△ Less
Submitted 11 September, 2021;
originally announced September 2021.
-
Scalability of spin FPGA: A Reconfigurable Architecture based on spin MOSFET
Authors:
Tetsufumi Tanamoto,
Hideyuki Sugiyama,
Tomoaki Inokuchi,
Takao Marukame,
Mizue Ishikawa,
Kazutaka Ikegami,
Yoshiaki Saito
Abstract:
Scalability of Field Programmable Gate Array (FPGA) using spin MOSFET (spin FPGA) with magnetocurrent (MC) ratio in the range of 100% to 1000% is discussed for the first time. Area and speed of million-gate spin FPGA are numerically benchmarked with CMOS FPGA for 22nm, 32nm and 45nm technologies including 20% transistor size variation. We show that area is reduced and speed is increased in spin FP…
▽ More
Scalability of Field Programmable Gate Array (FPGA) using spin MOSFET (spin FPGA) with magnetocurrent (MC) ratio in the range of 100% to 1000% is discussed for the first time. Area and speed of million-gate spin FPGA are numerically benchmarked with CMOS FPGA for 22nm, 32nm and 45nm technologies including 20% transistor size variation. We show that area is reduced and speed is increased in spin FPGA owing to the nonvolatile memory function of spin MOSFET.
△ Less
Submitted 8 April, 2011;
originally announced April 2011.
