Showing 1–17 of 17 results for author: Thawonmas, R

Fighting Game Adaptive Background Music for Improved Gameplay

Authors: Ibrahim Khan, Thai Van Nguyen, Chollakorn Nimpattanavong, Ruck Thawonmas

Abstract: This paper presents our work to enhance the background music (BGM) in DareFightingICE by adding adaptive features. The adaptive BGM consists of three different categories of instruments playing the BGM of the winner sound design from the 2022 DareFightingICE Competition. The BGM adapts by changing the volume of each category of instruments. Each category is connected to a different element of the… ▽ More This paper presents our work to enhance the background music (BGM) in DareFightingICE by adding adaptive features. The adaptive BGM consists of three different categories of instruments playing the BGM of the winner sound design from the 2022 DareFightingICE Competition. The BGM adapts by changing the volume of each category of instruments. Each category is connected to a different element of the game. We then run experiments to evaluate the adaptive BGM by using a deep reinforcement learning AI agent that only uses audio as input (Blind DL AI). The results show that the performance of the Blind DL AI improves while playing with the adaptive BGM as compared to playing without the adaptive BGM. △ Less

Submitted 5 March, 2024; originally announced March 2024.

Comments: This is an updated version of our IEEE CoG 2023 paper (https://ieeexplore.ieee.org/document/10333245). This version has revised the description of the association between the distance between the two players (PD) and the instrument's volume on page 2. arXiv admin note: substantial text overlap with arXiv:2303.15734

ACM Class: I.2; H.5.2; H.5

Enhanced DareFightingICE Competitions: Sound Design and AI Competitions

Authors: Ibrahim Khan, Chollakorn Nimpattanavong, Thai Van Nguyen, Kantinan Plupattanakit, Ruck Thawonmas

Abstract: This paper presents a new and improved DareFightingICE platform, a fighting game platform with a focus on visually impaired players (VIPs), in the Unity game engine. It also introduces the separation of the DareFightingICE Competition into two standalone competitions called DareFightingICE Sound Design Competition and DareFightingICE AI Competition--at the 2024 IEEE Conference on Games (CoG)--in w… ▽ More This paper presents a new and improved DareFightingICE platform, a fighting game platform with a focus on visually impaired players (VIPs), in the Unity game engine. It also introduces the separation of the DareFightingICE Competition into two standalone competitions called DareFightingICE Sound Design Competition and DareFightingICE AI Competition--at the 2024 IEEE Conference on Games (CoG)--in which a new platform will be used. This new platform is an enhanced version of the old DareFightingICE platform, having a better audio system to convey 3D sound and a better way to send audio data to AI agents. With this enhancement and by utilizing Unity, the new DareFightingICE platform is more accessible in terms of adding new features for VIPs and future audio research. This paper also improves the evaluation method for evaluating sound designs in the Sound Design Competition which will ensure a better sound design for VIPs as this competition continues to run at future CoG. To the best of our knowledge, both of our competitions are first of their kind, and the connection between the competitions to mutually improve the entries' quality with time makes these competitions an important part of representing an often overlooked segment within the broader gaming community, VIPs. △ Less

Submitted 27 April, 2024; v1 submitted 5 March, 2024; originally announced March 2024.

Comments: This paper describes a new competition platform using Unity for our competitions at the 2024 IEEE Conference on Games (CoG 2024). It was accepted for presentation at CoG 2024. However, we recently discovered a much more effective way to do this task without using Unity, leading to our decision to withdraw the paper from CoG 2024 and ArXiv

ACM Class: I.2; H.5.2; H.5.5

ChatGPT4PCG 2 Competition: Prompt Engineering for Science Birds Level Generation

Authors: Pittawat Taveekitworachai, Febri Abdullah, Mury F. Dewantoro, Yi Xia, Pratch Suntichaikul, Ruck Thawonmas, Julian Togelius, Jochen Renz

Abstract: This paper presents the second ChatGPT4PCG competition at the 2024 IEEE Conference on Games. In this edition of the competition, we follow the first edition, but make several improvements and changes. We introduce a new evaluation metric along with allowing a more flexible format for participants' submissions and making several improvements to the evaluation pipeline. Continuing from the first edi… ▽ More This paper presents the second ChatGPT4PCG competition at the 2024 IEEE Conference on Games. In this edition of the competition, we follow the first edition, but make several improvements and changes. We introduce a new evaluation metric along with allowing a more flexible format for participants' submissions and making several improvements to the evaluation pipeline. Continuing from the first edition, we aim to foster and explore the realm of prompt engineering (PE) for procedural content generation (PCG). While the first competition saw success, it was hindered by various limitations; we aim to mitigate these limitations in this edition. We introduce diversity as a new metric to discourage submissions aimed at producing repetitive structures. Furthermore, we allow submission of a Python program instead of a prompt text file for greater flexibility in implementing advanced PE approaches, which may require control flow, including conditions and iterations. We also make several improvements to the evaluation pipeline with a better classifier for similarity evaluation and better-performing function signatures. We thoroughly evaluate the effectiveness of the new metric and the improved classifier. Additionally, we perform an ablation study to select a function signature to instruct ChatGPT for level generation. Finally, we provide implementation examples of various PE techniques in Python and evaluate their preliminary performance. We hope this competition serves as a resource and platform for learning about PE and PCG in general. △ Less

Submitted 4 March, 2024; originally announced March 2024.

ACM Class: I.2.7; I.2.8

Large Language Models are Null-Shot Learners

Authors: Pittawat Taveekitworachai, Febri Abdullah, Ruck Thawonmas

Abstract: This paper presents null-shot prompting. Null-shot prompting exploits hallucination in large language models (LLMs) by instructing LLMs to utilize information from the "Examples" section that never exists within the provided context to perform a task. While reducing hallucination is crucial and non-negligible for daily and critical uses of LLMs, we propose that in the current landscape in which th… ▽ More This paper presents null-shot prompting. Null-shot prompting exploits hallucination in large language models (LLMs) by instructing LLMs to utilize information from the "Examples" section that never exists within the provided context to perform a task. While reducing hallucination is crucial and non-negligible for daily and critical uses of LLMs, we propose that in the current landscape in which these LLMs still hallucinate, it is possible, in fact, to exploit hallucination to increase performance in performing tasks compared to standard zero-shot prompting. Experiments with eight LLMs show improvements in performance across the majority of eight datasets, including reading comprehension, arithmetic reasoning, and closed-book question answering. The observed inconsistency in increased relative performance across the LLMs also potentially indicates a different degree of inherent hallucination in each model. These differences show that it is possible to utilize null-shot prompting as a way to detect degrees of hallucination in LLMs using existing benchmarking datasets. We also perform ablation studies, including experimenting with a modified version of null-shot prompting that incorporates ideas from zero-shot chain-of-thought prompting, which shows different trends of results. △ Less

Submitted 14 February, 2024; v1 submitted 16 January, 2024; originally announced January 2024.

Comments: 28 pages; added Gemini Pro results, error analysis, and a discussion on confabulation

Achieving Fairness in DareFightingICE Agents Evaluation Through a Delay Mechanism

Authors: Chollakorn Nimpattanavong, Thai Van Nguyen, Ibrahim Khan, Ruck Thawonmas, Worawat Choensawat, Kingkarn Sookhanaphibarn

Abstract: This paper proposes a delay mechanism to mitigate the impact of latency differences in the gRPC framework--a high-performance, open-source universal remote procedure call (RPC) framework--between different programming languages on the performance of agents in DareFightingICE, a fighting game research platform. The study finds that gRPC latency differences between Java and Python can significantly… ▽ More This paper proposes a delay mechanism to mitigate the impact of latency differences in the gRPC framework--a high-performance, open-source universal remote procedure call (RPC) framework--between different programming languages on the performance of agents in DareFightingICE, a fighting game research platform. The study finds that gRPC latency differences between Java and Python can significantly impact real-time decision-making. Without a delay mechanism, Java-based agents outperform Python-based ones due to lower gRPC latency on the Java platform. However, with the proposed delay mechanism, both Java-based and Python-based agents exhibit similar performance, leading to a fair comparison between agents developed using different programming languages. Thus, this work underscores the crucial importance of considering gRPC latency when developing and evaluating agents in DareFightingICE, and the insights gained could potentially extend to other gRPC-based applications. △ Less

Submitted 26 December, 2023; originally announced December 2023.

Comments: This is an updated version of our 2023 IEEE Conference on Games paper of the same title where (a) Eqn. 3 has been revised, (b) Eqn. 5 has been added for clarity, and (c) an appendix has been added that contains the link to the source code and raw data

ACM Class: C.4; H.4

Adaptive Background Music for a Fighting Game: A Multi-Instrument Volume Modulation Approach

Authors: Ibrahim Khan, Thai Van Nguyen, Chollakorn Nimpattanavong, Ruck Thawonmas

Abstract: This paper presents our work to enhance the background music (BGM) in DareFightingICE by adding an adaptive BGM. The adaptive BGM consists of five different instruments playing a classical music piece called "Air on G-String." The BGM adapts by changing the volume of the instruments. Each instrument is connected to a different element of the game. We then run experiments to evaluate the adaptive B… ▽ More This paper presents our work to enhance the background music (BGM) in DareFightingICE by adding an adaptive BGM. The adaptive BGM consists of five different instruments playing a classical music piece called "Air on G-String." The BGM adapts by changing the volume of the instruments. Each instrument is connected to a different element of the game. We then run experiments to evaluate the adaptive BGM by using a deep reinforcement learning AI that only uses audio as input (Blind DL AI). The results show that the performance of the Blind DL AI improves while playing with the adaptive BGM as compared to playing without the adaptive BGM. △ Less

Submitted 5 March, 2024; v1 submitted 28 March, 2023; originally announced March 2023.

Comments: In the updated version, the description of the association between the distance between the two players (PD) and the instrument's volume on page 3 has been revised

ACM Class: I.2; H.5.2; H.5

ChatGPT4PCG Competition: Character-like Level Generation for Science Birds

Authors: Pittawat Taveekitworachai, Febri Abdullah, Mury F. Dewantoro, Ruck Thawonmas, Julian Togelius, Jochen Renz

Abstract: This paper presents the first ChatGPT4PCG Competition at the 2023 IEEE Conference on Games. The objective of this competition is for participants to create effective prompts for ChatGPT--enabling it to generate Science Birds levels with high stability and character-like qualities--fully using their creativity as well as prompt engineering skills. ChatGPT is a conversational agent developed by Open… ▽ More This paper presents the first ChatGPT4PCG Competition at the 2023 IEEE Conference on Games. The objective of this competition is for participants to create effective prompts for ChatGPT--enabling it to generate Science Birds levels with high stability and character-like qualities--fully using their creativity as well as prompt engineering skills. ChatGPT is a conversational agent developed by OpenAI. Science Birds is selected as the competition platform because designing an Angry Birds-like level is not a trivial task due to the in-game gravity; the quality of the levels is determined by their stability. To lower the entry barrier to the competition, we limit the task to the generation of capitalized English alphabetical characters. We also allow only a single prompt to be used for generating all the characters. Here, the quality of the generated levels is determined by their stability and similarity to the given characters. A sample prompt is provided to participants for their reference. An experiment is conducted to determine the effectiveness of several modified versions of this sample prompt on level stability and similarity by testing them on several characters. To the best of our knowledge, we believe that ChatGPT4PCG is the first competition of its kind and hope to inspire enthusiasm for prompt engineering in procedural content generation. △ Less

Submitted 20 March, 2024; v1 submitted 27 March, 2023; originally announced March 2023.

Comments: This paper accepted for presentation at IEEE CoG 2023 is made available for participants of ChatGPT4PCG Competition (https://chatgpt4pcg.github.io/) and readers interested in relevant areas. In this PDF version, the affiliation symbol of Julian Togelius has been revised

ACM Class: I.2.7; I.2.8

Improving Data Transfer Efficiency for AIs in the DareFightingICE using gRPC

Authors: Chollakorn Nimpattanavong, Ibrahim Khan, Thai Van Nguyen, Ruck Thawonmas, Worawat Choensawat, Kingkarn Sookhanaphibarn

Abstract: This paper presents a new communication interface for the DareFightingICE platform, a Java-based fighting game focused on implementing AI for controlling a non-player character. The interface uses an open-source remote procedure call, gRPC to improve the efficiency of data transfer between the game and the AI, reducing the time spent on receiving information from the game server. This is important… ▽ More This paper presents a new communication interface for the DareFightingICE platform, a Java-based fighting game focused on implementing AI for controlling a non-player character. The interface uses an open-source remote procedure call, gRPC to improve the efficiency of data transfer between the game and the AI, reducing the time spent on receiving information from the game server. This is important because the main challenge of implementing AI in a fighting game is the need for the AI to select an action to perform within a short response time. The DareFightingICE platform has been integrated with Py4J, allowing developers to create AIs using Python. However, Py4J is less efficient at handling large amounts of data, resulting in excessive latency. In contrast, gRPC is well-suited for transmitting large amounts of data. To evaluate the effectiveness of the new communication interface, we conducted an experiment comparing the latency of gRPC and Py4J, using a rule-based AI that sends a kick command regardless of the information received from the game server. The experiment results showed not only a 65\% reduction in latency but also improved stability and eliminated missed frames compared to the current interface. △ Less

Submitted 11 March, 2023; originally announced March 2023.

Comments: The paper is made publically available for prospective participants of the 2023 DareFightingICE Competition. https://tinyurl.com/DareFightingICE/ It has been accepted for presentation at the 2023 8th International Conference on Business and Industrial Research. https://icbir.tni.ac.th/

ACM Class: C.4; H.4

A Deep Reinforcement Learning Blind AI in DareFightingICE

Authors: Thai Van Nguyen, Xincheng Dai, Ibrahim Khan, Ruck Thawonmas, Hai V. Pham

Abstract: This paper presents a deep reinforcement learning agent (AI) that uses sound as the input on the DareFightingICE platform at the DareFightingICE Competition in IEEE CoG 2022. In this work, an AI that only uses sound as the input is called blind AI. While state-of-the-art AIs rely mostly on visual or structured observations provided by their environments, learning to play games from only sound is s… ▽ More This paper presents a deep reinforcement learning agent (AI) that uses sound as the input on the DareFightingICE platform at the DareFightingICE Competition in IEEE CoG 2022. In this work, an AI that only uses sound as the input is called blind AI. While state-of-the-art AIs rely mostly on visual or structured observations provided by their environments, learning to play games from only sound is still new and thus challenging. We propose different approaches to process audio data and use the Proximal Policy Optimization algorithm for our blind AI. We also propose to use our blind AI in evaluation of sound designs submitted to the competition and define two metrics for this task. The experimental results show the effectiveness of not only our blind AI but also the proposed two metrics. △ Less

Submitted 30 June, 2022; v1 submitted 16 May, 2022; originally announced May 2022.

Comments: 2022 IEEE Conference on Games (CoG 2022)

ACM Class: I.2; H.5.2; H.5.5

DareFightingICE Competition: A Fighting Game Sound Design and AI Competition

Authors: Ibrahim Khan, Thai Van Nguyen, Xincheng Dai, Ruck Thawonmas

Abstract: This paper presents a new competition -- at the 2022 IEEE Conference on Games (CoG) -- called DareFightingICE Competition. The competition has two tracks: a sound design track and an AI track. The game platform for this competition is also called DareFightingICE, a fighting game platform. DareFightingICE is a sound-design-enhanced version of FightingICE, used earlier in a competition at CoG until… ▽ More This paper presents a new competition -- at the 2022 IEEE Conference on Games (CoG) -- called DareFightingICE Competition. The competition has two tracks: a sound design track and an AI track. The game platform for this competition is also called DareFightingICE, a fighting game platform. DareFightingICE is a sound-design-enhanced version of FightingICE, used earlier in a competition at CoG until 2021 to promote artificial intelligence (AI) research in fighting games. In the sound design track, participants compete for the best sound design, given the default sound design of DareFightingICE as a sample, where we define a sound design as a set of sound effects combined with the source code that implements their timing-control algorithm. Participants of the AI track are asked to develop their AI algorithm that controls a character given only sound as the input (blind AI) to fight against their opponent; a sample deep-learning blind AI will be provided by us. Our means to maximize the synergy between the two tracks are also described. This competition serves to come up with effective sound designs for visually impaired players, a group in the gaming community which has been mostly ignored. To the best of our knowledge, DareFightingICE Competition is the first of its kind within and outside of CoG. △ Less

Submitted 15 June, 2022; v1 submitted 3 March, 2022; originally announced March 2022.

Comments: 2022 IEEE Conference on Games

ACM Class: I.2; H.5.2; H.5.5

Toward a Minecraft Mod for Early Detection of Alzheimer's Disease in Young Adults

Authors: Satoko Ito, Ruck Thawonmas, Pujana Paliyawan

Abstract: This paper proposes a Minecraft-based system for early detection of Alzheimer's disease in young adults. Early detection, where spatial navigation is a crucial key, is regarded as an important way to prevent the disease. The proposed system is compared with a recent existing and thoroughly studied system using a game called Sea Hero Quest (SHQ), by analyzing spatial navigational patterns of player… ▽ More This paper proposes a Minecraft-based system for early detection of Alzheimer's disease in young adults. Early detection, where spatial navigation is a crucial key, is regarded as an important way to prevent the disease. The proposed system is compared with a recent existing and thoroughly studied system using a game called Sea Hero Quest (SHQ), by analyzing spatial navigational patterns of players. Our preliminary results show that spatial navigational patterns in both systems are highly correlated, indicating that the proposed system is likely as effective as the SHQ system for the detection task. △ Less

Submitted 25 January, 2022; originally announced January 2022.

Comments: This work was presented online as a poster presentation at the 14th ACM SIGGRAPH Conference on Motion, Interaction and Games,Nov. 10-12, 2021. A full version of this paper (in Japanese) is available on https://art-science.org/journal/v20n5/v20n5pp254/artsci-v20n5pp254.pdf. Supplementary material for the full version is on https://osf.io/u5yn8/

ACM Class: H.5.1

Impressions of the GDMC AI Settlement Generation Challenge in Minecraft

Authors: Christoph Salge, Claus Aranha, Adrian Brightmoore, Sean Butler, Rodrigo Canaan, Michael Cook, Michael Cerny Green, Hagen Fischer, Christian Guckelsberger, Jupiter Hadley, Jean-Baptiste Hervé, Mark R Johnson, Quinn Kybartas, David Mason, Mike Preuss, Tristan Smith, Ruck Thawonmas, Julian Togelius

Abstract: The GDMC AI settlement generation challenge is a PCG competition about producing an algorithm that can create an "interesting" Minecraft settlement for a given map. This paper contains a collection of written experiences with this competition, by participants, judges, organizers and advisors. We asked people to reflect both on the artifacts themselves, and on the competition in general. The aim of… ▽ More The GDMC AI settlement generation challenge is a PCG competition about producing an algorithm that can create an "interesting" Minecraft settlement for a given map. This paper contains a collection of written experiences with this competition, by participants, judges, organizers and advisors. We asked people to reflect both on the artifacts themselves, and on the competition in general. The aim of this paper is to offer a shareable and edited collection of experiences and qualitative feedback - which seem to contain a lot of insights on PCG and computational creativity, but would otherwise be lost once the output of the competition is reduced to scalar performance values. We reflect upon some organizational issues for AI competitions, and discuss the future of the GDMC competition. △ Less

Submitted 6 August, 2021; originally announced August 2021.

Comments: 28 pages, 5 figures

Towards An Angry-Birds-like Game System for Promoting Mental Well-being of Players Using Art-Therapy-embedded PCG

Authors: Zhou Fang, Pujana Paliyawan, Ruck Thawonmas, Tomohiro Harada

Abstract: This paper presents an integration of a game system and the art therapy concept for promoting the mental well-being of video game players. In the proposed game system, the player plays an Angry-Birds-like game in which levels in the game are generated based on images they draw. Upon finishing a game level, the player also receives positive feedback (praising words) toward their drawing and the gen… ▽ More This paper presents an integration of a game system and the art therapy concept for promoting the mental well-being of video game players. In the proposed game system, the player plays an Angry-Birds-like game in which levels in the game are generated based on images they draw. Upon finishing a game level, the player also receives positive feedback (praising words) toward their drawing and the generated level from an Art Therapy AI. The proposed system is composed of three major parts: (1) a drawing recognizer that identifies what object is drawn by the player (Sketcher), (2) a level generator that converts the drawing image into a pixel image, then a set of blocks representing a game level (PCG AI), and (3) the Art Therapy AI that encourages the player and improves their emotion. This paper describes an overview of the system and explains how its major components function. △ Less

Submitted 6 November, 2019; originally announced November 2019.

Comments: 2019 IEEE 8th Global Conference on Consumer Electronics

A Personalized Method for Calorie Consumption Assessment

Authors: Yunshi Liu, Pujana Paliyawan, Takahiro Kusano, Tomohiro Harada, Ruck Thawonmas

Abstract: This paper proposes an image-processing-based method for personalization of calorie consumption assessment during exercising. An experiment is carried out where several actions are required in an exercise called broadcast gymnastics, especially popular in Japan and China. We use Kinect, which captures body actions by separating the body into joints and segments that contain them, to monitor body m… ▽ More This paper proposes an image-processing-based method for personalization of calorie consumption assessment during exercising. An experiment is carried out where several actions are required in an exercise called broadcast gymnastics, especially popular in Japan and China. We use Kinect, which captures body actions by separating the body into joints and segments that contain them, to monitor body movements to test the velocity of each body joint and capture the subject's image for calculating the mass of each body joint that differs for each subject. By a kinetic energy formula, we obtain the kinetic energy of each body joint, and calories consumed during exercise are calculated in this process. We evaluate the performance of our method by benchmarking it to Fitbit, a smart watch well-known for health monitoring during exercise. The experimental results in this paper show that our method outperforms a state-of-the-art calorie assessment method, which we base on and improve, in terms of the error rate from Fitbit's ground-truth values. △ Less

Submitted 11 February, 2018; originally announced February 2018.

Comments: The AAAI 2018 Spring Symposium on Beyond Machine Intelligence: Understanding Cognitive Bias and Humanity for Well-Being, March 26-28, 2018, Stanford University, Palo Alto, California USA

Adaptive Motion Gaming AI for Health Promotion

Authors: Pujana Paliyawan, Takahiro Kusano, Yuto Nakagawa, Tomohiro Harada, Ruck Thawonmas

Abstract: This paper presents a design of a non-player character (AI) for promoting balancedness in use of body segments when engaging in full-body motion gaming. In our experiment, we settle a battle between the proposed AI and a player by using FightingICE, a fighting game platform for AI development. A middleware called UKI is used to allow the player to control the game by using body motion instead of t… ▽ More This paper presents a design of a non-player character (AI) for promoting balancedness in use of body segments when engaging in full-body motion gaming. In our experiment, we settle a battle between the proposed AI and a player by using FightingICE, a fighting game platform for AI development. A middleware called UKI is used to allow the player to control the game by using body motion instead of the keyboard and mouse. During gameplay, the proposed AI analyze health states of the player; it determines its next action by predicting how each candidate action, recommended by a Monte-Carlo tree search algorithm, will induce the player to move, and how the player's health tends to be affected. Our result demonstrates successful improvement in balancedness in use of body segments on 4 out of 5 subjects. △ Less

Submitted 4 April, 2017; originally announced April 2017.

Comments: A revised version of our paper for 2017 AAAI Spring Symposium Series (Well-Being AI: From Machine Learning to Subjective Oriented Computing), San Francisco,USA, Mar. 27-29, 2017. Revised contents, due to our correction of (8), are highlighted in red. Many apologies, but the effectiveness of the proposed method/approach in the paper still holds

ACM Class: I.2.1; K.8.0

Procedural Generation of Angry Birds Levels using Building Constructive Grammar with Chinese-Style and/or Japanese-Style Models

Authors: YuXuan Jiang, Misaki Kaidan, Chun Yin Chu, Tomohiro Harada, Ruck Thawonmas

Abstract: This paper presents a procedural generation method that creates visually attractive levels for the Angry Birds game. Besides being an immensely popular mobile game, Angry Birds has recently become a test bed for various artificial intelligence technologies. We propose a new approach for procedurally generating Angry Birds levels using Chinese style and Japanese style building structures. A conduct… ▽ More This paper presents a procedural generation method that creates visually attractive levels for the Angry Birds game. Besides being an immensely popular mobile game, Angry Birds has recently become a test bed for various artificial intelligence technologies. We propose a new approach for procedurally generating Angry Birds levels using Chinese style and Japanese style building structures. A conducted experiment confirms the effectiveness of our approach with statistical significance. △ Less

Submitted 26 April, 2016; originally announced April 2016.

ACM Class: I.2.1; J.5

Journal ref: Proc. of ASIAGRAPH 2016, Toyama, Japan, pp. 53-54, Mar. 5-6, 2016

Image Colorization Using a Deep Convolutional Neural Network

Authors: Tung Nguyen, Kazuki Mori, Ruck Thawonmas

Abstract: In this paper, we present a novel approach that uses deep learning techniques for colorizing grayscale images. By utilizing a pre-trained convolutional neural network, which is originally designed for image classification, we are able to separate content and style of different images and recombine them into a single image. We then propose a method that can add colors to a grayscale image by combin… ▽ More In this paper, we present a novel approach that uses deep learning techniques for colorizing grayscale images. By utilizing a pre-trained convolutional neural network, which is originally designed for image classification, we are able to separate content and style of different images and recombine them into a single image. We then propose a method that can add colors to a grayscale image by combining its content with style of a color image having semantic similarity with the grayscale one. As an application, to our knowledge the first of its kind, we use the proposed method to colorize images of ukiyo-e a genre of Japanese painting?and obtain interesting results, showing the potential of this method in the growing field of computer assisted art. △ Less

Submitted 26 April, 2016; originally announced April 2016.

ACM Class: I.2.6; I.4.9; J.5

Journal ref: Proc. of ASIAGRAPH 2016, Toyama, Japan, pp. 49-50, Mar. 5-6, 2016