-
Mimetic Poet
Authors:
Jon McCormack,
Elliott Wilson,
Nina Rajcic,
Maria Teresa Llano
Abstract:
This paper presents the design and initial assessment of a novel device that uses generative AI to facilitate creative ideation, inspiration, and reflective thought. Inspired by magnetic poetry, which was originally designed to help overcome writer's block, the device allows participants to compose short poetic texts from a limited vocabulary by physically placing words on the device's surface. Up…
▽ More
This paper presents the design and initial assessment of a novel device that uses generative AI to facilitate creative ideation, inspiration, and reflective thought. Inspired by magnetic poetry, which was originally designed to help overcome writer's block, the device allows participants to compose short poetic texts from a limited vocabulary by physically placing words on the device's surface. Upon composing the text, the system employs a large language model (LLM) to generate a response, displayed on an e-ink screen. We explored various strategies for internally sequencing prompts to foster creative thinking, including analogy, allegorical interpretations, and ideation. We installed the device in our research laboratory for two weeks and held a focus group at the conclusion to evaluate the design. The design choice to limit interactions with the LLM to poetic text, coupled with the tactile experience of assembling the poem, fostered a deeper and more enjoyable engagement with the LLM compared to traditional chatbot or screen-based interactions. This approach gives users the opportunity to reflect on the AI-generated responses in a manner conducive to creative thought.
△ Less
Submitted 3 June, 2024;
originally announced July 2024.
-
Towards A Diffractive Analysis of Prompt-Based Generative AI
Authors:
Nina Rajcic,
Maria Teresa Llano,
Jon McCormack
Abstract:
Recent developments in prompt-based generative AI has given rise to discourse surrounding the perceived ethical concerns, economic implications, and consequences for the future of cultural production. As generative imagery becomes pervasive in mainstream society, dominated primarily by emerging industry leaders, we encourage that the role of the CHI community be one of inquiry; to investigate the…
▽ More
Recent developments in prompt-based generative AI has given rise to discourse surrounding the perceived ethical concerns, economic implications, and consequences for the future of cultural production. As generative imagery becomes pervasive in mainstream society, dominated primarily by emerging industry leaders, we encourage that the role of the CHI community be one of inquiry; to investigate the numerous ways in which generative AI has the potential to, and already is, augmenting human creativity. In this paper, we conducted a diffractive analysis exploring the potential role of prompt-based interfaces in artists' creative practice. Over a two week period, seven visual artists were given access to a personalised instance of Stable Diffusion, fine-tuned on a dataset of their work. In the following diffractive analysis, we identified two dominant modes adopted by participants, AI for ideation, and AI for production. We furthermore present a number of ethical design considerations for the future development of generative AI interfaces.
△ Less
Submitted 4 March, 2024;
originally announced March 2024.
-
No Longer Trending on Artstation: Prompt Analysis of Generative AI Art
Authors:
Jon McCormack,
Maria Teresa Llano,
Stephen James Krol,
Nina Rajcic
Abstract:
Image generation using generative AI is rapidly becoming a major new source of visual media, with billions of AI generated images created using diffusion models such as Stable Diffusion and Midjourney over the last few years. In this paper we collect and analyse over 3 million prompts and the images they generate. Using natural language processing, topic analysis and visualisation methods we aim t…
▽ More
Image generation using generative AI is rapidly becoming a major new source of visual media, with billions of AI generated images created using diffusion models such as Stable Diffusion and Midjourney over the last few years. In this paper we collect and analyse over 3 million prompts and the images they generate. Using natural language processing, topic analysis and visualisation methods we aim to understand collectively how people are using text prompts, the impact of these systems on artists, and more broadly on the visual cultures they promote. Our study shows that prompting focuses largely on surface aesthetics, reinforcing cultural norms, popular conventional representations and imagery. We also find that many users focus on popular topics (such as making colouring books, fantasy art, or Christmas cards), suggesting that the dominant use for the systems analysed is recreational rather than artistic.
△ Less
Submitted 24 January, 2024;
originally announced January 2024.
-
Is Writing Prompts Really Making Art?
Authors:
Jon McCormack,
Camilo Cruz Gambardella,
Nina Rajcic,
Stephen James Krol,
Maria Teresa Llano,
Meng Yang
Abstract:
In recent years Generative Machine Learning systems have advanced significantly. A current wave of generative systems use text prompts to create complex imagery, video, even 3D datasets. The creators of these systems claim a revolution in bringing creativity and art to anyone who can type a prompt. In this position paper, we question the basis for these claims, dividing our analysis into three are…
▽ More
In recent years Generative Machine Learning systems have advanced significantly. A current wave of generative systems use text prompts to create complex imagery, video, even 3D datasets. The creators of these systems claim a revolution in bringing creativity and art to anyone who can type a prompt. In this position paper, we question the basis for these claims, dividing our analysis into three areas: the limitations of linguistic descriptions, implications of the dataset, and lastly, matters of materiality and embodiment. We conclude with an analysis of the creative possibilities enabled by prompt-based systems, asking if they can be considered a new artistic medium.
△ Less
Submitted 1 February, 2023; v1 submitted 25 January, 2023;
originally announced January 2023.
-
Message Ritual: A Posthuman Account of Living with Lamp
Authors:
Nina Rajcic,
Jon McCormack
Abstract:
As we become increasingly entangled with digital technologies, the boundary between human and machine is progressively blurring. Adopting a performative, posthumanist perspective resolves this ambiguity by proposing that such boundaries are not predetermined, rather they are enacted within a certain material configuration. Using this approach, dubbed `Entanglement HCI', this paper presents \emph{M…
▽ More
As we become increasingly entangled with digital technologies, the boundary between human and machine is progressively blurring. Adopting a performative, posthumanist perspective resolves this ambiguity by proposing that such boundaries are not predetermined, rather they are enacted within a certain material configuration. Using this approach, dubbed `Entanglement HCI', this paper presents \emph{Message Ritual} -- a novel, integrated AI system that encourages the re-framing of memory through machine generated poetics. Embodied within a domestic table lamp, the system listens in on conversations occurring within the home, drawing out key topics and phrases of the day and reconstituting them through machine generated poetry, delivered to household members via SMS upon waking each morning. Participants across four households were asked to live with the lamp over a two week period. We present a diffractive analysis exploring how the lamp \emph{becomes with} participants and discuss the implications of this method for future HCI research.
△ Less
Submitted 26 January, 2023;
originally announced January 2023.
-
Mirror Ritual: An Affective Interface for Emotional Self-Reflection
Authors:
Nina Rajcic,
Jon McCormack
Abstract:
This paper introduces a new form of real-time affective interface that engages the user in a process of conceptualisation of their emotional state. Inspired by Barrett's Theory of Constructed Emotion, `Mirror Ritual' aims to expand upon the user's accessible emotion concepts, and to ultimately provoke emotional reflection and regulation. The interface uses classified emotions -- obtained through f…
▽ More
This paper introduces a new form of real-time affective interface that engages the user in a process of conceptualisation of their emotional state. Inspired by Barrett's Theory of Constructed Emotion, `Mirror Ritual' aims to expand upon the user's accessible emotion concepts, and to ultimately provoke emotional reflection and regulation. The interface uses classified emotions -- obtained through facial expression recognition -- as a basis for dynamically generating poetry. The perceived emotion is used to seed a poetry generation system based on OpenAI's GPT-2 model, fine-tuned on a specially curated corpus. We evaluate the device's ability to foster a personalised, meaningful experience for individual users over a sustained period. A qualitative analysis revealed that participants were able to affectively engage with the mirror, with each participant developing a unique interpretation of its poetry in the context of their own emotional landscape.
△ Less
Submitted 20 April, 2020;
originally announced April 2020.
-
Mirror Ritual: Human-Machine Co-Construction of Emotion
Authors:
Nina Rajcic,
Jon McCormack
Abstract:
Mirror Ritual is an interactive installation that challenges the existing paradigms in our understanding of human emotion and machine perception. In contrast to prescriptive interfaces, the work's real-time affective interface engages the audience in the iterative conceptualisation of their emotional state through the use of affectively-charged machine generated poetry. The audience are encouraged…
▽ More
Mirror Ritual is an interactive installation that challenges the existing paradigms in our understanding of human emotion and machine perception. In contrast to prescriptive interfaces, the work's real-time affective interface engages the audience in the iterative conceptualisation of their emotional state through the use of affectively-charged machine generated poetry. The audience are encouraged to make sense of the mirror's poetry by framing it with respect to their recent life experiences, effectively `putting into words' their felt emotion. This process of affect labelling and contextualisation works to not only regulate emotion, but helps to construct the rich personal narratives that constitute human identity.
△ Less
Submitted 15 April, 2020;
originally announced April 2020.
-
Camera Obscurer: Generative Art for Design Inspiration
Authors:
Dilpreet Singh,
Nina Rajcic,
Simon Colton,
Jon McCormack
Abstract:
We investigate using generated decorative art as a source of inspiration for design tasks. Using a visual similarity search for image retrieval, the \emph{Camera Obscurer} app enables rapid searching of tens of thousands of generated abstract images of various types. The seed for a visual similarity search is a given image, and the retrieved generated images share some visual similarity with the s…
▽ More
We investigate using generated decorative art as a source of inspiration for design tasks. Using a visual similarity search for image retrieval, the \emph{Camera Obscurer} app enables rapid searching of tens of thousands of generated abstract images of various types. The seed for a visual similarity search is a given image, and the retrieved generated images share some visual similarity with the seed. Implemented in a hand-held device, the app empowers users to use photos of their surroundings to search through the archive of generated images and other image archives. Being abstract in nature, the retrieved images supplement the seed image rather than replace it, providing different visual stimuli including shapes, colours, textures and juxtapositions, in addition to affording their own interpretations. This approach can therefore be used to provide inspiration for a design task, with the abstract images suggesting new ideas that might give direction to a graphic design project. We describe a crowdsourcing experiment with the app to estimate user confidence in retrieved images, and we describe a pilot study where Camera Obscurer provided inspiration for a design task. These experiments have enabled us to describe future improvements, and to begin to understand sources of visual inspiration for design tasks.
△ Less
Submitted 5 March, 2019;
originally announced March 2019.
