How can you measure the creativity of generative AI models for art?

For a long time we have assumed that Talent + Skills + Resources = Creativity. AI has the ability to close the skills and resource gap required for creativity, and if the value of skills and resources is reduced... Whatever it remains, it must be talent... And talent will always shine on equal skills and resources, with AI it will shine exponentially. Having an idea and being able to transform it into true art will no longer be dependent on financial means or skills learned over time. One more thing... People with disabilities aren't able to fully explore their creativity due to the lack of tools... How would it be a comic or a movie directed by a blind person? An opera composed by a deaf person? Thanks to AI we will know it... soon!

One way to quell the fears many artists justifiably have is to involve them in measuring creativity of arts being generated by AI this will help reduce their fears of being replaced by AI as well as improve the performance of AI models towards becoming more creative because after all we as data scientists and AI professionals cannot pass a judgment as to whether the art being produced is creative or not and this true for every type of art not just paintings, poetry or novel writing, by doing this we engage a community of artists to improve the creativity of AI models and foster the perception that AI is just a tool that’s here to help artists to be more creative and not to replace them or substitute that human touch to artistic pieces

I think it's important to note that there are more ways to measure creativity than just using traditional frameworks and models. This is especially true in AI - You need to have a deep understanding of how new technologies and artistic expression work together. Because creativity has many sides, we need to look into the subtleties of originality and see how AI can not only copy but also go beyond the limits of art inspired by humans. When I look into art made by AI, I see that algorithmic processes and creative expressions are always interacting in new ways, pushing the limits of what is normally thought of as creative in the art world.

Creativity is the ability to generate ideas, solutions, or works that are both novel and valuable. It's not limited to artistic expression; it also encompasses problem-solving, innovation, and the generation of new ideas across a range of disciplines. Creativity involves a mix of divergent thinking, which is the ability to generate a wide variety of options, and convergent thinking, which is the ability to find the single, correct answer to a problem. While traditionally associated with human intelligence and cognition, the concept of creativity is increasingly being explored in the context of artificial intelligence. However, measuring creativity, whether in humans or AI, is complex and often subjective, involving criteria that can vary.

Measuring creativity is intricate, involving assessing multiple facets like individual traits, the creation process, and the social context. Various models like the 4P and 4C models and tools like the Consensual Assessment Technique and Guilford Measures offer different lenses to assess creativity, ranging from everyday problem-solving to extraordinary achievements. While psychometrics and unrelated word association may provide insights into people's creative capabilities, it's vital to acknowledge the subjectivity inherent in evaluating creativity. It's a multifaceted concept that cannot be wholly encapsulated by a single method or model.

From my perspective as an AI Engineer, I think that creativity is very important in generative AI because it creates an environment where the AI can do more than just copy and really come up with new ideas. Creativity is important not only for coming up with something new, but also for AI to be able to present something that makes people think, reflect, and feel a wide range of emotions, just like art made by humans. The goal is to have generative AI models that can improve the human experience by adding something completely new and unexpected to the conversation about art, adding new perspectives and ways of expressing themselves to our cultural tapestry.

Creativity in generative AI elevates its capabilities from mere replication to generating novel and meaningful content. In artistic fields, this allows AI to produce works that can evoke emotion and provoke thought, similar to human artistry. Beyond the arts, a creative AI can innovate in areas like science, product development, and strategy by coming up with novel solutions. Incorporating creativity also sparks important ethical and philosophical discussions about the role of AI in human endeavors. Overall, creativity enhances the practical applications and intellectual discourse surrounding generative AI.

From my perspective, measuring creativity in generative AI for art can be complex. Creativity isn't just about novelty but also value. For instance, an AI might produce a unique artwork, but if it doesn't resonate or invoke emotion, is it truly creative? One approach could be to employ a blend of quantitative metrics (like deviation from training data) and human-centric evaluations (like audience reactions or expert reviews). Real-world examples include AI-created paintings being auctioned, their value partly determined by human appreciation. The importance of creativity in generative AI lies in its ability to push boundaries, offering fresh perspectives and innovations beyond mere replication.

Creativity is essential for AI to be able to produce art that makes people think, reflect, and feel a wide range of emotions. Generative AI has the potential to revolutionize education by providing personalized learning experiences, creating engaging and interactive content, and automating administrative tasks. Few applications of generative It can create engaging and interactive learning materials, such as simulations, games, and virtual worlds. Mostly used to automate administrative tasks, such as grading assignments and providing feedback to students. In addition to these specific applications, generative AI can also be used to support research in education.

Creativity is important for generative AI because its ability to provide creative prose and imagery validates it as valued collaborator, editor and assistant.

As a natural media artist and also running a Creative studio myself that gives Creative AI solutions to our clients, I feel evaluating generative AI art creativity involves: 1. Originality: Check if works present new perspectives or merely replicate known styles. As a process within our studio no team member is allowed to use artist names in prompt. 2. Aesthetic Value: we should gauge emotional evocation. Diverse feedback helps. 3. Complexity & Technique: Assess the model's finesse in harmonizing elements and maintaining coherence. 4. Consistency & Versatility:Look for sustained quality and adaptability across themes. At the end of the day we should remember that creativity is subjective and is assessed based on various factors.

The issue is we don’t really have a measure for creativity full stop, so trying to apply one to AI might be a fools errand. Indeed, should we even try? Creativity is subjective, in all its forms.

Novelty: One of the basic measures is how novel the output is compared to existing works. Unique compositions, structures, or techniques can indicate a high level of creativity. Aesthetic Value: Subjective, but important. You can use human evaluators to rate the aesthetic appeal of the generated art. Complexity: Analyze the complexity of the generated work. Highly complex works may indicate more advanced creative capabilities. Emotional Impact: Survey human audiences to determine the emotional resonance of the artwork. Emotional engagement is often a sign of creative depth. Context Sensitivity: Assess whether the AI can generate art that is contextually relevant or makes cultural references

From my point of view, the conundrum lies in quantifying the intangible nature of creativity. A blend of human and computational evaluations might offer a solution. For instance, using algorithms to assess novelty and then crowd-sourcing feedback can gauge emotional resonance. A real-world parallel is the blend of critics' reviews and audience ratings in the movie industry. While algorithms can spot patterns and deviations, the human element evaluates the emotional and aesthetic appeal, ensuring a holistic measure of creativity.

Creativity measures are deeply subjective, by essence, as descriptions of a human process by some humans' behavior w.r.t. other humans. The frameworks used all have a human anchor (4-P: person is one of the Ps, 4-C: all about the person; BVSR: surprise is subjective; ...). Computer can calculate a "distance" between ideas/images represented by "embeddings". But the creativity metric must include "surprise" and acceptability, which must be learned from humans. This is exactly what SD models or LLMs are doing: learn some "areas of acceptability" vs "areas of gibberish". A comment above says artists could be part of the evaluation. This is in line with psychology research, stating that domain expertise is key in judging creativity.

Novelty: One of the basic measures is how novel the output is compared to existing works. Unique compositions, structures, or techniques can indicate a high level of creativity. Aesthetic Value: Subjective, but important. You can use human evaluators to rate the aesthetic appeal of the generated art. Complexity: Analyze the complexity of the generated work. Highly complex works may indicate more advanced creative capabilities. Emotional Impact: Survey human audiences to determine the emotional resonance of the artwork. Emotional engagement is often a sign of creative depth. Context Sensitivity: Assess whether the AI can generate art that is contextually relevant or makes cultural references, showing an understanding.

As you mentioned GenerativeAI applications are not limited to art or Music or short essays. For example , StyleGAN creates diverse imagery, while MuseNet generates music on all genres. GPT-3 excels in text, aiding literary arts and education. DALL-E translates text prompts to visuals, enabling unique art. GPT-4, though speculative, might offer nuanced text and multimodal capabilities, enriching narrative and cross-media artistry. Multi-model development, combining models like StyleGAN and GPT-3, can create robust systems for multisensory art. In drug discovery, Diffdock for binding pose prediction using diffusion method and denovo drug design to generate drug-like mols and many more.

All the Stable Diffusion based models to generate images from prompts. Several pretrained LLMs trained to generate text and prompted to generate prose or poetry in a given style. MLLM to generate images and text in synchronicity. And the space is wide open for creative minds to combine these generate artifacts that artists can use as inspiration combining all human senses.

Measuring the creativity of generative AI models for art is a nuanced challenge. Various approaches exist, but a single universal metric is lacking. Human evaluation involves ranking artworks based on novelty, value, surprise, and quality, albeit subjectively and with cost implications. Computational evaluation, using mathematical or statistical formulas, offers a more objective and efficient approach. Hybrid evaluation combines both, leveraging strengths and mitigating weaknesses, though it's more complex to implement. For instance, a hybrid approach might combine expert human judgment with computational metrics like diversity and uniqueness of generated art pieces.

Generative image models are still in development, but they have the potential to revolutionize how we interact with images. In addition to being able to replace (in many cases, not all) the need for a photoshoot or illustration, they can be used to generate images of objects that don't exist yet, helping inspire new products and ideas. There are several developers in this field, with the main ones being DALL-E, which I used and now realize is still lagging behind the other two main ones: Midjourney and Leonardo.ai. (see my article comparing DALL-E and Midjourney).

One of the most disturbing aspects of the first generation of text-to-image applications was how cultural stereotypes were being baked into their output. A prompt to output a "leader", "an engineer", or a "beautiful person" would almost invariably display an image of a white person. Progress is being made in this regard, especially with some models proactively pushing default diversity. Whether you like it or not, in the coming years images generated by tools like Midjourney are going to supplant traditional photos and illustrations in media. In education, children will use these tools to learn and create. In gaming we'll use these services to generate real time interactive visuals. So correcting these biases is extremely important.

Authenticity: AI-generated art often faces questions about its authenticity and originality, as it is machine-made. Quality: Ensuring the generated art meets high aesthetic standards is tricky, as AI might lack nuanced understanding of human emotions and art history. Ethics: Issues like data privacy, copyright, and authorship are still gray areas in AI-generated art. Computational Costs: High-quality generative models require significant computational power, which can be expensive. Opportunities: Mass Customization: AI can generate personalized art for individual tastes on a large scale. Accessibility: Generative AI can make art creation more accessible to people without traditional artistic skills. Innovation: AI can be more novel

Measuring the creativity of generative AI models for art can be challenging but can be done through various methods: Human Evaluation-Experts or diverse groups assess originality and artistic quality. Diversity: Check the range of diverse outputs generated. Surprise: Assess how surprising or unexpected the art is. Artistic Quality: Subjectively evaluate it against established criteria. User Engagement: Measure user interaction and impact. Constraints: Evaluate adaptability to different styles or themes. Aesthetic Consistency: Strive for a balance between novelty and coherence. Cross-Domain Creativity: Check versatility across styles. Long-Term Evolution: Monitor improvement over time with feedback and training.

Generative AI models can help artists create art more efficiently.However Generative AI models can also be biased if they are trained on biased data. This can lead to the discrimination. There is a concern that generative AI models may not be truly original, as they are based on existing data. This also raises questions about ownership of the content. there is a risk that these models could be used to create deepfakes or other forms of misleading content.Nevertheless Generative AI models have opened up new avenues for creativity and innovation

Copyrights, false information, biases, over reliance on technology. AI is by no means perfect. We should engage with it in a mindful manner, reflecting with critical thinking on the output it gives us.

Its worth noting that involving artists in measuring the creativity of AI-generated art serves multiple purposes. It addresses artists' concerns, improves AI model performance, incorporates diverse perspectives, and fosters a collaborative environment where AI and human creativity can thrive together. By recognizing AI as a tool to enhance artistic expression rather than a substitute for human touch, we can promote a harmonious relationship between AI and the art

Gen AI can be used as a collaboration tool between humans and machines. We can use these models to generate ideas and inspiration for their work, and work together with generative AI to create new forms

Want to analyze creativity of GenAI Art? Check how its prompt structure and allowed parameters. 'Chat Prompt' is your canvas to the creativity in the Generative AI world at the moment. :). It could involved what kind of input types can be given and how beautifully a person can translate an art into descriptive words.