[Frontline] Can AI Be “Creative”?

Kunimatsu: There are a variety of phases in manga production, including scenario writing, character creation, drawing, and frame layout. In this project, AI was used in two phases: scenario writing and character creation.

Creating the characters posed the greatest challenge for us. To develop original characters by reproducing Tezuka’s drawings, we needed to make the AI “learn” about Tezuka’s characters. But the problem was that his works rarely contained frontal headshots of the characters. We needed the data of hundreds of thousands of headshots, but we only had several thousand pictures of faces we could use.
This was why we used what is known as the transfer learning technique. Specifically, we made the AI “learn” Tezuka’s manga characters after first exposing it to hundreds of thousands of photos of human faces. This technique enabled us to create quintessential Tezuka manga characters, which led to the birth of the protagonist, PHAEDO. The AI-developed characters have the distinctive characteristics of Tezuka’s characters, while being somewhat original at the same time.

For the scenario, the concrete method we employed was to first allow the AI to study 160 volumes of Tezuka’s manga and then have it develop more than 100 plots based on what it had learned. However, only about 20% of them made sense as a story.
So, from there, we started working together with Makoto Tezuka and other manga creators at Tezuka Productions, interpreting the stories in our own ways, adding new settings, and complementing the identified flaws in the storylines. Through this process, we built an entertaining scenario. In other words, based on the manga plots developed by the AI, human creators structured the storyline.

Tokui: Having read PHAEDO, I felt that there were some gaps in the storyline. The story develops in a way that no human could imagine, which hints at the involvement of a machine. This was in fact what I found unconventional and appealing about it.
I engage in research on AI-based music production, and I present AI DJ performances in which AI and humans take turns selecting and playing one song at a time. The AI DJ project often inspires me because the songs chosen by AI are often unexpected. So my project shares something in common with the TEZUKA 2020 project.

Kunimatsu: That is so true. AI comes up with outrageous ideas that spice up the story. For example, the AI proposed in the scenario that the protagonist was an actor. This setting is not an essential part of the main storyline, but incorporating that element resulted in a unique and atypical story.

Sadoshima: Frankly, I felt it was an intriguing challenge and represented a momentous step forward in the exploration of the future of the creative industries. I also think that getting the next step right will be critically important.

Today, many manga readers enjoy manga on their smartphones. This means that the readers’ eye movements can be monitored by using the face recognition function of their smartphones, allowing us to obtain information about which frames particularly interested the readers and which ones were difficult to understand.

Osamu Tezuka was greatly interested in creating animation and live-action versions of manga. It’s no exaggeration to say that he focused greater attention than any other manga artist on providing readers with effective visual guidance.
That’s why I think that by collecting reader data to analyze how PHAEDO was enjoyed and leveraging the analysis to create the next story, we can produce manga that more closely resemble Tezuka’s works. Such an approach will serve as a stepping-stone for shaping the future of the creative industries.

Sadoshima: In the past, the definition of “creative” has changed every few centuries, but going forward, I believe this kind of shift will take place every few years.
Up to now, the most strenuous part of any creative activity was the production, or making materials. With photography, for example, it used to be quite laborious just to handle the equipment, develop the films, or apply effects, so taking photos itself presented value.
But today, cameras are simple enough for anyone to use and photos can be processed like a pro on smartphones, you know. The ability demanded of creators is shifting from “making materials” to “combining materials.”
This goes for the world of manga too. Within five years or so, I expect that anyone will be able to produce a pretty good manga by choosing pictures from a manga creation tool. Humans will only need to add the finishing touches to the selected words and pictures. It seems to me that what sets a manga creator apart from their peers will be whether they have a good eye for making revisions.

Kunimatsu: As an engineer, I think it will be quite a while before AI can replace humans in creating manga with “heart and soul.”
At this stage, AI is still just a tool for humans to use. As in the case of PHAEDO, the AI acquires the ability to produce materials by learning from large quantities of existing materials, but when it comes to combining materials, the AI is can only make random selections, which is not enough to create entertaining stories.
However precise AI becomes, humans will continue to take on the role of combining materials and giving manga their heart and soul so that people really connect with them.

Tokui: In addition to the quality of the materials and the inherent ingenuity required to combine them, the “background” is another relevant factor.
When we respond to a piece of music, it’s not necessarily just because of its beautiful melody or lyrics. Rather, the backstory, including who composed the piece and the thinking behind the composition, also contributes greatly to the depth of our feeling. While the Beatles have the story of being a group of four everyday guys from Liverpool, AI has none. Will we be able to relate to an AI-made song that sounds just like a Beatles song? I’m quite curious to know.

Sadoshima: I don’t think so. On YouTube, for example, everyone uploads things like their morning routine and meals they eat using similar formats, don’t they? It’s this format that makes the uniqueness of each piece of work stand out and tests the creators’ skills.

Tokui: One of the key elements, I believe, will be how much we can “misuse” AI. I say this because in the world of music, innovations are often rooted in the misuse of tools.
Jimi Hendrix playing the guitar with his tongue is a complete misuse of the instrument (laughs).
DJ turntables are also another example of a misused tool. When you think about it, DJs touching records with their hands is pretty wild.
AI tends to become a “black box,” possibly making it hard to trigger such misuse, which is concerning. As a creator, I’d like to explore how we can develop AI tools that offer room for some misuse.

Sadoshima: Interesting! I’ve recently been engaged in research based on the theme of “digging deep into ambiguity,” and what you’ve just explained has a lot in common with our research.
Ambiguity is crucial in the realm of creativity because human emotion itself is ambiguous, rarely describable in black-and-white terms. But generally speaking, technological evolution is based on the premise that ambiguity should be eliminated. The progress of technology has been synonymous with drawing out perfect answers as quickly as possible.
However, if machines could acquire ambiguity again at this stage, they could become better assistants to creators.

Tokui: With self-driving cars, there shouldn’t be any ambiguity (laughs). On the other hand, it would be more interesting if an AI trained to play go had some weaknesses and imperfections, rather than being flawless. Such elements of ambiguity are in demand, especially in the entertainment and creative sectors.

Kunimatsu: We have gone through many cycles of trial and error to use technology in the creative realm. KIOXIA was originally a memory device company, and we have actually utilized AI in the manufacturing process used to make semiconductors. For this application, our only goal for AI was to make it gain higher precision, but under the TEZUKA 2020 project, the key was to use technology with room for ambiguity. For the engineers, including myself, this was really challenging, and the experience broadened our horizons.
KIOXIA upholds the mission of uplifting the world with “memory.” Our hope is to capitalize on cutting-edge technology to produce not just “memory” in the sense of devices that store data, but “memories” that touch people’s hearts. The TEZUKA 2020 project is our first step in this endeavor. By contributing to the challenge of producing the first new Tezuka manga in 30 years, we have succeeded in delivering inspiration and excitement to the world. I hope to continue generating new value by using “memory” and to continue to bring excitement to the world.

The content and profile are current as of the time of the interview (September 2020).