• Executive summary
• How AI is changing the media we consume
• Can AI become a Hollywood star?
• Moviemaking, made faster
• Should we let an algorithm climb the music charts?
• A threat to human artists?
• Pushing the musical boundaries
• A generational leap for gaming?
• The soaring costs of game development
• How game developers are using generative AI
• Learnings

Executive summary:

The creative industries always seemed beyond the reach of artificial intelligence (AI). In recent years, however, this long-held presumption has been challenged by the rise of generative AI. This phenomenon can be observed across virtually every kind of media, ranging from film and art to video games and even music. Sometimes it is working behind the scenes to streamline the creative process, but in many cases, it is replacing the artists themselves.

• Tools like OpenAI’s Sora and Google’s Veo can be used to create convincing, high-quality B-rolls in movies, lowering astronomical production costs.
• Gartner predicts that we will see a major blockbuster movie with 90% of its content generated by AI before 2030.
• “With the rapid evolution of these models and the underlying compute, it should be possible for a single person to create a TV show or movie in the near future”, says Lourens Swanepoel, data and AI lead at Avanade.
• A growing number of music artists are embracing AI, with Sir Paul McCartney famously using it to produce the final Beatles song.
• Game developers, including industry leaders like Ubisoft and Blizzard, are likewise increasingly turning to generative AI to streamline the game development process.

However, the growing use of generative AI within creative industries has raised serious concerns about job stability, intellectual property rights, and even the nature of creativity itself. Can something generated by an AI truly be considered a work of art? And what does it mean for human artists?

How AI is changing the media we consume

For decades, we’ve believed that certain roles—particularly those in creative industries—would remain safely in the hands of humans, immune to the growing influence of automation. The spark of imagination, emotional depth, and nuanced artistic expression are all perceived to be innately human and impossible to replicate via artificial intelligence (AI). It just makes basic sense to draw a distinction between subjective abilities and the objective skills of a computer, right?

The truth is, we might need to examine this belief. Over the last couple of years, generative AI has made enormous strides in virtually all creative industries, including art, music, video games and the written word. ChatGPT, for example, has grabbed the attention of millions with its ability to produce convincing, high-quality prose. Meanwhile, image-generation tools like DALL-E and Midjourney have proven adept at producing useable art and visual assets in a matter of seconds. 

Such developments have, inevitably, captured the attention of industry leaders, who are now exploring ways in which generative AI can be incorporated into their creative processes. However, not everybody is willing to take the prompt and get on board. Generative AI has, perhaps fairly, triggered a serious backlash from creatives concerned about job stability, their intellectual property rights and the very nature of creativity itself. So, how does this technology work, how is it currently being used, and can we mitigate its most harmful effects?

“With the rapid evolution of these models and the underlying compute, it should be possible for a single person to create a TV show or movie in the near future”.

Lourens Swanepoel, data and AI lead, Avanade

Can AI become a Hollywood star?

While filmmakers have been using AI for years, the arrival of generative AI promises to take this to a whole new level, streamlining every stage of the movie production process.

Filmmakers are no strangers to AI. Indeed, they’ve been using it for years—long before generative tools arrived on the scene. Among other things, the technology has been used to simulate large-scale battle scenes in numerous movies, with the iconic Battle of Helm’s Deep from The Lord of the Rings: The Two Towers perhaps being the most well-known. However, these early examples pale in comparison to the highly realistic content produced by today’s AI chatbots, image generators, and voice modifiers.

From assisting in scriptwriting to generating realistic visuals, generative AI has the potential to streamline every stage of the movie production process. “With the rapid evolution of these models and the underlying compute, it should be possible for a single person to create a TV show or movie in the near future”, says Lourens Swanepoel, data and AI lead at global professional services company Avanade. Gartner predicts that we will see a major blockbuster movie with 90% of its content generated by AI before 2030.

However, as AI continues to make inroads into the entertainment industry, filmmakers and audiences alike must reckon with whether the possibilities created by AI are worth the potential drawbacks. On the one hand, creators are understandably keen to streamline production processes and open up new avenues for experimentation. With the ability to generate realistic visuals and compelling narratives at a fraction of the cost and time required by traditional methods, AI could democratise the filmmaking process and allow for a greater diversity of voices and perspectives. 

On the other hand, some worry that the rapid proliferation of AI-made knock-offs could devalue the work of human artists and experts who have dedicated their professional lives to their craft. Others fear that the use of AI could be seen as a form of plagiarism due to the use of content created by other artists in the training process, leading to potentially serious legal and ethical challenges. For most industry creatives, the biggest concern of all is their job security. Proponents of the technology acknowledge that some jobs will be lost, but counter that even more new jobs will be created coaching and collaborating with the AI models.

Moviemaking, made faster

There are a number of AI tools out there that can generate convincing, film-quality footage, and some filmmakers are already using them in their work.

The development of AI tools that could transform filmmaking is picking up speed. In February 2014, OpenAI launched a new generative AI model named Sora, which is capable of generating truly cinema-esque scenes from simple text prompts or still images. Sora works in a variety of styles, from animation to black-and-white live-action, providing creators with a versatile tool for bringing their visions to life. Similarly, Google unveiled Veo, its own generative video model, at the 2024 Google I/O developer conference. Like Sora, Veo can generate high-quality videos of up to 60 seconds in response to text prompts, image inputs, and video prompts. To demonstrate the model’s capabilities, Google has partnered with the acclaimed filmmaker and artist Donald Glover (aka Childish Gambino) on a series of videos that showcase Veo’s ability to generate content in a variety of cinematic and visual styles.

So, how long will it be before we see a movie made entirely using AI? Several short films released in recent months give us a pretty good indication of the direction the industry is heading in. In March 2023, the web video production company Corridor released a short film titled Anime Rock, Paper Scissors, which was created using rotoscoping. This technique involves tracing over live-action footage frame by frame to create an animated look. Traditionally, this process has been done by human illustrators, but Corridor used Stable Diffusion to automate the task. The end product was strikingly similar to hand-drawn animation, with colourful, animated characters that nevertheless moved as if they were living actors.

In a similar vein, San Francisco-based visual artist Chad Nelson recently released Critterz, the first animated short using images generated exclusively by DALL-E. Over the course of a week, Nelson used the image generator to create thousands of photos of furry creatures and magical forests, which he then incorporated into his five-minute film. While Nelson still relied on human input for scriptwriting, voice acting, and animation, AI significantly accelerated the image creation process, condensing what might take a large staff six months to achieve into a matter of weeks. “Critterz is a vibrant example of how artists can use AI tools to unlock ideas that were once out of reach because of budget, time, or resources”, said Natalie Summers, a communications professional at OpenAI, which helped finance the film.

Should we let an algorithm climb the music charts?

Generative AI is having a dramatic impact on the music industry, allowing anyone to create original music with a simple press of a button.

Like other creative industries, the world of music is also being shaken up by generative AI. Computers have long played a crucial role in pushing the boundaries of sound and composition, but recent advances in deep learning promise to deliver a generational leap. Anyone, regardless of their musical background, can now create “original” songs at the push of a button. All they need to do is enter a text prompt, and the AI will generate a unique piece of music tailored to their specifications.

While the exact inner workings of music generation tools remain a closely guarded secret, the basic principles behind them have already been explained by OpenAI and Google. The process involves feeding the AI model a vast array of music samples coupled with written lyrics and text descriptions of the audio. This allows the AI to learn patterns, styles, and structures, enabling it to generate “original” compositions that can be strikingly similar to the music created by the artists in the training data.

This similarity has also given rise to concerns about the potential misuse of these tools. Some individuals have already exploited AI music generators to create and release tracks that mimic the style of real artists—sometimes without clearly labelling them as AI-generated. A major recent example of this are the fake diss tracks which were released during the highly-publicised feud between Drake and Kendrick Lamar. In an effort to combat this issue, several music generation tools have implemented safeguards to prevent the intentional creation of musical deepfakes, with some outright refusing to generate music that mimics the style of a real artist. Despite these measures, the AI may, unfortunately, still occasionally produce music that bears a strong resemblance to a real person’s work.

A threat to human artists?

Many musicians have voiced their concerns that the use of generative AI could threaten their livelihoods.

Another point of contention surrounding AI music generators is the use of copyrighted material in their training data. Tech companies have developed a habit of scraping data from sources without permission and failing to compensate for its use. Unsurprisingly, several lawsuits have been filed against developers of generative AIs, citing illegal use of copyrighted data. The developers, in turn, have defended their actions by claiming that the use of training data falls under the ‘fair use’ doctrine.

Many musicians have publicly expressed their discontent with the practice of using their work to train generative AIs without consent or compensation. On April 1, the Artist Right Alliance (ARA) released an open letter signed by more than 200 artists—including chart-toppers Billie Eilish and Nicki Minaj—calling on tech platforms to stop this practice. “Unethical AI use not only impacts the ability of artists to make a living, it threatens to destroy the music listening experience for all of us”, says Jen Jacobsen, executive director of the ARA. “If AI developers and online platforms choose to engage in practices that devalue human artists, we could end up limiting the quality, diversity, and accessibility of music for generations to come. Our very culture depends on getting this right”.

“Musicians are already beginning to react to AI and use AI as a way of enhancing, augmenting, their creative capabilities”.

Daren Tang, director-general, World Intellectual Property Organization (WIPO)

While it seems unlikely that AI-created music will entirely replace human musicians in the near future, it is clear that AI will play an increasingly important role in shaping our experiences with new music. For musicians, AI can serve as a powerful tool for streamlining the creative process, generating new ideas and concepts, and pushing the boundaries of their art. For listeners, AI-driven music discovery and recommendation systems will make it easier than ever to find the music they love while at the same time providing them with novel and exciting ways to experience music.

As generative music becomes more prevalent and sophisticated, there is also a risk that it could draw attention and resources away from traditional forms of musical culture, such as live performance, studio recording, and radio airplay. Just as the rise of digital streaming has disrupted the economics of the music industry and forced artists to adapt to new models of distribution and monetization, the proliferation of AI-driven music creation could have far-reaching consequences for the way we create, consume, and value music as a society.

Pushing the musical boundaries

Despite objections from their colleagues, a growing number of artists are experimenting with AI tools.

One of the most prominent music generators currently available to the general market is Udio. Developed by former Google DeepMind engineers, the app allows users to create as many as 1,200 songs per month for free by entering a simple text prompt. When a user opens the app, they are presented with a text box, where they need to describe what kind of song they are looking to create and then choose one or more genre and mood tags suggested by the app. They can also enter their own lyrics or have the app generate them automatically.

Once they click on the ‘Create’ button, the app will proceed to generate two 32-second-long songs within five minutes. Users can then download the song they like, share it with their friends, or publish it online. In case they don’t like any of the two songs, they can remix them by adding additional sections until they get a result they are satisfied with. “Udio enables everyone from classically trained musicians to those with pop star ambitions to hip-hop fans to people who just want to have fun with their friends to create awe-inspiring songs in mere moments”, says David Ding, co-founder and CEO of Udio. Besides Udio, there are a number of other tools that offer similar features and functionalities. Some of the most notable names include Suno, Stability AI’s Stable Audio, Google’s MusicLM, and Meta’s Audiocraft.

While some artists view this technology as a threat to their livelihoods, others are embracing it as a powerful tool for exploring new avenues of expression. “Musicians are already beginning to react to AI and use it as a way of enhancing, augmenting, their creative capabilities”, says Daren Tang, director-general of the World Intellectual Property Organization (WIPO). For example, pop artist Grimes has launched a website where people can use a clone of her voice to create and distribute their own songs, with any royalties made from the songs split between the two parties.

It’s important to bear in mind that AI’s utility extends beyond mere composition. In November 2023, Sir Paul McCartney made headlines when he announced that he was releasing the final Beatles song, which was made by using an AI to extract John Lennon’s voice from an old demo. Perhaps understandably, this has led to a serious backlash from angry fans, prompting McCartney to issue a statement clarifying that AI wasn’t actually used to generate any new vocals but to clean up some existing ones. “We’ve seen some confusion and speculation about it”, he said. “Can’t say too much at this stage, but to be clear, nothing has been artificially or synthetically created. It’s all real, and we all play on it. We cleaned up some existing recordings—a process which has gone on for years”.

“Artists, writers and coders will always be at the heart of the development process, and while AI can now do a better job at assisting creators in their workflow, it’s the artistic vision and perspective of individuals that are essential in the creation of games”.

Yves Jacquier, executive director, Ubisoft La Forge

A generational leap for gaming?

Although video games have always incorporated some form of AI, generative AI has the potential to revolutionise the way games are designed, developed, and experienced.

Perhaps more than any other creative medium, generative AI is expected to have a significant impact on gaming. In fact, some major studios have already announced that their upcoming titles will include AI-driven experiences. Meanwhile, a range of startups are exploring ways to reimagine the game design process with new AI-powered development tools.

However, it’s worth noting that video games have always incorporated some form of artificial intelligence. Indeed, AI has been used to put computer-controlled opponents in motion for decades. These AI entities were not “intelligent” in the modern sense, following predetermined scripts and patterns designed by the game’s programmers. While they provided a level of challenge and interactivity, they also lacked the ability to learn or adapt.

As gaming hardware became more sophisticated and computing power increased, developers began to experiment with more advanced AI techniques, such as finite state machines, decision trees, and pathfinding algorithms. These methods allowed for more complex non-player character (NPC) behaviours, such as the dynamic reactions of enemies in stealth games or the strategic decision-making of computer-controlled players in strategy titles. However, these AI systems were still largely based on predefined rules and heuristics, limiting their capacity for truly lateral decision-making.

The arrival of machine learning and deep neural networks marks a dramatic turning point in how AI is used in the gaming industry. Generative AI, in particular, has the potential to revolutionise the way games are designed, developed, and experienced. Unlike traditional AI methods, which rely on explicit programming and rule-based systems, generative AI leverages the power of large neural networks to identify patterns, learn from vast amounts of data, and generate novel content.

The soaring costs of game development

The costs of game development have skyrocketed in recent years, often exceeding hundreds of millions of dollars.

One of the key advantages of generative AI is its ability to dramatically reduce the time and cost associated with game development. The astronomical costs associated with developing modern video games have put strain on even the most well-established developers and publishers. One of the most notable examples of this trend is Rockstar Games’ Red Dead Redemption 2, which took nearly 8 years to build and reportedly cost more than $500 million, making it one of the most expensive media works ever created.

By training on existing game assets, such as concept art and level designs, AI can assist developers in rapidly prototyping and iterating new ideas. This not only accelerates the production pipeline but also allows for greater experimentation and risk-taking, as the financial and temporal costs of failure are significantly reduced. Generative AI also has the potential to create entirely new forms of interactive entertainment by enabling the procedural generation of content on an unprecedented scale. Imagine a game world that can dynamically generate unique quests, characters, and environments tailored to each player’s preferences and gameplay style, or a narrative engine that can craft compelling storylines and branching paths based on the player’s choices and actions.

The integration of AI does not, however, spell the end for human creatives. Rather, it presents an opportunity for artists to shift their focus. As lower production costs reduce the risk associated with development, developers will be more inclined to experiment with novel ideas, potentially leading to the invention of entirely new game genres. Moreover, the democratisation of game creation tools will enable the rise of ‘micro studios’ consisting of just one or two employees that can nevertheless produce high-quality, commercially viable games. The surprise hit Among Us is a great example of how success is possible, even if you don’t have thousands of employees and a vast budget at your disposal. Developed by a team of just three, the game went on to sell more than 10 million copies on the Steam platform alone.

While concerns have been raised about the impact of AI on job security, some experts argue that these fears are largely unfounded. Just as the introduction of motion capture technology decades ago did not replace animators, AI is expected to serve as a tool to assist creators rather than eliminate them. “While the future may involve more technology, it doesn’t take away the human in the loop”, explains Yves Jacquier, Executive Director at Ubisoft La Forge. “Artists, writers and coders will always be at the heart of the development process, and while AI can now do a better job at assisting creators in their workflow, it’s the artistic vision and perspective of individuals that are essential in the creation of games”.

How game developers are using generative AI

Developers are increasingly using generative AI to reduce the time it takes to create new game assets, such as characters and game environments.

Numerous video games are on the market today that were developed in part with AI. A prime example is the Microsoft Flight Simulator series, which allows players to fly over a photorealistic 3D representation of the entire planet Earth. The 2023 entry in the franchise was developed in collaboration between Microsoft and blackshark.ai, a company specialising in AI-based object detection and 3D digital twin technology. Its AI was trained on vast amounts of 2D satellite imagery, allowing it to generate a shockingly accurate 3D model of the Earth’s surface.

Blizzard Entertainment, the studio behind hit titles like Diablo, World of Warcraft, and Overwatch, is one of the first in the industry to incorporate generative AI into the game development process. The company has developed an internal tool called Blizzard Diffusion, which is trained on assets from its own games. By feeding the AI with a vast library of concept art, character designs, and game environments, its developers can rapidly generate new ideas and variations, significantly reducing the time and effort required to produce new game assets.

It isn’t just visual assets we need to consider, though. With open-world games often requiring hundreds of thousands of lines of dialogue, the task of writing background chatter and minor character interactions can be incredibly daunting. To this end, major gaming publisher Ubisoft—best known for the Assassin’s Creed franchise—has developed a tool called Ghostwriter to assist in the creation of in-game dialogue. Ghostwriter automatically generates first drafts of character dialogues, which scriptwriters can then refine and polish as needed. Given the highly robotic nature of many in-game character interactions, it’s possible gamers won’t even notice the difference.

Learnings

From the silver screen to the recording studio, AI is radically changing the way we tell stories, create worlds, and connect with audiences. We’re witnessing the birth of a new kind of creativity: one that blurs the lines between human imagination and machine intelligence.

• Generative AI can create realistic visuals and compelling narratives at a fraction of the cost and time required by traditional methods.
• It can enable anyone, regardless of their musical background, to create “original” songs at the push of a button.
• Generative AI can also dramatically reduce the time and cost associated with game development.
• However, there are thorny questions about intellectual property rights, job security for human artists, and the very nature of creativity itself.
• Many are worried about AI-generated knock-offs flooding the market and devaluing the work of human creators.

While valid concerns do exist and must be addressed, it’s hard not to feel a sense of excitement about the future of entertainment, where anyone with a story to tell can bring their vision to life without being held back by technical limitations or financial barriers. Just make sure we keep our focus on enabling human creatives—not replacing them.