AI use cases in the entertainment industry Electric Sheep, Mubert, Unreal Engine 5, Descript, Uhmbrella, and My AI
Electric Sheep | Using AI to introduce accuracy and efficiency into the VFX industry
Rotoscoping, an element of visual effects (VFX) where the background of either an image or video is removed so that effects can be added, has traditionally been an intricate and time-consuming endeavour. To make sure to accurately trace around images in the foreground and avoid any accidental overcuts, VFX artists are employed over long hours to manually cut around them frame-by-frame. This obviously eats into production time and budget. There is also scope for human error which, with audiences who are exposed to ever more realistic VFX content at their disposal, could make or break a project.
However, thanks to machine learning technology, specifically generative adversarial networks (GANs) – a subdivision of deep learning, Electric Sheep is working to address these issues. By training its AI on high quality footage via deals with VFX Houses (the video-IP holders), Electric Sheep can rotoscope more accurately than the several humans normally hired to execute the task, and in a significantly shorter span of time (360 or more times faster). Furthermore, as it is machine-powered, the number of foreground obstacles does not affect the speed and accuracy at which the rotoscoping can be delivered, unlike with humans, where the number of objects to rotoscope would add more working hours to the task.
Not only does this allow the industry to deliver higher fidelity VFX in their video content, but it also significantly reduces costs and shortens project timelines. Notably, this means that TV shows and films that are based around a particular event can be released while the event is still relevant and still a talking point among audiences.
Descript | AI powdered tools simplify a podcaster’s workflow
In the attention recession, creators must be well versed in a variety of formats and platforms to grow and retain an audience. A podcaster cannot simply be a podcaster, they must also be a social media expert, a video producer, and a seasoned editor to ensure the growth of the show. With these new responsibilities, creators must spread their limited resources across multiple avenues to develop their show – this is where AI-powered platforms such as Descript come into play.
Descript is an all-in-one podcast and video tool that saves creators essential time in their creative process by simplifying crucial tasks. For instance, Descript’s AI technology removes background noise and echoes from audio files, giving every creator a studio quality show. In the editing process, AI identifies and removes filler words that a creator would have had to spend more time manually removing. Moreover, creators can add words to their recordings using an AI clone of their voice that is powered by text instead of re-recording an entire episode. In addition to podcast creation and editing, Descirpt’s video tools help creators develop and publish promotional clips for social media. Creators can centralise their work on Descript rather than spreading tasks on multiple platforms.
Mubert | Building an AI music ecosystem
AI music generation platform Mubert is building a four-way ecosystem between artists, content creators, brands, and listeners, shaking up the production library industry. In doing so, it provides solutions to several challenges AI brings such as copyright concerns, how to monetise the technology, and where AI music generation leaves human artists.
Most content creators need music for their content, be it a soundtrack for a video or the opening theme of a podcast. Because most cannot afford to licence popular music, they have turned to production music libraries like Epidemic Sound. Mubert is revolutionising this industry by allowing content creators to instantly generate and fine-tune so-called “royalty-free” music in the exact style they need. Brands and developers can do the same on a larger scale, and Mubert’s clients thus far include Adidas and PwC. Mubert monetises this tool via subscriptions, which range from $14 per month for independent creators to $199 for companies generating music for an app, platform, or metaverse.
Mubert does not have to worry about copyright concerns, because it pays real artists and producers to contribute to its dataset via its Mubert Studio arm. The model is similar to that of royalty-free production libraries. Mubert buys tracks, samples, and loops from artists and producers, and pays them a revenue share based on how their contributions are used. By covering its copyright bases, Mubert is able to commercialise its model faster, while also providing a new revenue stream to human artists.
Mubert’s fourth and final division is its free music streaming app, Mubert Play. Mubert Play specialises in mood music, providing music for sleep, focus, workouts, and meditation. Each music “stream” is unique — listeners will never hear the same thing twice. The ecosystem continues here, as Mubert Play users can save anything they especially like as a one-minute track to be used in content creation.
My AI | How Snapchat is throwing its chatbot into the generative-AI ring
Snapchat has decided to launch its own generative-AI chatbot onto its platform. Following the hype around OpenAI’s ChatGPT it was only a matter of time before other companies decided to utilise its invite-only developer platform, Foundry, which allows enterprises to run its newest machine-learning model, GPT-3.5. While Snapchat has taken this step into incorporating AI into its platform, it is currently only available to its Snapchat+ users in the US, who pay $3.99 per month – which is cheaper than the current $20 ChatGPT Pro subscription – although it offers its features in a much smaller capacity.
Rather than being presented as a search engine where users seek out answers to prompts as and when, Snapchat’s chatbot My AI is displayed as a friend in the direct messages section of the app (at the top of the page for easy access). Once users add My AI as a friend, they can treat it like any other connection on the app and start a conversation right away. For further personalisation, users can give their AI a name and customise their chat wallpaper.
Unlike ChatGPT, Snapchat’s My AI is restricted in responses to reflect Snap’s trust and safety guidelines. Understanding its age demographic, Snap has prevented My AI from responding to or with anything of a sexual, violent, or political nature, in addition to swearing or writing school assignments. Snap, however, is conscious of the newness of this feature, and has included a disclaimer that urges users to be aware that the bot may “hallucinate”, and to report any issues. The company is counting on the feedback from this select group of users to improve the feature before it is officially rolled out to the rest of its consumers.
This is just the beginning of AI being implemented into social media platforms and we should expect a lot more as machine learning continues to develop. Generative-AI chatbots are set to become normalised in the very near future.
Unreal Engine 5 | Building games worlds with AI
AI is poised to exponentially expand the capabilities of more than just games developers and audiences. Unreal Engine 5 is one of the first of what will likely be many companies to build on this opportunity.
The ‘real time 3D creation tool’ allows users (be they games developers, casual creators, or even the likes of city planners and architects) to generate entire virtual worlds that are high-definition, adaptive, and scalable. In short, it used to take teams of developer’s months to build environments with limited functionality. Now, a single person can use AI to generate fully interactive, expanding worlds virtually overnight.
Especially as games move into lower-margin streaming subscription revenue models, the capability AI will bring to reducing manpower and time required to build games will be hugely important. Moreover, the quality of games can improve as supportive AI functionality can reduce the likelihood of errors, improve render quality and fidelity, and bring expanded functionality to the digital environments through interactive elements. This means more hours of play and more creative control in-game for players (functionality that still drives the popularity of Minecraft).
While visual graphics may improve, the risk is the to the quality of storytelling. Many games are carefully designed so that every minute of play contributes to a broader story arc within the game – even side quests explore aspects of the broader world the game is set in, often contributing in some way to add value to the the main storyline (like Red Dead Redemption, The Witcher, and The Last of Us). Opening these gaming worlds too wide, with artificial creation taking away this more deliberate attention to detail, will mean that detail becomes overwhelming (but useless), rather than deliberate (and contributory).
Games companies will undoubtedly take both the good and the bad on board, using AI as a tool to expand its capabilities – much as music and video have been empowered and challenged by digital editing tools and CGI, respectively.
Uhmbrella | Answering the rights question
Rights attribution is going to be one of the key struggles for AI and AI-empowered entertainment industries in the coming months. The lawsuits surrounding image generators created by MidJourney, DeviantArt, and StabilityAI are already calling into question who should be remunerated for casual usage – much less commercial usage – of content generated by AI.
The existing terms and conditions for OpenAI, for example, broadly state that the end user is responsible – and thus owns – the content they generate. However, OpenAI has also begun to monetise use of DALL-E through the purchases of credits, a model which other platforms are likely to follow. Meanwhile, the creators responsible for the works populating the datasets these generators are drawing from have no way of getting attribution, much less remuneration, for the use of their works. This complex issue of monetisation and remuneration is going to make it difficult to ensure safe commercial usage and the legal grounds for charging for use of the generators. The second-order impacts of this will be to make the next rush in the space one for who has the ‘cleanest’, rights-cleared datasets. However, the implications will play out differently in the different entertainment industries. While a battle for datasets may settle commercial usage it will not sort out the problems of rampant user-generated content on social platforms (which also provide creator funds, complicating things further).
Games companies are likely to be able to use their own datasets, relying instead on the likes of Unreal Engine 5 to build out their ideas in-house. Video has close hold of its IP, allowing companies like Electric Sheep to make deals with video IP holders to train their AIs with high-quality data that can then be quality controlled and used at a high-end commercialised level. Music, however, will see the issues it has already seen from streaming – low per-stream rates, unaccredited social platform use, and a shift to collaborative music creation on social platforms – exacerbated by AI.
Uhmbrella is one company working to address this gap in the space. Essentially, it offers a plug-in to allow artists to stamp the different elements of their tracks on the blockchain as they are created. This means that from creation through to distribution, remixing, social co-creation, sampling, and beyond, the elements can all be easily traced back to the original creator (in theory; the product still is not live yet). This offers a use for NFTs and blockchain in helping to solve the broader rights questions that will become ever-more pressing for creators and rightsholders alike as AI makes ownership, re-creation, and attribution a more amorphous issue.
Roles
This report is relevant to the following roles:
Cultural trends