Google’s New AI Can Make Music From Just a Few Words

Google has developed an impressive new AI system, MusicLM, that can generate music in any genre from a text description. However, the company has no plans to release it due to potential risks.

While other AI systems for composing music have been developed, such as Riffusion, Dance Diffusion, Google’s AudioML, and OpenAI’s Jukebox, MusicLM is unique in its ability to create complex and high-fidelity songs due to its advanced technology and extensive training data.

MusicLM, as described in an academic paper, was trained using a dataset of 280,000 hours of music. This training allows it to generate coherent songs based on descriptions of “significant complexity” such as “enchanting jazz song with a memorable saxophone solo and a solo singer” or “Berlin ’90s techno with a low bass and strong kick.”

The resulting songs are similar to those created by human artists, although they may not be as creative or musically cohesive. The samples produced by MusicLM are highly impressive, considering that it does not involve human musicians or instrumentalists.

Here is a sample. https://twitter.com/i/status/1619031717834272769

Even with detailed and complex descriptions, MusicLM is able to capture nuances like instrumental riffs, melodies, and moods.

MusicLM’s capabilities extend beyond just generating short clips of songs. Google researchers have demonstrated that the system can build on existing melodies, whether they are hummed, sung, whistled, or played on an instrument.

Additionally, MusicLM can take several descriptions written in sequence, such as “time to meditate,” “time to wake up,” “time to run,” or “time to give 100%,” and create a melodic “story” or narrative that can range up to several minutes in length, making it ideal for movie soundtracks.

Here are a few more samples. https://twitter.com/i/status/1618946445629071361

MusicLM can also be instructed through a combination of pictures and captions, or generate audio that is “played” by a specific type of instrument in a particular genre. The AI “musician’s” experience level can also be set and the system can create music inspired by places, epochs, or requirements, like motivational music for workouts.

Still, as mentioned earlier, Google does not plan to release MusicLM to the public due to the risks and challenges posed by such a system. For instance, it has the potential to make copyrighted material from the training data it received.

The creators wrote:

We acknowledge the risk of potential misappropriation of creative content associated to the use case. We strongly emphasize the need for more future work in tackling these risks associated to music generation.