A Spanish orchestra creates and performs a work made with AI for the first time: “It was a pastiche” | Technology

“Tell him to do a work like Shostakovich, with force, perennial, with metal and percussion instruments, with many dynamic contrasts,” explains Rocío León, artistic sub-delegate of the RTVE Orchestra and Choir. The petition is in favor of artificial intelligence that creates music from text. After a few minutes, the machine returns a melody. Something very metallic begins to sound, then suddenly falls silent and the tone changes: “It sounds like a village orchestra,” says César Peña, member of the association. RTVE Innovation Laboratory. “Now she’s going rock, she’s figured out the metal genre, you can turn it off now,” he adds. We have been hearing for a few minutes: “Do you have the essence of this man (Shostakovich)? » asks Isabella Rocafull, also a member of Lab a León, the artistic sub-delegate. “No,” he answers without hesitation.

This request was only an additional test of the second session to create with AI a work of a few minutes for symphony orchestra. So far, the result was bad. León then tried another route: “Tell him to see how Bach would write a piece for a symphony orchestra in D major for action cartoons,” he asks. The result is also strident: “I don’t like it, it doesn’t have a clear structure, which is fundamental in Bach. There was a moment after the introduction which was a disaster. It seemed like it started well, but then it got muddled again,” says León.

The objective of the Lab and the RTVE Orchestra was to perform for the first time a symphonic piece created by artificial intelligence. EL PAÍS accompanied different stages of the process and spoke with several protagonists. The last two pieces were played at the start of a concert already scheduled at the Monumental Theater on November 17. “For the first time in the history of this theater, we will start with a small experiment and, in the end, we will reveal the mystery,” arranger Borja Arias, who also led the training of these two pieces. The first was transcribed as it was created by the AI, the second was arranged to make more sense. When he was finished, he revealed the trick: “The two works you just heard were composed by artificial intelligence,” he said. The audience then uttered an “Ooohh” long. They were surprised by the novelty, but not impressed by the artwork.

Music with emojis

Lab RTVE has documented the process called “Lab Orquesta” and they have been telling it on television, radio and the web as well as in native pieces for their TikTok channel and other networks since this Tuesday. They looked at various tools for creating music with AI. Only one supported elaborate text requests, which produced inconsistent results. There are others already available that only allow you to describe the query with labels or emojis, and which return something less elaborate.

AI has become a tool for creating text and images since 2022, alongside ChatGPT and other graphics tools such as Dall-E or Midjourney. Musicians have seen this process up close, but in their sector they have not experienced such an explosion: “What we are going to ask the machine is to construct, to the extent that it can and knows how to do so , a musical composition. » explained Manuel Ventero, general director of the Orchestra, at the beginning of the process. “We feel that the result will be, in the worst case, very good,” he added.

But the complexity of classical music poses an even more complicated challenge for the machine: It’s one thing to fill in the background of a YouTube video, another to create a work performed by dozens of musicians. The process showed that, for the moment, AI does not yet have the abilities of Mozart or Bach, or even those of a diligent student of composition: “It does very immature things,” explains Arias. They also tried an artificial intelligence tool to transcribe the music, but it went wrong.

“It’s more Indiana Jones”

Requests (or instructions) orchestra managers and journalists from the RTVE Lab first developed ideas, as is done for example with illustrations. The proposals aspired for the machine to combine composers, genres and instruments. But the AI ​​produced messy sounds, like a puzzle ungovernable. The first day of testing, last September on the stage of the Monumental Theater, already indicated that it was not going to be easy: “It’s more Indiana Jones”, “there are influences here”. reggaetoneras“, “it’s not a leak” or “he doesn’t understand this language”, were some of the comments from this session.

After around thirty tests, those responsible for the project finally settled on a very standard request to obtain an acceptable result: “A piece of music to be orchestrated for a symphony orchestra by a human composer”. The music was mediocre and left composers alone: ​​AI won’t take away their jobs, just yet.

Human work remains essential. The arranger Borja Arias received approximately 20 minutes of AI-created music collected into ten tracks. One transcribed it and another arranged it: “I tried to take what (the machine) wanted to tell me, I took few elements, because there were a lot” , he says. This is how Arias compares what he received from the machine with other arts: “Musical coherence is abstract and difficult to achieve. When you compose, you have to stick to a style. This piece had many contributions and it was unclear where they came from. It was as if he started to tell you the story of a protagonist who is a firefighter and then all of a sudden he continued to tell you the story of a carpenter, then a flower girl and finally ‘a judge,’ he explains.

The RTVE project anticipated that the orchestra musicians would not know the identity of the author until the end. They rehearsed and performed the piece as if it were anonymous. Lab journalists interviewed the orchestra members before they learned the truth. They felt something strange, without knowing what: “I didn’t like the first work (created by AI and only transcribed), but hey, it’s as true as we read it once “, declared trumpeter Borja Antón after the first rehearsal. “Maybe with a little more reading and work we can give it a little more shape, but it’s a bit basic work.” When he was told who had composed the piece, he was not surprised: “A lot of it suits me. The first job was very bad. There was nowhere to take the concept. In the second (arranged by Arias), the melody is more catchy, the structure is better achieved,” he added.

Useful for inspiration

To the artists’ trained ears, it was obvious that something was wrong: “If that doesn’t sound familiar, it might sound like a mix of film scores that then try to be more classical then contemporary, but without be innovative. It’s a pastiche. It wasn’t unpleasant to listen to, it’s true. I imagined that the musicians would think that he was a mediocre composer,” explains León, the artistic subdelegate.

For some reason, AI has a harder time creating something musically meaningful for an orchestra: “There are things that AI doesn’t know are very difficult to play. There’s a high C that stays there for a long time. It’s impossible to do. Physically, this is not possible. And then, there are no different shots, everything happens at once: you hear the melody with the harmony at the same time and you don’t recognize anything,” explains trumpeter Antón.

The difficulty of classical music does not necessarily have to be repeated in other styles. “To compose symphonic music, the truth is that you have a lot left,” says arranger Arias. “It left me quite calm and it also demonstrated the value of this type of music because of its harmonic and contrapuntal complexity and quite a deep message. With other styles, the AI ​​is a little closer to an optimal result. This leaves me calm because there is still room for improvement.

This lack of complexity does not prevent it from also being a useful tool to inspire composers in their work or to imagine less sophisticated works, according to León: “Composers could take it into account as a little help and not find themselves facing the blank page. To give an air of cinematic composition, AI can serve as a filling, atmospheric musical moment. Not for an orchestra, which is something deeper and richer.

You can follow EL PAÍS Technology In Facebook And X or sign up here to receive our weekly newsletter.

_