Artificial intelligence dances flamenco

Rhythms and algorithms: A Japanese-European cooperation is presenting dance at the vanguard of artificial intelligence use. The perhaps surprising protagonist of the project is choreographer and flamenco dancer Israel Galván, the maverick avantgarde star from Seville.

On stage, Israel Galván hammers out his zapateado (foot drumming) or scrapes his feet in black resin, creating a kind of noise music. He is a composer and a musician at the same time, and sometimes he acts like an orchestra conductor. But the ‘musicians’ don’t always follow his lead.

He engages in a dialogue with the receivers of his signals, which include a wooden construction he calls Beti - a block of wood on four ‘paws’ that make it hop. Another consists of four vibrating poles that sometimes hop, too. Yet another unit is made of four upright instruments with a positively cubist appearance, hammering out their rhythms. Alongside these three very abstract units, there are flamenco shoes on the stage. All four units alternately respond to Galván’s signals, having received them indirectly via artificial intelligence (AI). They dance, and sometimes they dance to a different drummer. There is the red shoe, a reference to an earlier work by Galván, like a ghost of itself, answering him. It has weights inside so that it doesn’t tip over when drumming its heel. This shoe occasionally starts a dialogue with another, beige-coloured pair. It is one of the defining features of “Israel & イスラエル”, or “Israel & Israel”, that the high-tech is concealed behind the low-tech.

Live music goes with flamenco productions at contemporary arts venues like sangria with tapas. But recently the Spanish flamenco dancer and choreographer Israel Galván has embraced the less likely bedfellows, Japanese labs and software engineers, and started bringing the two poles ingeniously together. In 2018 he travelled to Tokyo for the first time and met the engineers from the Yamaguchi Center for Arts and Media (YCAM) and the Qosmo agency. The outcome of their cooperation was a performance that is not a flamenco piece, Galván insists, but a research work.

Galván’s agent is French and convinced the Maison de la Culture du Japon à Paris to help set up the ocean-spanning cooperation. But the idea came from Japan, where flamenco is booming. The producer at YCAM, Akiko Takeshita, is an admirer of Galván’s. He doesn’t always perform with traditional music ensembles consisting of a singer, guitarist and percussionist, but has been known to invite a heavy metal group to appear on stage with him, a contemporary pianist, a Japanese Butoh dancer and even circus family cats. In other words, he is always on the lookout for new encounters to redefine the boundaries of his art.

Technology and handiwork

It is understandable if people have reservations about artificial intelligence, and for the choreographer, AI is more unpredictable than a flesh-and-blood partner. Yet it contains a large human element, which unfolds its impact when “Israel & イスラエル” is performed live. The developer Tokui taught the artificial intelligence flamenco by feeding the model hour upon hour of Galván’s step sequences. The extreme precision of the virtuoso soloist’s heel-kicks make him the ideal protagonist of the project led by YCAM and Qosmo. The hardware was developed at the YCAM under the aegis of Owaki, whose first attempt to equip Galván’s flamenco shoes with piezoelectric sensors was a crashing failure. The sensors smashed after three seconds as Galván’s heels and feet drummed so hard against the floor. If they had simply installed microphones it wouldn’t have happened. But they wanted to analyse the dance and not just the acoustics. So, Owaki developed more robust sensors. Today he has two different shoe versions – one for Galván’s work in the lab in Tokyo and one for use on stage. While the latter are simplified, the lab shoes are equipped with a complex system of sensors that not only analyses the signals and the speed at which they are sent but also the tilt angles and moments of contact between the feet and the floor.

“Luckily, we don’t need to use the hypersensitive system for the performances, as the AI has already saved all Galván’s typical patterns,” explains Owaki. Galván’s stage shoes, then, are ‘only’ fitted with four sensors. They register his zapateado and send the signals to the computer. To what extent the AI deviates from Galván’s patterns and so challenges his reactions and scope for creativity depends on how much autonomy Owaki concedes the AI.

In some scenes the model copies the dancer while in others it deviates from the input. Owaki can tighten the reins on the AI or let them loose. “I change the randomization settings during the performance. So, there is handiwork involved,” he remarks with amusement. In the footloose, creative sequences, Galván says he gets envious of the AI’s capabilities. But envy is probably not quite the right term for the feeling. The sting is deeper. Despite his avantgarde attitude, Galván remains an advocate of traditional handiwork, or rather footwork. That applies especially to his shoes – they are as sacred to a flamenco dancer as soccer boots are to a footballer.

Dancing phenomena

Altogether, three pairs of shoes appear on the stage. Only one of them are on Galván’s feet. Those are fitted with two piezo technology sensors each. While Galván dances, the data travels via MaxMSP to the AI, for which Tokui used Python as the programming language, to get from Machine Learning to the level of Deep Learning. To rule out any errors in the AI’s data analysis, he also uses a programme that regulates sound effects in cinematic productions. During the performance he directly controls the musical signal sequences using Ableton Live sequencer software. For data processing on tour, he only needs one single, sufficiently fast graphics processor.

To learn Galván’s flamenco identity, on the other hand, he used a configuration he devised himself, made of several Nvidia processors. He needed two days to develop the model, he says. The AI responds to the live-generated input by answering, again via MaxMSP, through the apparatuses placed on the stage, who address Galván in a ping-pong exchange of input and output, back-to-back, each in its own way.

https://www.der-theaterverlag.de/On the stage it might look a lot like craft; the quirky, abstract constructions are deliberately designed to lend the AI a ‘cute’ appearance and trigger familiar emotions. But the next or last stage is already foreseeable, with the growing use of holograms on the music scene. Should Galván one day agree to have his physical appearance and gestures fed into AI, his image could continue creating ever more Galván solos. Into eternity.