Real virtuality – a quantum leap

The interactive dance film “VR_I” by choreographer Gilles Jobin allows five people to move and act freely within virtual surroundings. In 3D and across 360°, he puts the audience members centre-stage: They become avatars, who can interact and even talk to each other simultaneously with the 15-minute performance. The Geneva-based artist’s work has met an enthusiastic response and is enjoying international success.

The news seems nothing out of the ordinary: Geneva-based choreographer Gilles Jobin is taking his interactive piece “VR_I” on a tour of the Far East, stopping at Shanghai, Seoul, Tokyo, Singapore … Nothing out of the ordinary? In this case, it is. A successful tour of the Far East is not just good for revenue. Crucially, it shows that this European creation in virtual reality can even impress audiences in the heartlands of technology. So far, over 12,000 people in 30 cities on four continents have seen the work, the compagnie announced in April 2019. Not only that, “VR_I” is the most extensively touring work of virtual reality in the world – even though only five people at a time can watch and take part in the work with the rather sober, workaday title. The charm of “VR_I” lies in the opportunity it opens to immerse yourself so deeply in a virtual world that you become part of it.

The five dancers of the Compagnie Gilles Jobin, including the boss himself, meet their avatars from the audience in several different 360° landscapes that fully surround the visitors. Starting in a cave, the dramaturgy leads out into a rusty-coloured desert landscape. Later, we see how five giants construct an extremely futuristic, glass villa here. The rooms and terraces of the villa provide perfect stages for choreographies performed by the virtual quintet (Susana Panadés Diaz, Victoria Chiu, Diya Naidu,Tidiani N’Diaye and Gilles Jobin).

At one point, the virtual Jobin grows to five metres high and seems to watch the other, smaller compagnie members as If they were quaint little toys. Then we see the dancers in life-size again before becoming giants ourselves – the dancers now only appear on the scene as dwarfs. Suddenly, the giants reappear and dismantle the walls. Now we are standing outdoors and looking at an urban cityscape, which could be straight out of an estate agent’s advertising brochure. The five compagnie members are dancing in a park, restored to their original sizes, and almost seem as if they were performing for us.

Virtual self-knowledge

Jobin is thrilled with the new experience: “The way it plays with relative sizes intensifies the effect of the choreography and chimes with every age group, from children to grandparents.” While we watch all this with interest, throughout the piece we also see other figures standing or walking around, sitting on the floor or even dancing themselves. They are the four other people wearing VR headsets. We don’t see them as they are in real life but as virtual figures. But we know that each of us sees the other four in their authentic locations, positions and gestures. And if you look down at your own body you get an idea of how the others see you. That can be quite a surprise, as your avatar’s shape, clothing and gender can contrast starkly with the real you. With the avatar identities transferred on to them in real time via computer, the participants enjoy complete freedom of movement, despite being immersed in the virtual world. It is precisely this that is so fascinating about “VR_I”, and marks it out from other 3D spectacles, where the audience either sits or stands and can only move their heads to experience more.

A performance, a game or a film?

Just how far “VR_I” is at the vanguard of new developments is illustrated by the lack of a term to denote this virtual and participatory video extravaganza. Performance? Enactment? Nobody is facing us live on stage. But it is not a video game either, although the aesthetics are reminiscent of one.

And how best to describe the five members of the public who experience Jobin’s work? They are not a passive audience nor necessarily participants or visitors to an exhibition but a bit of all three, as they are simultaneously passive and active, at least if they want to be.

If you attend a showing of “VR_I” with friends, you can talk to them and even touch them during the experience. The difficulty of defining the visitors’ status arises precisely from the fact that their actions and interrelations are barely distinguishable from those in real life. And any reference to a stage or a theatre is also irrelevant, precisely because the virtual space puts the audience members’ avatars on stage and in the middle of the action.

Linking art and science

During the 15-minute performance, Jobin and two technicians sit at the edge of the space and control the effects. What makes a choreographer create a work at the forefront of technology? Collaboration. “VR_I” is an excellent example of a fruitful cooperation between art, science and industry. The technology for “VR_I” was developed by Artanim, a private research institute based in Geneva (like Jobin). Founded in 2011 by Sylvain Chagué, Caecilia Charbonnier and Clementine Lo, the self-named Motion Capture Center based the concept on findings from its own medical research.

Both branches of Artanim’s research are concerned with the body. Their main commission, in cooperation with hospitals and surgeons, is developing artificial joints in 3D animations. Virtual models help to better understand the physiological processes triggered in the joints by the body’s movements. But they can also be used to set fictional figures in motion in virtual reality. And this led them to develop a project entitled “Real Virtuality”, which can be used in medical contexts, games, virtual viewings, fun parks and various other ways.

VR technology with full mobility

The technology package developed by Artanim consists of a motion capture system, computer programmes, algorithms and 3D scans of the dancers. Jobin and his team created the virtual surroundings, which were then integrated in the Artanim platform. The motion capture system is based on a total of 16 infrared cameras (Vicon Vero 2.2) which cover an area of 8 x 5.5 m. The 3 m-high support trusses span an area of 10 x 7.5 m. For the motion capture to work, every point on the playing space must be captured by at least two cameras.

To calculate the participants’ movement profiles, each one gets four Vicon markers strapped to their ankles and wrists. They are then also equipped with a backpack containing a portable VR PC (Zotac VR Go). This is connected to a server that calculates the wearer’s position in the space and transfers the data for the avatar on to it, integrating it in the virtual surroundings and putting its image on the VR headset (Oculus CV1). More markers are affixed to the backpacks and the 3D headsets. The server works with the programmes Unity 3D and 3D Studio Max.

According to Artanim, the technological development will steadily continue. In one or two years it will be possible to do away with the backpacks, allowing the visitors to move even more freely in the virtual world. But even now, the small backpacks are no heavy burden and you soon forget you are carrying it. But this will of course make even more extensive tours possible: “We only need three technicians and the material fits into seven 32 kg flight cases. The production costs are no higher than for a conventional duo or trio by my company and we have 95 to 98 percent capacity ticket sales,” Jobin is delighted to report. It seems as if the fun has only just started.