The art of illusion
This week I had the pleasure of working with the Palindrome dance company for a performance at Birmingham Conservatoire. The company is led by Robert Wechsler, who has spent 10 years researching and developing the EyeCon video motion tracking platform in collaboration with developer Friedrich Weiss.
As part of the performance, the group gave an hour-long workshop in which they talked about their techniques and how they were used. I was particularly interested to hear about their use of motion tracking since Palindrome have developed some of the most effective work I have seen, apparently achieving extremely accurate synchronisation between movement (gesture) and sound (music). The sync seems accurate in terms of both spatial and temporal localisation, for example dancers can trigger sonic events with tiny movements of their fingers or eyes even at some distance from the camera.
I use words like ‘seems accurate’ and ‘apparently’, because it became evident from the workshop that unlike many motion capture systems, EyeCon doesn’t work by giving accurate data about the dancer’s location in 3D space, neither does it capture data about the position of individual body parts. Instead it takes a 2D window of the space and tells us where the dancer is in that window, whether they are intersecting a marked area, the amount of movement within that area and degree of left-right symmetry. It can’t tell us the dancer’s absolute height, but it can tell us if they appear bigger or smaller within the window.
The word ‘appearance’ is critical here because in many regards EyeCon and the Palindrome approach is more about how things appear than how they are. In the following video clip, for example, it appears as though the motion capture system is accurately tracking the movements of eyes and mouth. In reality, it has nothing like that level of detail. Instead it knows roughly where movement is occurring, and the intensity of that movement. The appearance of eye movement controlling sound is an illusion, effectively created through the correlation of movement, motion sensing and audio triggering. We see the eyes move and we hear a sound, so our brains assume that there is some connection between eye movement and sound production. This may be the case, but often the system will just wait for any movement within a given window region and then react.
A Human Conversation from Jamie Bullock on Vimeo.
This approaches something I have used in my own work for a long time, and which I believe very strongly, which is that the audience’s perception of what happens is far more important than what actually happens. The obvious analogy in the world of live electronic music is score following. State of the art score following typically uses a stored model of the musical work (a representation of the score), and tries to match incoming to this model thereby giving a precise temporal location within the score as output. This approach is OK for music where a high degree of reproducibility is required, but personally I’m more interested in creating a more organic sense of interaction for the performer and audience. That is, I’m not so interested in where the performer is within a score, but rather where they are in the space of musical possibilities.
So as live electronics musicians, what can we learn from Palindrome and their approach? I think the key for me is that we need to ‘think smarter’ about how we use our technical resources in order to create synergies between art and technology where the whole is greater than the sum of constituent parts. We need to always be aware of perception and how the human mind always tries to connect cause and effect. For me this is an interesting space to play with both technically and artistically.
